From stuart.marks at oracle.com  Sat Oct  1 00:05:12 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 30 Sep 2016 17:05:12 -0700
Subject: Collection Design FAQ
In-Reply-To: <CAPs6150QMAD6enMAh+hb==3tFkmziCN3qiiyg+3ZuWpxcdPfww@mail.gmail.com>
References: <CAPs6150QMAD6enMAh+hb==3tFkmziCN3qiiyg+3ZuWpxcdPfww@mail.gmail.com>
Message-ID: <14237bba-e07f-5977-b088-9beb27c1ba2a@oracle.com>



On 9/22/16 11:13 PM, Kasper Nielsen wrote:
> I accidently bumped into an old friend today
> https://docs.oracle.com/javase/8/docs/technotes/guides/collections/designfaq.html
> There are couple of questions that might need an update after Java 8. Such
> as
>
> Why don't you provide an "apply" method in Collection to apply a given
> method ("upcall") to all the elements of the Collection?
> Why didn't you provide a "Predicate" interface, and related methods (e.g.,
> a method to find the first element in the Collection satisfying the
> predicate)?

Hi Kasper,

The collection design FAQ is certainly out of date.

Regarding an "apply" method, as R?mi mentioned, there is Iterable.forEach. For 
List, there is also List.replaceAll.

It was certainly possible to have a Predicate interface before Java 8, but the 
only way to use it would have been to use an anonymous inner class (or a 
pre-existing class that implemented it). This would have been pretty cumbersome 
to use. Of course, this is all different now that we have lambdas.

It's pretty straightforward to use streams to find the first matching element of 
a List, but less so to find the index of that element, like List.indexOf or 
List.lastIndexOf. Would it be useful if something like the following were added 
to List?

     int findIndexMatching(Predicate<E>)
     int findLastIndexMatching(Predicate<E>)

(doesn't have to be these names, of course)

Or is it sufficient to stream over the list indexes and search that way?

     IntStream.range(0, list.size())
              .filter(i -> predicate.test(list.get(i)))
              .findFirst();

s'marks

From stuart.marks at oracle.com  Sat Oct  1 00:31:37 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 30 Sep 2016 17:31:37 -0700
Subject: RFR: 8166840: Synthetic bridge constructor in ArrayList$Itr
	blocks inlining
In-Reply-To: <8454a098-c8f1-bec7-8131-20ce8686080c@oracle.com>
References: <8454a098-c8f1-bec7-8131-20ce8686080c@oracle.com>
Message-ID: <61b67377-8c22-13f3-fb94-fbb2944e8852@oracle.com>



On 9/28/16 4:48 AM, Claes Redestad wrote:
> as discussed recently on hotspot-compiler-dev[1], having a private class with no
> default constructor can lead to C2 failing to inline, due to the synthetic
> bridge constructor using a dummy argument of an uninitialized class. This is
> really a problem in C2, as well as something which could ultimately be resolved
> by nestmates...
>
> However, there is an easy workaround in adding an empty package-private
> constructor. In the most recently found case - a microbenchmark stressing
> MethodHandles.iteratedLoop - adding this to ArrayList$Itr lead to a 2.5-3x speedup.
>
> This is me asking nicely to do a quick-fix for this in java.util.ArrayList$Itr now:
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8166840
> Webrev: http://cr.openjdk.java.net/~redestad/8166840/webrev.01/
>
> [1]
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2016-September/024407.html

Hi, adding the package-access constructor seems like a reasonable thing to do.

However, should there be a comment explaining this? It seems like just having a 
no-op constructor is something that is liable to be cleaned up by a well-meaning 
individual who didn't happen to see this thread. Something like this:

     // avoid creation of synthetic class and bridge constructor
     Itr() {}

If this really is a problem in C2, should there also be a reference to a Hotspot 
bug that represents this problem?

On the other hand, addition of the constructor does remove a class from the 
runtime image, so maybe it's worth it to keep around even if the Hotspot bug is 
fixed. But having a comment would be good to prevent this code from being 
"cleaned up".

s'marks

From martinrb at google.com  Sat Oct  1 00:41:08 2016
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 30 Sep 2016 17:41:08 -0700
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of ArrayList
	for tracking XML IDs
Message-ID: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>

https://bugs.openjdk.java.net/browse/JDK-8167002
http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/

From matcdac at gmail.com  Sat Oct  1 11:39:42 2016
From: matcdac at gmail.com (Prakhar Makhija)
Date: Sat, 1 Oct 2016 17:09:42 +0530
Subject: ParallelStream Vs Stream Digest, Vol 113, Issue 94
In-Reply-To: <236b4ffe-91de-f6ad-534b-e084d4914f53@oracle.com>
References: <CAPu805pkvE8XLE784ffOnVw+mkRorb2i7H7QCzWPQ--BhwbP8w@mail.gmail.com>
	<93fb0f8d-54a9-a2ba-5799-32c2cd7edf2e@oracle.com>
	<CAPu805qJNitW_hSoiZxC1dUCD8wvMWsHhjjtVCYXsdaSKWRBfA@mail.gmail.com>
	<CA+tXMXOvJLTLWS3GPzjDGrmrw8YJvdTXJDs=NEK0KW+3Cx3guw@mail.gmail.com>
	<F5144BC4-151D-4C58-A492-7722310FCA4B@oracle.com>
	<236b4ffe-91de-f6ad-534b-e084d4914f53@oracle.com>
Message-ID: <CAPu805o_ONW0DfJ0zy5TP3t=nnMugkBWhH8j+tuD3up6y5b1Rw@mail.gmail.com>

Thanks for the material. I was looking forward to watch/read more about it
in detail.
On Oct 1, 2016 5:21 AM, "Stuart Marks" <stuart.marks at oracle.com> wrote:

>
>
> On 9/30/16 12:41 PM, John Rose wrote:
>
>> On Sep 29, 2016, at 11:39 PM, Louis Wasserman <wasserman.louis at gmail.com>
>> wrote:
>>
>>>
>>> You should absolutely not assume parallel streams are faster than
>>> sequential streams.
>>> http://gee.cs.oswego.edu/dl/html/StreamParallelGuidance.html <
>>> http://gee.cs.oswego.edu/dl/html/StreamParallelGuidance.html> is pretty
>>> much
>>> the iconic document on that subject, and explains circumstances under
>>> which
>>> parallelism is good, and when it is likely to be harmful.
>>>
>>
>> Stuart Marks and Brian Goetz gave an excellent talk on this at JavaOne
>> last week.
>>
>> You can view it here.  The talk is called "Thinking in Parallel".
>> https://youtu.be/WSxKI5S8j90?t=8h51m34s <https://youtu.be/WSxKI5S8j90?
>> t=8h51m34s>
>>
>> InfoQ wrote a useful summary of the talk, for those with no time for
>> video:
>> https://www.infoq.com/news/2016/09/JavaOne-2016-parallel-streams <
>> https://www.infoq.com/news/2016/09/JavaOne-2016-parallel-streams>
>>
>> But the video is very enjoyable, even if you think you know what they are
>> going to say.
>>
>> ? John
>>
>> P.S.  One influence of this year's presentation is an epic manifesto by
>> Guy Steele in 2010.
>> https://www.infoq.com/presentations/Thinking-Parallel-Programming <
>> https://www.infoq.com/presentations/Thinking-Parallel-Programming>
>>
>
>
> John, thanks for the plug!
>
> If you just want to look at the slides, they're linked from this post on
> my blog:
>
> https://stuartmarks.wordpress.com/2016/09/23/javaone-2016-sessions/
>
> s'marks
>

From claes.redestad at oracle.com  Sat Oct  1 12:45:31 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Sat, 1 Oct 2016 14:45:31 +0200
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
Message-ID: <57EFAFEB.6010006@oracle.com>


On 2016-10-01 02:41, Martin Buchholz wrote:
> https://bugs.openjdk.java.net/browse/JDK-8167002
> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/
>

+1, but I have to ask what the intended benefit of writing:

   ((fIds != null) ? fIds : (fIds = new HashSet<>())).add(name);

rather than keeping the pre-existing pattern:

   if (fIds == null) fIds = new HashSet<>();
   fIds.add(name);

If this is about bytecode optimization to help with inlining or such,
the latter actually generate a more compact method (14 vs 16 bytecodes).

/Claes

From lance.andersen at oracle.com  Sat Oct  1 16:42:55 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Sat, 1 Oct 2016 12:42:55 -0400
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
Message-ID: <FA5B8910-87C5-4DCA-B848-A86C872779B5@oracle.com>

This looks fine to me also


> On Sep 30, 2016, at 8:41 PM, Martin Buchholz <martinrb at google.com> wrote:
> 
> https://bugs.openjdk.java.net/browse/JDK-8167002
> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From martinrb at google.com  Sat Oct  1 16:54:39 2016
From: martinrb at google.com (Martin Buchholz)
Date: Sat, 1 Oct 2016 09:54:39 -0700
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <57EFAFEB.6010006@oracle.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
	<57EFAFEB.6010006@oracle.com>
Message-ID: <CA+kOe0-LcMPp5Xu9Fcs4b7vi1eCvU-KmdPAK_sRessqwfQVdMg@mail.gmail.com>

On Sat, Oct 1, 2016 at 5:45 AM, Claes Redestad <claes.redestad at oracle.com>
wrote:

>
> On 2016-10-01 02:41, Martin Buchholz wrote:
>
>> https://bugs.openjdk.java.net/browse/JDK-8167002
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-v
>> alidation-by-hash/
>>
>>
> +1, but I have to ask what the intended benefit of writing:
>
>   ((fIds != null) ? fIds : (fIds = new HashSet<>())).add(name);
>
> rather than keeping the pre-existing pattern:
>
>   if (fIds == null) fIds = new HashSet<>();
>   fIds.add(name);
>
> If this is about bytecode optimization to help with inlining or such,
> the latter actually generate a more compact method (14 vs 16 bytecodes).


The intent was (by instinct) to not code a re-read of a field, but ... yeah
.... that wasn't achieved.
That could be done correctly using

    public void addId(String name) {
        HashSet<String> x;
        if ((x = fIds) == null) fIds = x = new HashSet<>();
        x.add(name);
    }

but I'll just revert to
        if (fIds == null) fIds = new HashSet<>();
        fIds.add(name);

From claes.redestad at oracle.com  Sat Oct  1 18:07:19 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Sat, 1 Oct 2016 20:07:19 +0200
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <CA+kOe0-LcMPp5Xu9Fcs4b7vi1eCvU-KmdPAK_sRessqwfQVdMg@mail.gmail.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
	<57EFAFEB.6010006@oracle.com>
	<CA+kOe0-LcMPp5Xu9Fcs4b7vi1eCvU-KmdPAK_sRessqwfQVdMg@mail.gmail.com>
Message-ID: <57EFFB57.3080606@oracle.com>



On 2016-10-01 18:54, Martin Buchholz wrote:
>
> On Sat, Oct 1, 2016 at 5:45 AM, Claes Redestad
> <claes.redestad at oracle.com <mailto:claes.redestad at oracle.com>> wrote:
>
>
>     On 2016-10-01 02:41, Martin Buchholz wrote:
>
>         https://bugs.openjdk.java.net/browse/JDK-8167002
>         <https://bugs.openjdk.java.net/browse/JDK-8167002>
>         http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/
>         <http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/>
>
>
>     +1, but I have to ask what the intended benefit of writing:
>
>        ((fIds != null) ? fIds : (fIds = new HashSet<>())).add(name);
>
>     rather than keeping the pre-existing pattern:
>
>        if (fIds == null) fIds = new HashSet<>();
>        fIds.add(name);
>
>     If this is about bytecode optimization to help with inlining or such,
>     the latter actually generate a more compact method (14 vs 16 bytecodes).
>
>
> The intent was (by instinct) to not code a re-read of a field, but ...
> yeah .... that wasn't achieved.
> That could be done correctly using
>
>      public void addId(String name) {
>          HashSet<String> x;
>          if ((x = fIds) == null) fIds = x = new HashSet<>();
>          x.add(name);
>      }
>
> but I'll just revert to
>          if (fIds == null) fIds = new HashSet<>();
>          fIds.add(name);
>

Ok, thanks for the explanation!

From peter.levart at gmail.com  Sun Oct  2 21:51:39 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 2 Oct 2016 23:51:39 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
Message-ID: <bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>

Hi,

I added an exhaustive jtreg test that covers all possible situations.

 From the set of the following classes:

package a;
public class PublicSuper {...}

package a;
class Package {...}

package b;
public class PublicSub extends a.PublicSuper {...}

package b;
class Package {...}

it creates a set of all possible triplets:

(currentClass, memberClass, targetClass)

where:

currentClass - the class making the reflective access
memberClass - the member's declaring class
targetClass - the target object's class (for accessing instance fields 
and methods - must be equal to or subclass of memberClass)

For each such triplet it checks the reflective access to each of the 
following members:

{private, package, protected, public} x {instance, static} x {field, method}

and:

  {private, package, protected, public} x {constructor}

When running on unpatched build 9-ea+137, the result is:

http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/AccessControlTest_unpatched.jtr

When running on patched build of 9, the result is:

http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/AccessControlTest_patched.jtr


The difference is exactly in the following two cases which fail for 
unpatched and are fixed by the patch:

b.PublicSub accessing a.PublicSuper's PROTECTED_STATIC_FIELD - expected 
allowed, actual denied : FAILURE
b.PublicSub accessing a.PublicSuper's PROTECTED_STATIC_METHOD - expected 
allowed, actual denied : FAILURE

QED.


This is new webrev:

http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.02/

I think the test proves the effect of the patch is as intended, 
therefore it should not be a problem to review it.


Regards, Peter


On 10/01/2016 12:20 AM, Peter Levart wrote:
> Hi,
>
> I have a fix for a 10 year old bug (JDK-6378384 
> <https://bugs.openjdk.java.net/browse/JDK-6378384>). It was marked as 
> a duplicate of a 19 year old bug (JDK-4032740 
> <https://bugs.openjdk.java.net/browse/JDK-4032740>) which is marked as 
> a duplicate of a 17 year old bug (JDK-4283544 
> <https://bugs.openjdk.java.net/browse/JDK-4283544>) which is still 
> open. But this bug is not a strict duplicate. This bug only concerns 
> reflective access to protected members.
>
> Here's a proposed fix:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.01/
>
>
> The bug manifests itself as not being able to access protected static 
> methods or fields from a subclass located in a different package. 
> Instance protected methods and fields can be accessed, and using an 
> undocumented trick, also static methods and fields, but the trick is 
> very subtle. The specification for Field.get/set and Method.invoke 
> says, respectively:
>
>      * <p>If the underlying field is a static field, the {@code obj} 
> argument
>      * is ignored; it may be null.
>
> and:
>
>      * <p>If the underlying method is static, then the specified 
> {@code obj}
>      * argument is ignored. It may be null.
>
> Well, it is not exactly so! The obj argument is used as a 'target' 
> even for protected static members and it is ensured that its class is 
> equal or a subclass of the class that accesses the member. So if you 
> pass an instance of a subclass of the protected method's declaring 
> class into the get/set/invoke, you can access the static protected 
> member. If you pass 'null', you get IllegalAccessException.
>
> The problem is in the design of 
> jdk.internal.reflect.Reflection#ensureMemberAccess method which is 
> used to check reflective access. It takes an Object 'target' argument, 
> which is supposed to be null when accessing static methods/fields and 
> it is null also when accessing constructors. Because of constructors 
> and the method's API, it has to be overly restrictive as it must only 
> allow calling protected constructors from within the constructor's 
> declaring class itself or same package, while protected static methods 
> could be called from any subclass.
>
> By redesigning the API of this method, replacing Object 'target' 
> parameter with Class<?> 'targetClass' parameter and by passing the 
> constructor's declaring class into this method instead of null, 
> reflective checks suddenly start to look more like JLS dictates (but 
> still a long way from it, unfortunately).
>
> As a bonus, sun.reflect.misc.ReflectUtil#ensureMemberAccess method 
> (used from AtomicXXXFieldUpdater classes) does not need the following 
> comment any more:
>
>      * Reflection.ensureMemberAccess is overly-restrictive
>      * due to a bug. We awkwardly work around it for now.
>
> ...as it can now delegate straight to Reflection.ensureMemberAccess 
> without invoking it twice with different modified member access 
> modifiers and performing part of the check itself.
>
> java.lang.reflect.AccessibleObject#checkAccess delegates to 
> Reflection.ensureMemberAccess and caches the result, so it had to be 
> modified too.
>
> Constructor now passes it's declaring class to the 'targetClass' 
> parameter and Filed/Method obey the spec and REALLY IGNORE 'obj' 
> parameter in get/set/invoke on a static member.
>
> All java/lang/reflect and java/util/concurrent/atomic tests pass with 
> this patch applied except the following:
>
> java/lang/reflect/AccessibleObject/ModuleSetAccessibleTest.java: Test 
> java.lang.reflect.AccessibleObject with modules
> java/lang/reflect/Generics/TestBadSignatures.java: Test bad signatures 
> get a GenericSignatureFormatError thrown.
> java/lang/reflect/Method/invoke/TestPrivateInterfaceMethodReflect.java: 
> Reflection support for private methods in interfaces
> java/lang/reflect/Module/AddExportsTest.java: Test Module isExported 
> methods with exports changed by -AddExportsTest
> java/lang/reflect/Proxy/ProxyModuleMapping.java: Basic test of proxy 
> module mapping and the access to Proxy class
> java/lang/reflect/WeakPairMap/Driver.java: Functional test for 
> WeakPairMap
>
> ...which all fail because of:
>
>     javac: invalid flag: -XaddExports:java.base/jdk.internal....
>
>
>
> Regards, Peter
>
>


From christoph.langer at sap.com  Mon Oct  3 06:11:41 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 3 Oct 2016 06:11:41 +0000
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <CA+kOe0-LcMPp5Xu9Fcs4b7vi1eCvU-KmdPAK_sRessqwfQVdMg@mail.gmail.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
	<57EFAFEB.6010006@oracle.com>
	<CA+kOe0-LcMPp5Xu9Fcs4b7vi1eCvU-KmdPAK_sRessqwfQVdMg@mail.gmail.com>
Message-ID: <5405f7f80c914d459efb4394f694d199@DEWDFE13DE11.global.corp.sap>

Hi Martin,

this also looks like a good catch to me, +1.

When you push, could you insert the copyright block from, e.g. src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11DocumentScannerImpl.java at the place where you currently find the " reserved comment block ",  first 4 lines? The XML copyright headers get cleaned up as they are touched...

Thanks and best regards
Christoph
 
> -----Original Message-----
> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf
> Of Martin Buchholz
> Sent: Samstag, 1. Oktober 2016 18:55
> To: Claes Redestad <claes.redestad at oracle.com>
> Cc: core-libs-dev <core-libs-dev at openjdk.java.net>
> Subject: Re: RFR: 8167002: JAXP schema validator: Use HashSet instead of
> ArrayList for tracking XML IDs
> 
> On Sat, Oct 1, 2016 at 5:45 AM, Claes Redestad <claes.redestad at oracle.com>
> wrote:
> 
> >
> > On 2016-10-01 02:41, Martin Buchholz wrote:
> >
> >> https://bugs.openjdk.java.net/browse/JDK-8167002
> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-v
> >> alidation-by-hash/
> >>
> >>
> > +1, but I have to ask what the intended benefit of writing:
> >
> >   ((fIds != null) ? fIds : (fIds = new HashSet<>())).add(name);
> >
> > rather than keeping the pre-existing pattern:
> >
> >   if (fIds == null) fIds = new HashSet<>();
> >   fIds.add(name);
> >
> > If this is about bytecode optimization to help with inlining or such,
> > the latter actually generate a more compact method (14 vs 16 bytecodes).
> 
> 
> The intent was (by instinct) to not code a re-read of a field, but ... yeah
> .... that wasn't achieved.
> That could be done correctly using
> 
>     public void addId(String name) {
>         HashSet<String> x;
>         if ((x = fIds) == null) fIds = x = new HashSet<>();
>         x.add(name);
>     }
> 
> but I'll just revert to
>         if (fIds == null) fIds = new HashSet<>();
>         fIds.add(name);

From chris.hegarty at oracle.com  Mon Oct  3 08:53:00 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 3 Oct 2016 09:53:00 +0100
Subject: RFR 9: 8155760 Implement Serialization Filtering
In-Reply-To: <ef156ca9-91d7-fe6f-1706-1d784666623a@oracle.com>
References: <ed740304-d7d8-e50d-db96-d13ca9b0f2cb@Oracle.com>
	<c1e74e63-02cd-caf7-7ca1-bb81fd26234f@oracle.com>
	<7a1bd8d7-1bf6-7357-6fcd-b7c83b08e52b@Oracle.com>
	<A43E4781-9662-477D-8C07-389C3C1B2C56@oracle.com>
	<3ce4aef4-e0fc-4d4f-dfd6-484e12f89e2a@Oracle.com>
	<9666c25e-ef9c-8ea2-6175-019801b2e75f@Oracle.com>
	<58bbd16f-f6d4-f51d-1073-5dd7ee7a1f04@oracle.com>
	<8f483d12-aa78-a1d8-69fd-88c19e7ca7ea@Oracle.com>
	<64af9c79-6d69-d619-de5b-4ae7d6107247@Oracle.com>
	<ef156ca9-91d7-fe6f-1706-1d784666623a@oracle.com>
Message-ID: <52034d36-f895-021a-45ea-e398bdb42244@oracle.com>

Roger,

On 14/09/16 10:46, Chris Hegarty wrote:
> On 08/09/16 20:09, Roger Riggs wrote:
>...
> This looks very good Roger, just a few comments:
>
> 1) The pattern separator in the java.security file should be ';'
>    Right?
>        925 #jdk.serialFilter=pattern,pattern
>                                    ^^^

Strike this, it seems to have been fixed in the most recent version.


> 2) A question on the excepted usage. During the initialization of
>    OIS the process-wide filter is cached in an instance field,
>    'serialFilter'. A subsequent change to the process-wide filter
>    will not affect the OIS instance. I think this is ok, just
>    checking the expected usage, as the example in the OIF class
>    description reads the process-wide filter ever time. Maybe
>    the example should be changed slightly to no promote this type
>    of usage? Maybe just remove the call to getSerialFilter?
>
> 3) Are third-party OIS implementations required, or expected, to
>    "callback" to the filter? The spec, of course, would appear to
>    allow it, but not require it? Just wondering if this is required,
>    or not, as it is not clear to me.

One more additional comment:

  4) Since filtering is not controlled by the Security Manager,
     does it make sense for its configuration to live in the
     java.security file?

-Chris.

> -Chris.
>
>
>> SpecDiff:
>> http://cr.openjdk.java.net/~rriggs/filter-diffs/overview-summary.html
>>
>> Javadoc (subset)
>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/ObjectInputStream.html
>>
>>
>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/ObjectInputFilter.html
>>
>>
>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/SerializablePermission.html
>>
>>
>>
>> Thanks, Roger
>>
>>
>>

From ramanand.patil at oracle.com  Mon Oct  3 12:27:23 2016
From: ramanand.patil at oracle.com (Ramanand Patil)
Date: Mon, 3 Oct 2016 05:27:23 -0700 (PDT)
Subject: RFR: 8166875: (tz) Support tzdata2016g
Message-ID: <44b5d97a-222a-4d46-8896-8a2115108081@default>

HI all,
Please review the latest TZDATA integration (tzdata2016g) to JDK9.
Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/

All the TimeZone related tests are passed after integration.
[BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since they verify the renamed TZID "Asia/Yangon"].

Regards,
Ramanand.

From Roger.Riggs at Oracle.com  Mon Oct  3 14:01:18 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 3 Oct 2016 10:01:18 -0400
Subject: RFR 9: 8155760 Implement Serialization Filtering
In-Reply-To: <52034d36-f895-021a-45ea-e398bdb42244@oracle.com>
References: <ed740304-d7d8-e50d-db96-d13ca9b0f2cb@Oracle.com>
	<c1e74e63-02cd-caf7-7ca1-bb81fd26234f@oracle.com>
	<7a1bd8d7-1bf6-7357-6fcd-b7c83b08e52b@Oracle.com>
	<A43E4781-9662-477D-8C07-389C3C1B2C56@oracle.com>
	<3ce4aef4-e0fc-4d4f-dfd6-484e12f89e2a@Oracle.com>
	<9666c25e-ef9c-8ea2-6175-019801b2e75f@Oracle.com>
	<58bbd16f-f6d4-f51d-1073-5dd7ee7a1f04@oracle.com>
	<8f483d12-aa78-a1d8-69fd-88c19e7ca7ea@Oracle.com>
	<64af9c79-6d69-d619-de5b-4ae7d6107247@Oracle.com>
	<ef156ca9-91d7-fe6f-1706-1d784666623a@oracle.com>
	<52034d36-f895-021a-45ea-e398bdb42244@oracle.com>
Message-ID: <93ecbf84-ba86-a4bc-b478-7014a16d7589@Oracle.com>

Hi Chris,

Thanks for taking another look.

On 10/3/2016 4:53 AM, Chris Hegarty wrote:
> Roger,
>
> On 14/09/16 10:46, Chris Hegarty wrote:
>
> One more additional comment:
>
>  4) Since filtering is not controlled by the Security Manager,
>     does it make sense for its configuration to live in the
>     java.security file?
The primary function of serialization filtering is security related and 
it leverages the existing
configuration mechanism for security functions. Though slightly 
off-topic, it did not seem
worthwhile to create a separate configuration mechanism. I discussed the 
location and properties
with the security team and they have reviewed the changes.

Thanks, Roger

>
> -Chris.
>
>> -Chris.
>>
>>

/Webrev: 
http://cr.openjdk.java.net/~rriggs/webrev-serial-filter-jdk9-8155760/ 
<http://cr.openjdk.java.net/%7Erriggs/webrev-serial-filter-jdk9-8155760/>/

>>> SpecDiff:
>>> http://cr.openjdk.java.net/~rriggs/filter-diffs/overview-summary.html
>>>
>>> Javadoc (subset)
>>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/ObjectInputStream.html 
>>>
>>>
>>>
>>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/ObjectInputFilter.html 
>>>
>>>
>>>
>>> http://cr.openjdk.java.net/~rriggs/filter-javadoc/java/io/SerializablePermission.html 
>>>
>>>
>>>
>>>
>>> Thanks, Roger
>>>
>>>
>>>


From daniel.fuchs at oracle.com  Mon Oct  3 14:28:13 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Mon, 3 Oct 2016 15:28:13 +0100
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
Message-ID: <0ed94078-47eb-bf5d-9d64-898ca0c564c7@oracle.com>

Hi Martin,

This looks good to me.

best regards,

-- daniel

On 01/10/16 01:41, Martin Buchholz wrote:
> https://bugs.openjdk.java.net/browse/JDK-8167002
> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/


From martinrb at google.com  Mon Oct  3 15:24:40 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 3 Oct 2016 08:24:40 -0700
Subject: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
In-Reply-To: <44b5d97a-222a-4d46-8896-8a2115108081@default>
References: <44b5d97a-222a-4d46-8896-8a2115108081@default>
Message-ID: <CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>

Hi Ramanand,
Pleased to meet you!

I expected to see Yangon added to ZoneName, because of the existing
reference to Rangoon

java/time/test/java/time/format/ZoneName.java:179:        "Asia/Rangoon",
"Myanmar", "Asia/Rangoon",

Is ZoneName.java trying to maintain a comprehensive list of zone names?

"""Yangon (???????) is a combination of the two words yan (???) and koun
(????), which mean "enemies" and "run out of", respectively. It is also
translated as "End of Strife"."""


On Mon, Oct 3, 2016 at 5:27 AM, Ramanand Patil <ramanand.patil at oracle.com>
wrote:

> HI all,
> Please review the latest TZDATA integration (tzdata2016g) to JDK9.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
> Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/
>
> All the TimeZone related tests are passed after integration.
> [BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since
> they verify the renamed TZID "Asia/Yangon"].
>
> Regards,
> Ramanand.
>

From rachna.goel at oracle.com  Mon Oct  3 15:29:52 2016
From: rachna.goel at oracle.com (Rachna Goel)
Date: Mon, 3 Oct 2016 20:59:52 +0530
Subject: RFR:JDK-8146750:java.time.Month.getDisplayName() return incorrect
	narrow names with JRE provider.
Message-ID: <af0cbce6-6fb3-a971-1b2c-149893bb4721@oracle.com>

Hi,

Please review fix for JDK-8146750.

Bug : https://bugs.openjdk.java.net/browse/JDK-8146750

Fix: http://cr.openjdk.java.net/~rgoel/JDK-8146750/webrev.04/

Fix is to retrieve Narrow and Narrow_Standalone Month names as well as 
Day names one by one, as they can be duplicate.

Thanks,
Rachna


From chris.hegarty at oracle.com  Mon Oct  3 15:42:36 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 3 Oct 2016 16:42:36 +0100
Subject: [9] RFR of JDK-8085192: java/rmi/activation/Activatable tests
	fail intermittently due to "Port already in use"
In-Reply-To: <E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
References: <3c0bdf63-f55c-5b93-6d74-58e26916cb3b@oracle.com>
	<f881b76f-2a76-5abd-3118-7f6fe12d9205@Oracle.com>
	<e3b084d3-de5e-700f-2f69-22f91a0f80bf@oracle.com>
	<02147682-541f-9746-b6da-2cc2800165b5@Oracle.com>
	<3E39A1E9-0764-404D-9824-85CFB75113FF@oracle.com>
	<E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
Message-ID: <02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>

Here is an updated version of this ( ready for review ):

   http://cr.openjdk.java.net/~chegar/8085192_webrev.02/

Changes from previous:

1) Updated Activation/rmid to NOT redirect stderr, if an
    implementation specific system property is used ( we can
    discuss the name )

2) For now, I added createRMIDOnEphemeralPort, rather than
    change the current implementation, as it is being used in
    other places. We can revert this once other usages are
    updated and verified.

-Chris.

On 29/09/16 20:09, Chris Hegarty wrote:
> On 29 Sep 2016, at 16:25, Chris Hegarty <chris.hegarty at oracle.com> wrote:
>>
>> I have asked Hamlin to hold off on this for a day or so.  I have an
>> alternative proposal that eliminates the free port anti-pattern.
>
> It is possible to use the inheritedChannel mechanism to have the rmid
> process create the server channel on an ephemeral port and report it
> back to the test, i.e. remove the free port pattern.
>
> http://cr.openjdk.java.net/~chegar/8085192_webrev/
>
> 1) The port number is reported from rmid to the test over stdout.
>
> 2) All tests pass except CheckAnnotations.java, which looks for stderr
>     ( see 3 below ). I think the stderr check can be removed, and the
>     just check stdout.
>
> 3)  rmid, when using inheritChannel, redirects stderr to a tmp file, so
>      it is not possible to get the processes stderr over the processes
>      stderr pipe. I did?t find that this was an issue when testing ( other
>      than having to clear out /tmp )
>
> 4) This could be expanded to other tests, outside of activation, to
>      remove more usages of free port.
>
> This is not yet complete, I just want to share the idea to see if it is a
> runner, or not.
>
> -Chris.
>

From huizhe.wang at oracle.com  Mon Oct  3 16:28:44 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Mon, 03 Oct 2016 09:28:44 -0700
Subject: RFR: 8167002: JAXP schema validator: Use HashSet instead of
	ArrayList for tracking XML IDs
In-Reply-To: <0ed94078-47eb-bf5d-9d64-898ca0c564c7@oracle.com>
References: <CA+kOe0_aqJQpbGs-4oPEaz-Lqkes9nZL2x98fh6vKT5+KYbFhQ@mail.gmail.com>
	<0ed94078-47eb-bf5d-9d64-898ca0c564c7@oracle.com>
Message-ID: <57F2873C.9060408@oracle.com>

+1, great improvement!

Best,
Joe

On 10/3/16, 7:28 AM, Daniel Fuchs wrote:
> Hi Martin,
>
> This looks good to me.
>
> best regards,
>
> -- daniel
>
> On 01/10/16 01:41, Martin Buchholz wrote:
>> https://bugs.openjdk.java.net/browse/JDK-8167002
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/xml-id-validation-by-hash/ 
>>
>

From huizhe.wang at oracle.com  Mon Oct  3 16:37:27 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Mon, 03 Oct 2016 09:37:27 -0700
Subject: RFR: 8164479: Update JAX-WS RI integration to latest version
In-Reply-To: <435c44ae-51fc-c727-915c-6acb499d72a9@oracle.com>
References: <1c63a0dc-d38d-0148-016c-30b15721d6a6@oracle.com>
	<3F91468F-78B2-4D3D-80D2-E429D8FAFC61@oracle.com>
	<57E5909E.3050702@oracle.com>
	<ec591897-d2e6-32f5-9dd0-97b95c76863d@oracle.com>
	<5754D4FE-0D20-4527-B163-E9224E3A4FDD@oracle.com>
	<5ac5eeec-fe39-0728-1ee4-2bc8684722d1@oracle.com>
	<435c44ae-51fc-c727-915c-6acb499d72a9@oracle.com>
Message-ID: <57F28947.5050102@oracle.com>

Nice to see fewer dependencies on internal API!

As you have removed the exports, the related comment can go with it as well:
      // reflection access from 
com.sun.xml.internal.ws.api.streaming.XMLStreamWriterFactory
-    exports com.sun.xml.internal.stream.writers to java.xml.ws;

Best,
Joe

On 9/25/16, 4:41 PM, Aleks Efimov wrote:
> Hi Alan, Joe, Mandy, Roman,
>
> Suggested changes to the comment section (will bring this change to 
> standalone JAXB) and to the exported internal API were made:
> com.sun.xml.internal.stream.writers, 
> com.sun.org.apache.xml.internal.resolver, 
> com.sun.org.apache.xml.internal.resolver.tools dependencies were removed.
>
> The updated webrev:
> http://cr.openjdk.java.net/~aefimov/8164479/01
>
> Best Regards,
> Aleksej
>
>
> On 25/09/16 17:49, Alan Bateman wrote:
>> On 24/09/2016 18:57, Mandy Chung wrote:
>>
>>> You can run jdeps --check java.xml.ws on your local build with this 
>>> change.
>>>
>>> This will analyze the dependences and any unused qualified exports.
>>>
>> Good idea. I think java.activation's module-info.java will need 
>> updating too as it no longer requires java.desktop (we can decide at 
>> a later time whether to change this to `requires static` and change 
>> this code to use a reflection guard + static reference).
>>
>> -Alan
>

From vladimir.kozlov at oracle.com  Mon Oct  3 21:49:40 2016
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 3 Oct 2016 14:49:40 -0700
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
Message-ID: <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>

Hi Volker,

Can you prepare combined patch (or set of patches) based on latest 
reviews together with s390 code as it will be in final push?

We want to run it through our pre-integration testing to verify that it 
does not have problems.

Thanks,
Vladimir

On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
> other) state:
>
> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
>
> Vladimir
>
> On 9/29/16 9:55 AM, Volker Simonis wrote:
>> Hi Vladimir,
>>
>> thanks a lot for reviewing and endorsing the JEP.
>>
>> I've linked all the relevant issues to the JEP  (they all have a link
>> to a webrev) and change the state to "Submitted".
>>
>> There's just one more small shared change we need for the port for
>> which we haven't opened a bug now because we are still working on
>> simplifying it. The current version looks as follows:
>>
>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-constant_table_offset.patch
>>
>>
>> What are the next steps? Should I add a "jdk9-fc-request" label to t
>> he JEP and add a corresponding "FC Extension Request" comment to it?
>> Or will this be done automatically once I move it to "Candidate"?
>>
>> Is there anything left to do before I can move it to "Candidate" state?
>>
>> Thanks a lot and best regards,
>> Volker
>>
>>
>>
>>
>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com> wrote:
>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
>>>>
>>>> Hi,
>>>>
>>>> can you please review and endorse the following draft JEP for the
>>>> integration of the Linux/s390x port into the jkd9 master repository:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
>>>
>>>
>>> Good.
>>> Add links to webrevs in a comment. It will help to get umbrella FC
>>> extension
>>> approval.
>>>
>>>>
>>>> As detailed in the JEP, the Linux/s390x requires very few shared
>>>> changes and we therefore don't foresee any impact on the existing
>>>> platforms at all. Following you can find a short description of the
>>>> planned changes:
>>>>
>>>> hotspot:
>>>> =======
>>>>
>>>> Out for review:
>>>> 8166560: [s390] Basic enablement of s390 port.
>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/
>>>>
>>>> Reviewed:
>>>> 8166562: C2: Suppress relocations in scratch emit.
>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/
>>>>
>>>> Will send RFR soon (depends on 8166560):
>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01
>>>
>>>
>>> Wrong link.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>>
>>>>
>>>> We are still investigating the need of these shared changes:
>>>>
>>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch
>>>>
>>>>
>>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch
>>>>
>>>>
>>>> And finally the patch with the s390x-only platform files. We are still
>>>> editing these to get them into OpenJdk style and shape.
>>>> Hotspot passes most jck, jtreg and spec tests with these.
>>>>
>>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch
>>>>
>>>>
>>>> top-level repository:
>>>> ===============
>>>>
>>>> The following is just adding some s390x specific compiler flags to
>>>> flags.m4
>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
>>>>
>>>> jdk repository:
>>>> ============
>>>>
>>>> This one just adds a new jvm.cfg file for s390x
>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
>>>>
>>>>
>>>> And finally we plan to do one more change which fixes the jtreg test
>>>> on Linux/s390x. But this is mainly for the correct detection of the
>>>> platform and for excluding the tests which are not appropriate for
>>>> s390x.
>>>>
>>>> Thank you and best regards,
>>>> Volker
>>>>
>>>

From naoto.sato at oracle.com  Mon Oct  3 22:17:31 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Mon, 3 Oct 2016 15:17:31 -0700
Subject: [9] RFR: 8166645: Include locales plugin throws InternalError
	with "*" specified.
In-Reply-To: <d6bf4d12-e904-f3bb-8383-dab6250c2f46@oracle.com>
References: <d6bf4d12-e904-f3bb-8383-dab6250c2f46@oracle.com>
Message-ID: <431dcb9b-6dfc-6ee7-73ca-96c3b1a7f401@oracle.com>

According to an internal review, I added some explanation to the 
modification. No logic modifications.

http://cr.openjdk.java.net/~naoto/8166645/webrev.01/

Naoto

On 9/27/16 11:43 AM, Naoto Sato wrote:
> Hello,
>
> Please review the changes for the following issue:
>
> https://bugs.openjdk.java.net/browse/JDK-8166645
>
> The proposed fix is located at:
>
> http://cr.openjdk.java.net/~naoto/8166645/webrev.00/
>
> The fix makes sure that the supported language tags created by the
> plugin are in the original supported language tags.
>
> Naoto

From mandy.chung at oracle.com  Tue Oct  4 02:53:52 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 3 Oct 2016 19:53:52 -0700
Subject: [9] RFR: 8166645: Include locales plugin throws InternalError
	with "*" specified.
In-Reply-To: <431dcb9b-6dfc-6ee7-73ca-96c3b1a7f401@oracle.com>
References: <d6bf4d12-e904-f3bb-8383-dab6250c2f46@oracle.com>
	<431dcb9b-6dfc-6ee7-73ca-96c3b1a7f401@oracle.com>
Message-ID: <3A8B2A45-96AA-42C2-B8F8-8803C3EAAF71@oracle.com>


> On Oct 3, 2016, at 3:17 PM, Naoto Sato <naoto.sato at oracle.com> wrote:
> 
> According to an internal review, I added some explanation to the modification. No logic modifications.
> 
> http://cr.openjdk.java.net/~naoto/8166645/webrev.01/

This looks okay.  

I suggest to keep Set<String> originals after splitting the input byte array such that
  325             .filter(t -> original.indexOf(t) != -1)

can be replaced with .filter(originals::contains)

Mandy

From peter.levart at gmail.com  Tue Oct  4 06:51:43 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 4 Oct 2016 08:51:43 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
Message-ID: <e96f4505-a9fd-7024-d80c-efd48d3b291f@gmail.com>

To a potential reviewer...

The change might look scary at first as it touches reflective access 
controls and by that, security of the platform, but:

- it is just (32 ins; 56 del; 62 mod) lines. The rest is a new jtreg 
test which is beneficial by itself as such test did not exist until now.
- the jtreg test is exhaustive and proves that the patch does not have 
undesired effects.

Thanks,

Peter Levart

On 10/02/2016 11:51 PM, Peter Levart wrote:
> Hi,
>
> I added an exhaustive jtreg test that covers all possible situations.
>
> From the set of the following classes:
>
> package a;
> public class PublicSuper {...}
>
> package a;
> class Package {...}
>
> package b;
> public class PublicSub extends a.PublicSuper {...}
>
> package b;
> class Package {...}
>
> it creates a set of all possible triplets:
>
> (currentClass, memberClass, targetClass)
>
> where:
>
> currentClass - the class making the reflective access
> memberClass - the member's declaring class
> targetClass - the target object's class (for accessing instance fields 
> and methods - must be equal to or subclass of memberClass)
>
> For each such triplet it checks the reflective access to each of the 
> following members:
>
> {private, package, protected, public} x {instance, static} x {field, 
> method}
>
> and:
>
>  {private, package, protected, public} x {constructor}
>
> When running on unpatched build 9-ea+137, the result is:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/AccessControlTest_unpatched.jtr
>
> When running on patched build of 9, the result is:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/AccessControlTest_patched.jtr
>
>
> The difference is exactly in the following two cases which fail for 
> unpatched and are fixed by the patch:
>
> b.PublicSub accessing a.PublicSuper's PROTECTED_STATIC_FIELD - 
> expected allowed, actual denied : FAILURE
> b.PublicSub accessing a.PublicSuper's PROTECTED_STATIC_METHOD - 
> expected allowed, actual denied : FAILURE
>
> QED.
>
>
> This is new webrev:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.02/
>
> I think the test proves the effect of the patch is as intended, 
> therefore it should not be a problem to review it.
>
>
> Regards, Peter
>
>
> On 10/01/2016 12:20 AM, Peter Levart wrote:
>> Hi,
>>
>> I have a fix for a 10 year old bug (JDK-6378384 
>> <https://bugs.openjdk.java.net/browse/JDK-6378384>). It was marked as 
>> a duplicate of a 19 year old bug (JDK-4032740 
>> <https://bugs.openjdk.java.net/browse/JDK-4032740>) which is marked 
>> as a duplicate of a 17 year old bug (JDK-4283544 
>> <https://bugs.openjdk.java.net/browse/JDK-4283544>) which is still 
>> open. But this bug is not a strict duplicate. This bug only concerns 
>> reflective access to protected members.
>>
>> Here's a proposed fix:
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.01/
>>
>>
>> The bug manifests itself as not being able to access protected static 
>> methods or fields from a subclass located in a different package. 
>> Instance protected methods and fields can be accessed, and using an 
>> undocumented trick, also static methods and fields, but the trick is 
>> very subtle. The specification for Field.get/set and Method.invoke 
>> says, respectively:
>>
>>      * <p>If the underlying field is a static field, the {@code obj} 
>> argument
>>      * is ignored; it may be null.
>>
>> and:
>>
>>      * <p>If the underlying method is static, then the specified 
>> {@code obj}
>>      * argument is ignored. It may be null.
>>
>> Well, it is not exactly so! The obj argument is used as a 'target' 
>> even for protected static members and it is ensured that its class is 
>> equal or a subclass of the class that accesses the member. So if you 
>> pass an instance of a subclass of the protected method's declaring 
>> class into the get/set/invoke, you can access the static protected 
>> member. If you pass 'null', you get IllegalAccessException.
>>
>> The problem is in the design of 
>> jdk.internal.reflect.Reflection#ensureMemberAccess method which is 
>> used to check reflective access. It takes an Object 'target' 
>> argument, which is supposed to be null when accessing static 
>> methods/fields and it is null also when accessing constructors. 
>> Because of constructors and the method's API, it has to be overly 
>> restrictive as it must only allow calling protected constructors from 
>> within the constructor's declaring class itself or same package, 
>> while protected static methods could be called from any subclass.
>>
>> By redesigning the API of this method, replacing Object 'target' 
>> parameter with Class<?> 'targetClass' parameter and by passing the 
>> constructor's declaring class into this method instead of null, 
>> reflective checks suddenly start to look more like JLS dictates (but 
>> still a long way from it, unfortunately).
>>
>> As a bonus, sun.reflect.misc.ReflectUtil#ensureMemberAccess method 
>> (used from AtomicXXXFieldUpdater classes) does not need the following 
>> comment any more:
>>
>>      * Reflection.ensureMemberAccess is overly-restrictive
>>      * due to a bug. We awkwardly work around it for now.
>>
>> ...as it can now delegate straight to Reflection.ensureMemberAccess 
>> without invoking it twice with different modified member access 
>> modifiers and performing part of the check itself.
>>
>> java.lang.reflect.AccessibleObject#checkAccess delegates to 
>> Reflection.ensureMemberAccess and caches the result, so it had to be 
>> modified too.
>>
>> Constructor now passes it's declaring class to the 'targetClass' 
>> parameter and Filed/Method obey the spec and REALLY IGNORE 'obj' 
>> parameter in get/set/invoke on a static member.
>>
>> All java/lang/reflect and java/util/concurrent/atomic tests pass with 
>> this patch applied except the following:
>>
>> java/lang/reflect/AccessibleObject/ModuleSetAccessibleTest.java: Test 
>> java.lang.reflect.AccessibleObject with modules
>> java/lang/reflect/Generics/TestBadSignatures.java: Test bad 
>> signatures get a GenericSignatureFormatError thrown.
>> java/lang/reflect/Method/invoke/TestPrivateInterfaceMethodReflect.java: 
>> Reflection support for private methods in interfaces
>> java/lang/reflect/Module/AddExportsTest.java: Test Module isExported 
>> methods with exports changed by -AddExportsTest
>> java/lang/reflect/Proxy/ProxyModuleMapping.java: Basic test of proxy 
>> module mapping and the access to Proxy class
>> java/lang/reflect/WeakPairMap/Driver.java: Functional test for 
>> WeakPairMap
>>
>> ...which all fail because of:
>>
>>     javac: invalid flag: -XaddExports:java.base/jdk.internal....
>>
>>
>>
>> Regards, Peter
>>
>>
>


From david.holmes at oracle.com  Tue Oct  4 07:56:15 2016
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 4 Oct 2016 17:56:15 +1000
Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port.
In-Reply-To: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap>
References: <b384c60b06394f5aaf9a11376899a0aa@DEWDFE13DE50.global.corp.sap>
	<e3963e6f-2dec-f00b-0489-67c4d7822959@oracle.com>
	<CA+3eh13Ca5O=f_+CUHdguP35ZMqWsuEVum+mtbZ5+9xPJDaO3w@mail.gmail.com>
	<4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap>
Message-ID: <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com>

Hi Goetz,

On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote:
> Hi,
>
> here a new webrev for this change:
> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr02/
>
> I reverted the major reorderings in macros.hpp and os_linux.cpp.
> David asked me to do so, and I guess it makes reviewing more simple.

Thanks. That said ...

> Also this fixes the issue spotted by David which actually was wrong.
> The other renaming of ARM to ARM32 was correct, though, as
> AARCH64 is defined in both ARM 64-bit ports, and is checked before.
> So the existing case checking ARM is only reached if !LP64.
> I documented this ...

... We actually have a bug with the Elf32_* defines in os_linux.cpp

1769 #elif  (defined ARM) // ARM must come before AARCH64 because the 
closed 64-bit port requires so.
1770   static  Elf32_Half running_arch_code=EM_ARM;

Aarch64 should come first otherwise we will set the wrong value. I've 
been told it would only affect an error message but still ... can I ask 
you to fix this please. Thanks.

---
src/share/vm/code/codeCache.cpp

I'd prefer to just see something like:

S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is 
used

---

Otherwise seems okay.

Thanks,
David

> Also I removed the change in mutex.hpp.  Maybe we contribute
> the full change this was part of, but independent of the s390 port.
>
> I withdraw the part of the change to jdk introducing jvm.cfg.  Volker wants to
> do a separate change for this.
>
> Best regards,
>   Goetz.
>
>
>
>> -----Original Message-----
>> From: Volker Simonis [mailto:volker.simonis at gmail.com]
>> Sent: Dienstag, 27. September 2016 19:58
>> To: David Holmes <david.holmes at oracle.com>
>> Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; hotspot-
>> dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.net>
>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port.
>>
>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes <david.holmes at oracle.com>
>> wrote:
>>> Hi Goetz,
>>>
>>> I see a change not related directly to S390 ie change from ARM to ARM32 in
>>> src/os/linux/vm/os_linux.cpp
>>>
>>
>> The change looks a little confusing because Goetz reordered the ifdef
>> cascades alphabetically (which I think is good).
>>
>> Besides that, the only real change not related to s390 is indeed the
>> change from ARM to ARM32 which happend two times in the file.
>>
>> @Goetz: have you done this intentionally?
>>
>>> It will be a while before I can go through this in any detail.
>>>
>>> David
>>>
>>>
>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote:
>>>>
>>>> Hi,
>>>>
>>>> This change is part of the s390 port. It contains some basic adaptions
>>>> needed for a full hotspot port for linux s390x.
>>>>
>>>> It defines the required macros, platform names and includes.
>>>>
>>>> The s390 port calles CodeCache::contains() in current_frame(), which is
>>>> used in NMT. As NMT already collects stack traces before the CodeCache
>> is
>>>> initialized, contains() needs a check for this.
>>>>
>>>>  Wherever a row of platforms are listed, I sorted them alphabetically.
>>>>
>>>>  The jdk requires the file jvm.cfg.
>>>>
>>>> Please review. I please need a sponsor.
>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>> basic_s390/hotspot.wr01/
>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/
>>>>
>>>> Best regards,
>>>>   Goetz.
>>>>
>>>

From christoph.langer at sap.com  Tue Oct  4 08:33:19 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 4 Oct 2016 08:33:19 +0000
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
Message-ID: <b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>

Hi Lucy,

FWIW: I ran a build on AIX and this looks ok.

I also assume in your final version you'll update all copyright years where it's not 2016 yet? Other than that the changes look ok to me - but I'm neither a reviewer nor a deep expert in the area of your changes.

Best regards
Christoph

> -----Original Message-----
> From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu,
> Yingqi
> Sent: Freitag, 30. September 2016 18:55
> To: Lu, Yingqi <yingqi.lu at intel.com>; Alan Bateman
> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>;
> Kharbas, Kishor <kishor.kharbas at intel.com>
> Subject: RE: Proposal for adding O_DIRECT support into JDK 9
> 
> Hi All,
> 
> Please find the most recent version of the patch available at
> http://cr.openjdk.java.net/~igraves/8164900-2/
> 
> In this version, we have following two changes:
> 
> 1. Move O_DIRECT flag from StandardOpenOption to ExtendedOpenOption
> 2. Move the checks of O_DIRECT from native code to Java code.
> 
> Please let us know your feedback.
> 
> Thanks,
> Lucy
> 
> -----Original Message-----
> From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu,
> Yingqi
> Sent: Tuesday, September 27, 2016 9:57 AM
> To: Alan Bateman <Alan.Bateman at oracle.com>; core-libs-
> dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net
> Subject: RE: Proposal for adding O_DIRECT support into JDK 9
> 
> Alan,
> 
> Thank you for the explanation, we will modify the code accordingly and send it
> out soon for review.
> 
> Thanks,
> Lucy
> 
> -----Original Message-----
> From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
> Sent: Tuesday, September 27, 2016 8:45 AM
> To: Lu, Yingqi <yingqi.lu at intel.com>; core-libs-dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net
> Subject: Re: Proposal for adding O_DIRECT support into JDK 9
> 
> On 26/09/2016 19:50, Lu, Yingqi wrote:
> 
> > Alan, you mean readv0/write0 or read0/write0? I just want to make sure
> > :-)
> Apologies, I meant each of the native methods where the decision to use direct
> I/O or not would be a lot more maintainable in Java. You'll see that there are
> already code paths for direct vs. heap buffers.
> 
> 
> >
> > Anyone else has other opinions on where is the best home for O_DIRECT flag?
> The flags under jdk.unsupported will eventually be removed in the future JDK
> release?
> >
> > If we agree ExtendedOpenOpen is the best home for O_DIRECT, we can
> modify that for sure.
> >
> I think ExtendedOpenOption is the right place. It's still TDB as to whether to put
> these extensions but that should be transparent to anyone using this when on
> the class path.
> 
> -Alan

From ramanand.patil at oracle.com  Tue Oct  4 10:22:18 2016
From: ramanand.patil at oracle.com (Ramanand Patil)
Date: Tue, 4 Oct 2016 03:22:18 -0700 (PDT)
Subject: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
In-Reply-To: <CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>
References: <44b5d97a-222a-4d46-8896-8a2115108081@default>
	<CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>
Message-ID: <68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>

Hi Martin,
Thank you for your review and explanation of "Yangon". I liked the translation "End of Strife".

Looking at the description of the ZoneNames.java:
* The zid<->metazone mappings are based on CLDR metaZones.xml.
 * The alias mappings are based on Link entries in tzdb data files.

I had thought to not update this file because the CLDR metaZones.xml file doesn?t have this entry updated.
But I think you are correct, since Link entry has this alias mentioned, there is no harm in adding these entries to zidMap and aliasMap arrays.
Here is the updated Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.01/

Changes done:
	- Updated src/java.base/share/classes/java/time/format/ZoneName.java to include "Yangon" entry.
	- Removed unused imports from src/java.base/share/classes/java/time/format/ZoneName.java
	- Updated ZoneName.java in the test package as well to include "Yangon". [test/java/time/test/java/time/format/ZoneName.java]
	- Updated the bugID for test/java/time/test/java/time/format/TestZoneTextPrinterParser.java since this uses the "ZoneName.java" defined in test package.

Also, looks like ZoneName.java is trying to maintain a comprehensive list of zone names. Though I found very few zone names are missing from this file like: "Europe/Busingen", "America/Fort_Nelson", "Antarctica/Troll" etc...


Regards,
Ramanand.

From: Martin Buchholz [mailto:martinrb at google.com] 
Sent: Monday, October 03, 2016 8:55 PM
To: Ramanand Patil <ramanand.patil at oracle.com>
Cc: i18n-dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.net>
Subject: Re: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g

Hi Ramanand,
Pleased to meet you!

I expected to see Yangon added to?ZoneName, because of the existing reference to Rangoon

java/time/test/java/time/format/ZoneName.java:179: ? ? ? ?"Asia/Rangoon", "Myanmar", "Asia/Rangoon",

Is ZoneName.java trying to maintain a comprehensive list of zone names?

"""Yangon (???????) is a combination of the two words yan (???) and koun (????), which mean "enemies" and "run out of", respectively. It is also translated as "End of Strife"."""


On Mon, Oct 3, 2016 at 5:27 AM, Ramanand Patil <mailto:ramanand.patil at oracle.com> wrote:
HI all,
Please review the latest TZDATA integration (tzdata2016g) to JDK9.
Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/

All the TimeZone related tests are passed after integration.
[BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since they verify the renamed TZID "Asia/Yangon"].

Regards,
Ramanand.

From claes.redestad at oracle.com  Tue Oct  4 10:50:59 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 4 Oct 2016 12:50:59 +0200
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
Message-ID: <57F38993.6050505@oracle.com>

Hi,

as suggested by Stuart Marks in the review thread for JDK-8166840, we
should add a comment as to why an empty constructor is important in the
private inner Itr class, to avoid it being accidentally "cleaned up".

Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
Bug: https://bugs.openjdk.java.net/browse/JDK-8167005

Thanks!

/Claes

From shade at redhat.com  Tue Oct  4 10:52:23 2016
From: shade at redhat.com (Aleksey Shipilev)
Date: Tue, 4 Oct 2016 12:52:23 +0200
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <57F38993.6050505@oracle.com>
References: <57F38993.6050505@oracle.com>
Message-ID: <b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>

On 10/04/2016 12:50 PM, Claes Redestad wrote:
> Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/

OK.

-Aleksey


From claes.redestad at oracle.com  Tue Oct  4 10:55:53 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 4 Oct 2016 12:55:53 +0200
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
Message-ID: <57F38AB9.70506@oracle.com>


On 2016-10-04 12:52, Aleksey Shipilev wrote:
> On 10/04/2016 12:50 PM, Claes Redestad wrote:
>> Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>
> OK.
>

Thanks for the speedy review! :-)

/Claes

From naoto.sato at oracle.com  Tue Oct  4 12:42:58 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 4 Oct 2016 05:42:58 -0700
Subject: [9] RFR: 8166645: Include locales plugin throws InternalError
	with "*" specified.
In-Reply-To: <3A8B2A45-96AA-42C2-B8F8-8803C3EAAF71@oracle.com>
References: <d6bf4d12-e904-f3bb-8383-dab6250c2f46@oracle.com>
	<431dcb9b-6dfc-6ee7-73ca-96c3b1a7f401@oracle.com>
	<3A8B2A45-96AA-42C2-B8F8-8803C3EAAF71@oracle.com>
Message-ID: <fb0c3cc4-20da-b2b6-0e53-33925ec60f21@oracle.com>

Thanks, Mandy. I updated it as you suggested.

http://cr.openjdk.java.net/~naoto/8166645/webrev.02/

Naoto

On 10/3/16 7:53 PM, Mandy Chung wrote:
>
>> On Oct 3, 2016, at 3:17 PM, Naoto Sato <naoto.sato at oracle.com> wrote:
>>
>> According to an internal review, I added some explanation to the modification. No logic modifications.
>>
>> http://cr.openjdk.java.net/~naoto/8166645/webrev.01/
>
> This looks okay.
>
> I suggest to keep Set<String> originals after splitting the input byte array such that
>   325             .filter(t -> original.indexOf(t) != -1)
>
> can be replaced with .filter(originals::contains)
>
> Mandy
>

From peter.levart at gmail.com  Tue Oct  4 13:40:30 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 4 Oct 2016 15:40:30 +0200
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent with
	Class.getMethods() results + more
Message-ID: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>

Hi,

I have a fix for conformance (P2) bug (8062389 
<https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a 
conformance (P3) bug (8029459 
<https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance 
issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):

http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/

As Daniel Smith writes in 8029459 
<https://bugs.openjdk.java.net/browse/JDK-8029459>, the following 
situation is as expected:

interface I { void m(); }
interface J extends I { void m(); }
interface K extends J {}
K.class.getMethods() = [ J.m ]

But the following has a different result although it should probably 
have the same:

interface I { void m(); }
interface J extends I { void m(); }
interface K extends I, J {}
K.class.getMethods() = [ I.m, J.m ]

He then suggests the following algorithm:

An implementation of getMethods consistent with JLS 8 would include the 
following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
1) The class's/interface's declared (public) methods
2) The getMethods() of the superclass (if this is a class), minus any 
that have a match in (1)
3) The getMethods() of each superinterface, minus any that have a match 
in (1) or a _concrete_ match in (2) or a match from a more-specific 
class/interface in (2) or (3)

But even that is not sufficient for the following situation:

interface E { void m(); }
interface F extends E { void m(); }
abstract class G implements E {}
abstract class H extends G implements F {}
H.class.getMethods() = [ E.m, F.m ]

The desired result of H.class.getMethods() = [ F.m ]

So a more precise definition is required which is implemented in the fix.

The getMethods() implementation partitions the union of the following 
methods:

1) The class's/interface's declared public methods
2) The getMethods() of the superclass (if this is a class)
3) The non-static getMethods() of each direct superinterface

...into equivalence classes (sets) of methods with same signature 
(return type, name, parameter types). Within each such set, only the 
"most specific" methods are kept and others are discarded. The union of 
the kept methods is the result.

The "more specific" is defined as a partial order within a set of 
methods of same signature:

Method A is more specific than method B if:
- A is declared by a class and B is declared by an interface; or
- A is declared by the same type as or a subtype of B's declaring type 
and both are either declared by classes or both by interfaces (clearly 
if A and B are declared by the same type and have the same signature, 
they are the same method)

If A and B are distinct elements from the set of methods with same 
signature, then either:
- A is more specific than B; or
- B is more specific than A; or
- A and B are incomparable

A sub-set of "most specific" methods are the methods from the set where 
for each such method M, there is no method N != M such that N is "more 
specific" than M.

There can be more than one "most specific" method for a particular 
signature as they can be inherited from multiple unrelated interfaces, but:
- javac prevents compilation when among multiply inherited methods with 
same signature there is at least one default method, so in practice, 
getMethods() will only return multiple methods with same signature if 
they are abstract interface methods. With one exception: bridge methods 
that are created by javac for co-variant overrides are default methods 
in interfaces. For example:

     interface I { Object m(); }
     interface J1 extends I { Map m(); }
     interface J2 extends I { HashMap m(); }
     interface K extends J1, J2 {}

K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, default 
Object j2.m, abstract HashMap j2.m ]

But this is an extreme case where one can still expect multiple 
non-abstract methods with same signature, but different declaring type, 
returned from getMethods().

In order to also fix 8062389 
<https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and 
getMethods() share the same consolidation logic that results in a set of 
"most specific" methods. The partitioning in getMethods() is partially 
performed by collecting Methods into a HashMap where the keys are (name, 
parameter types) tuples and values are linked lists of Method objects 
with possibly varying return and declaring types. The consolidation, 
i.e. keeping only the set of most specific methods as new methods are 
encountered, is performed only among methods in the list that share same 
return type and also removes duplicates. getMethod() uses only one such 
list, consolidates methods that match given name and parameter types and 
returns the 1st method from the resulting list that has the most 
specific return type.

That's it for algorithms used. As partitioning partially happens using 
HashMap with (name, parameter types) keys, lists of methods that form 
values are typically kept short with most of them containing only a 
single method, so this fix also fixes performance issue 8061950 
<https://bugs.openjdk.java.net/browse/JDK-8061950>.

The patch also contains some optimizations around redundant copying of 
arrays and reflective objects.

The FilterNotMostSpecific jtreg test has been updated to accommodate for 
changed behavior. Both of the above extreme cases have been added to the 
test.

So, comments, please.

Regards, Peter


From chris.hegarty at oracle.com  Tue Oct  4 13:54:48 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Tue, 4 Oct 2016 14:54:48 +0100
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <57F38AB9.70506@oracle.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
Message-ID: <476DF2CC-20C9-4F6B-B3EE-04236EB70490@oracle.com>


> On 4 Oct 2016, at 11:55, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> 
> On 2016-10-04 12:52, Aleksey Shipilev wrote:
>> On 10/04/2016 12:50 PM, Claes Redestad wrote:
>>> Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/

+1

-Chris.


From mandy.chung at oracle.com  Tue Oct  4 13:57:20 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 4 Oct 2016 06:57:20 -0700
Subject: [9] RFR: 8166645: Include locales plugin throws InternalError
	with "*" specified.
In-Reply-To: <fb0c3cc4-20da-b2b6-0e53-33925ec60f21@oracle.com>
References: <d6bf4d12-e904-f3bb-8383-dab6250c2f46@oracle.com>
	<431dcb9b-6dfc-6ee7-73ca-96c3b1a7f401@oracle.com>
	<3A8B2A45-96AA-42C2-B8F8-8803C3EAAF71@oracle.com>
	<fb0c3cc4-20da-b2b6-0e53-33925ec60f21@oracle.com>
Message-ID: <8D6D6E38-FB83-4CA2-BB81-97D58FFC306D@oracle.com>


> On Oct 4, 2016, at 5:42 AM, Naoto Sato <naoto.sato at oracle.com> wrote:
> 
> Thanks, Mandy. I updated it as you suggested.
> 
> http://cr.openjdk.java.net/~naoto/8166645/webrev.02/

+1

Mandy

From goetz.lindenmaier at sap.com  Tue Oct  4 16:48:53 2016
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Tue, 4 Oct 2016 16:48:53 +0000
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
Message-ID: <b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>

Hi Vladimir, 

This webrev contains all the changes to hotspot needed for the port:
http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/

It includes 
http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/
http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/
http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/
which are out for review. Further it includes
the one change to relocate the pc-relative instructions where we didn't open
a bug for yet, and the new s390-files.

Altogether this passed all our tests that were running on the weekend
on linuxs390.

The s390-files though are not yet fully in shape, I'm still editing them to get
rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded by 
#ifdef SAPJVM  will go away in the end.

I hope to have the final versions by end of this week.

Best regards,
  Goetz.


> -----Original Message-----
> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net]
> On Behalf Of Vladimir Kozlov
> Sent: Montag, 3. Oktober 2016 23:50
> To: Volker Simonis <volker.simonis at gmail.com>
> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source Developers
> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
> dev at openjdk.java.net>
> Subject: Re: RFR: JEP draft for Linux/s3990x port
> 
> Hi Volker,
> 
> Can you prepare combined patch (or set of patches) based on latest
> reviews together with s390 code as it will be in final push?
> 
> We want to run it through our pre-integration testing to verify that it
> does not have problems.
> 
> Thanks,
> Vladimir
> 
> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
> > You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
> > other) state:
> >
> > http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
> >
> > Vladimir
> >
> > On 9/29/16 9:55 AM, Volker Simonis wrote:
> >> Hi Vladimir,
> >>
> >> thanks a lot for reviewing and endorsing the JEP.
> >>
> >> I've linked all the relevant issues to the JEP  (they all have a link
> >> to a webrev) and change the state to "Submitted".
> >>
> >> There's just one more small shared change we need for the port for
> >> which we haven't opened a bug now because we are still working on
> >> simplifying it. The current version looks as follows:
> >>
> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
> constant_table_offset.patch
> >>
> >>
> >> What are the next steps? Should I add a "jdk9-fc-request" label to t
> >> he JEP and add a corresponding "FC Extension Request" comment to it?
> >> Or will this be done automatically once I move it to "Candidate"?
> >>
> >> Is there anything left to do before I can move it to "Candidate" state?
> >>
> >> Thanks a lot and best regards,
> >> Volker
> >>
> >>
> >>
> >>
> >> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
> >> <vladimir.kozlov at oracle.com> wrote:
> >>> On 9/27/16 10:49 AM, Volker Simonis wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> can you please review and endorse the following draft JEP for the
> >>>> integration of the Linux/s390x port into the jkd9 master repository:
> >>>>
> >>>> https://bugs.openjdk.java.net/browse/JDK-8166730
> >>>
> >>>
> >>> Good.
> >>> Add links to webrevs in a comment. It will help to get umbrella FC
> >>> extension
> >>> approval.
> >>>
> >>>>
> >>>> As detailed in the JEP, the Linux/s390x requires very few shared
> >>>> changes and we therefore don't foresee any impact on the existing
> >>>> platforms at all. Following you can find a short description of the
> >>>> planned changes:
> >>>>
> >>>> hotspot:
> >>>> =======
> >>>>
> >>>> Out for review:
> >>>> 8166560: [s390] Basic enablement of s390 port.
> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> basic_s390/hotspot.wr01/
> >>>>
> >>>> Reviewed:
> >>>> 8166562: C2: Suppress relocations in scratch emit.
> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> scratch_emit/webrev.01/
> >>>>
> >>>> Will send RFR soon (depends on 8166560):
> >>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> scratch_emit/webrev.01
> >>>
> >>>
> >>> Wrong link.
> >>>
> >>> Thanks,
> >>> Vladimir
> >>>
> >>>
> >>>>
> >>>> We are still investigating the need of these shared changes:
> >>>>
> >>>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> 011-pass_PC_to_retAddrOffset.patch
> >>>>
> >>>>
> >>>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> 016-constant_table_offset.patch
> >>>>
> >>>>
> >>>> And finally the patch with the s390x-only platform files. We are still
> >>>> editing these to get them into OpenJdk style and shape.
> >>>> Hotspot passes most jck, jtreg and spec tests with these.
> >>>>
> >>>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> 101-zFiles.patch
> >>>>
> >>>>
> >>>> top-level repository:
> >>>> ===============
> >>>>
> >>>> The following is just adding some s390x specific compiler flags to
> >>>> flags.m4
> >>>> 8166800: [s390] Top-level build changes required for Linux/s390x
> >>>> https://bugs.openjdk.java.net/browse/JDK-8166800
> >>>>
> >>>> jdk repository:
> >>>> ============
> >>>>
> >>>> This one just adds a new jvm.cfg file for s390x
> >>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
> >>>> https://bugs.openjdk.java.net/browse/JDK-8166801
> >>>>
> >>>>
> >>>> And finally we plan to do one more change which fixes the jtreg test
> >>>> on Linux/s390x. But this is mainly for the correct detection of the
> >>>> platform and for excluding the tests which are not appropriate for
> >>>> s390x.
> >>>>
> >>>> Thank you and best regards,
> >>>> Volker
> >>>>
> >>>

From goetz.lindenmaier at sap.com  Tue Oct  4 16:52:04 2016
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Tue, 4 Oct 2016 16:52:04 +0000
Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port.
In-Reply-To: <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com>
References: <b384c60b06394f5aaf9a11376899a0aa@DEWDFE13DE50.global.corp.sap>
	<e3963e6f-2dec-f00b-0489-67c4d7822959@oracle.com>
	<CA+3eh13Ca5O=f_+CUHdguP35ZMqWsuEVum+mtbZ5+9xPJDaO3w@mail.gmail.com>
	<4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap>
	<41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com>
Message-ID: <c70a1c0fe0df4bb0bd84a6a3c13f446a@DEWDFE13DE50.global.corp.sap>

Hi David,

I fixed the change accordingly:
http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/

But do you really want me to incorporate a bugfix for ARM in the port?

Best regards,
  Goetz.

> -----Original Message-----
> From: David Holmes [mailto:david.holmes at oracle.com]
> Sent: Dienstag, 4. Oktober 2016 09:56
> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Volker Simonis
> <volker.simonis at gmail.com>
> Cc: hotspot-dev at openjdk.java.net; core-libs-dev <core-libs-
> dev at openjdk.java.net>
> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port.
> 
> Hi Goetz,
> 
> On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote:
> > Hi,
> >
> > here a new webrev for this change:
> > http://cr.openjdk.java.net/~goetz/wr16/8166560-
> basic_s390/hotspot.wr02/
> >
> > I reverted the major reorderings in macros.hpp and os_linux.cpp.
> > David asked me to do so, and I guess it makes reviewing more simple.
> 
> Thanks. That said ...
> 
> > Also this fixes the issue spotted by David which actually was wrong.
> > The other renaming of ARM to ARM32 was correct, though, as
> > AARCH64 is defined in both ARM 64-bit ports, and is checked before.
> > So the existing case checking ARM is only reached if !LP64.
> > I documented this ...
> 
> ... We actually have a bug with the Elf32_* defines in os_linux.cpp
> 
> 1769 #elif  (defined ARM) // ARM must come before AARCH64 because the
> closed 64-bit port requires so.
> 1770   static  Elf32_Half running_arch_code=EM_ARM;
> 
> Aarch64 should come first otherwise we will set the wrong value. I've
> been told it would only affect an error message but still ... can I ask
> you to fix this please. Thanks.
> 
> ---
> src/share/vm/code/codeCache.cpp
> 
> I'd prefer to just see something like:
> 
> S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is
> used
> 
> ---
> 
> Otherwise seems okay.
> 
> Thanks,
> David
> 
> > Also I removed the change in mutex.hpp.  Maybe we contribute
> > the full change this was part of, but independent of the s390 port.
> >
> > I withdraw the part of the change to jdk introducing jvm.cfg.  Volker wants
> to
> > do a separate change for this.
> >
> > Best regards,
> >   Goetz.
> >
> >
> >
> >> -----Original Message-----
> >> From: Volker Simonis [mailto:volker.simonis at gmail.com]
> >> Sent: Dienstag, 27. September 2016 19:58
> >> To: David Holmes <david.holmes at oracle.com>
> >> Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; hotspot-
> >> dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.net>
> >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port.
> >>
> >> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes
> <david.holmes at oracle.com>
> >> wrote:
> >>> Hi Goetz,
> >>>
> >>> I see a change not related directly to S390 ie change from ARM to ARM32
> in
> >>> src/os/linux/vm/os_linux.cpp
> >>>
> >>
> >> The change looks a little confusing because Goetz reordered the ifdef
> >> cascades alphabetically (which I think is good).
> >>
> >> Besides that, the only real change not related to s390 is indeed the
> >> change from ARM to ARM32 which happend two times in the file.
> >>
> >> @Goetz: have you done this intentionally?
> >>
> >>> It will be a while before I can go through this in any detail.
> >>>
> >>> David
> >>>
> >>>
> >>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> This change is part of the s390 port. It contains some basic adaptions
> >>>> needed for a full hotspot port for linux s390x.
> >>>>
> >>>> It defines the required macros, platform names and includes.
> >>>>
> >>>> The s390 port calles CodeCache::contains() in current_frame(), which is
> >>>> used in NMT. As NMT already collects stack traces before the
> CodeCache
> >> is
> >>>> initialized, contains() needs a check for this.
> >>>>
> >>>>  Wherever a row of platforms are listed, I sorted them alphabetically.
> >>>>
> >>>>  The jdk requires the file jvm.cfg.
> >>>>
> >>>> Please review. I please need a sponsor.
> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> >> basic_s390/hotspot.wr01/
> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> basic_s390/jdk.wr01/
> >>>>
> >>>> Best regards,
> >>>>   Goetz.
> >>>>
> >>>

From yingqi.lu at intel.com  Tue Oct  4 17:24:21 2016
From: yingqi.lu at intel.com (Lu, Yingqi)
Date: Tue, 4 Oct 2016 17:24:21 +0000
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>

Hi Christoph,

Thank you very much for trying our patch. 

We are still seeking the feedback from the community. When we get closer to the final version of the patch, we will modify the copyright years. Thank you for reminding us!

Anyone else has any feedback/comments?

Thanks,
Lucy

-----Original Message-----
From: Langer, Christoph [mailto:christoph.langer at sap.com] 
Sent: Tuesday, October 04, 2016 1:33 AM
To: Lu, Yingqi <yingqi.lu at intel.com>; Alan Bateman <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net
Cc: nio-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>; Kharbas, Kishor <kishor.kharbas at intel.com>
Subject: RE: Proposal for adding O_DIRECT support into JDK 9

Hi Lucy,

FWIW: I ran a build on AIX and this looks ok.

I also assume in your final version you'll update all copyright years where it's not 2016 yet? Other than that the changes look ok to me - but I'm neither a reviewer nor a deep expert in the area of your changes.

Best regards
Christoph

> -----Original Message-----
> From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of 
> Lu, Yingqi
> Sent: Freitag, 30. September 2016 18:55
> To: Lu, Yingqi <yingqi.lu at intel.com>; Alan Bateman 
> <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net; Kaczmarek, Eric 
> <eric.kaczmarek at intel.com>; Kharbas, Kishor <kishor.kharbas at intel.com>
> Subject: RE: Proposal for adding O_DIRECT support into JDK 9
> 
> Hi All,
> 
> Please find the most recent version of the patch available at 
> http://cr.openjdk.java.net/~igraves/8164900-2/
> 
> In this version, we have following two changes:
> 
> 1. Move O_DIRECT flag from StandardOpenOption to ExtendedOpenOption 2. 
> Move the checks of O_DIRECT from native code to Java code.
> 
> Please let us know your feedback.
> 
> Thanks,
> Lucy
> 
> -----Original Message-----
> From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of 
> Lu, Yingqi
> Sent: Tuesday, September 27, 2016 9:57 AM
> To: Alan Bateman <Alan.Bateman at oracle.com>; core-libs- 
> dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net
> Subject: RE: Proposal for adding O_DIRECT support into JDK 9
> 
> Alan,
> 
> Thank you for the explanation, we will modify the code accordingly and 
> send it out soon for review.
> 
> Thanks,
> Lucy
> 
> -----Original Message-----
> From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
> Sent: Tuesday, September 27, 2016 8:45 AM
> To: Lu, Yingqi <yingqi.lu at intel.com>; core-libs-dev at openjdk.java.net
> Cc: nio-dev at openjdk.java.net
> Subject: Re: Proposal for adding O_DIRECT support into JDK 9
> 
> On 26/09/2016 19:50, Lu, Yingqi wrote:
> 
> > Alan, you mean readv0/write0 or read0/write0? I just want to make 
> > sure
> > :-)
> Apologies, I meant each of the native methods where the decision to 
> use direct I/O or not would be a lot more maintainable in Java. You'll 
> see that there are already code paths for direct vs. heap buffers.
> 
> 
> >
> > Anyone else has other opinions on where is the best home for O_DIRECT flag?
> The flags under jdk.unsupported will eventually be removed in the 
> future JDK release?
> >
> > If we agree ExtendedOpenOpen is the best home for O_DIRECT, we can
> modify that for sure.
> >
> I think ExtendedOpenOption is the right place. It's still TDB as to 
> whether to put these extensions but that should be transparent to 
> anyone using this when on the class path.
> 
> -Alan

From stuart.marks at oracle.com  Tue Oct  4 20:28:47 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 4 Oct 2016 13:28:47 -0700
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <57F38AB9.70506@oracle.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
Message-ID: <66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>



On 10/4/16 3:55 AM, Claes Redestad wrote:
>
> On 2016-10-04 12:52, Aleksey Shipilev wrote:
>> On 10/04/2016 12:50 PM, Claes Redestad wrote:
>>> Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>>
>> OK.
>>
> Thanks for the speedy review! :-)

Thanks for looking at this. The shorter text in the bug report is ok too.

s'marks


From forax at univ-mlv.fr  Tue Oct  4 20:32:01 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 4 Oct 2016 22:32:01 +0200 (CEST)
Subject: Deprecate all java.util.concurrent.*FieldUpdater
Message-ID: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>

Given that Java 9 introduces a faster way to emit things like compareAndSet by using the VarHandke API and that AtomiReference (and likes) are now rewritten to use VarHandles directly,
i think it's time to deprecate all *FieldUpdater with something saying that they have been superseded by the VarHandle API.

R?mi
substitute dr deprecator

From martinrb at google.com  Tue Oct  4 21:19:33 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 4 Oct 2016 14:19:33 -0700
Subject: Deprecate all java.util.concurrent.*FieldUpdater
In-Reply-To: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
Message-ID: <CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>

VarHandle is a reasonable replacement for FieldUpdaters, but it's not yet
complete (where is accumulateAndGet?), and when do you deprecate something
when the replacement won't be ubiquitous for 10 years?

On Tue, Oct 4, 2016 at 1:32 PM, Remi Forax <forax at univ-mlv.fr> wrote:

> Given that Java 9 introduces a faster way to emit things like
> compareAndSet by using the VarHandke API and that AtomiReference (and
> likes) are now rewritten to use VarHandles directly,
> i think it's time to deprecate all *FieldUpdater with something saying
> that they have been superseded by the VarHandle API.
>
> R?mi
> substitute dr deprecator
>

From jbluettduncan at gmail.com  Tue Oct  4 21:20:59 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Tue, 4 Oct 2016 22:20:59 +0100
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
	<66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>
Message-ID: <CAPMys849wiV07AvrLqNBNPbo5kaOgLcA53rxX8znpsSkB0__Kw@mail.gmail.com>

The explanation which Stuart gives for this change in
https://bugs.openjdk.java.net/browse/JDK-8167005 seems to describe why this
constructor is needed much better than the comment itself does. So I wonder
if it's worth adding the link to the bug report in the comment. E.g.

// prevent generation of synthetic class required for access to private
// constructor. See: https://bugs.openjdk.java.net/browse/JDK-8167005

Kind regards,
Jonathan

On 4 October 2016 at 21:28, Stuart Marks <stuart.marks at oracle.com> wrote:

>
>
> On 10/4/16 3:55 AM, Claes Redestad wrote:
>
>>
>> On 2016-10-04 12:52, Aleksey Shipilev wrote:
>>
>>> On 10/04/2016 12:50 PM, Claes Redestad wrote:
>>>
>>>> Webrev: http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>>>>
>>>
>>> OK.
>>>
>>> Thanks for the speedy review! :-)
>>
>
> Thanks for looking at this. The shorter text in the bug report is ok too.
>
> s'marks
>
>

From martinrb at google.com  Tue Oct  4 21:27:55 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 4 Oct 2016 14:27:55 -0700
Subject: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
In-Reply-To: <68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>
References: <44b5d97a-222a-4d46-8896-8a2115108081@default>
	<CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>
	<68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>
Message-ID: <CA+kOe0-NNcbLi6C+1RFfx_ExfmJtnvLH_vBGSkuaK5WiJRH2OA@mail.gmail.com>

Looks good to me!



On Tue, Oct 4, 2016 at 3:22 AM, Ramanand Patil <ramanand.patil at oracle.com>
wrote:

> Hi Martin,
> Thank you for your review and explanation of "Yangon". I liked the
> translation "End of Strife".
>
> Looking at the description of the ZoneNames.java:
> * The zid<->metazone mappings are based on CLDR metaZones.xml.
>  * The alias mappings are based on Link entries in tzdb data files.
>
> I had thought to not update this file because the CLDR metaZones.xml file
> doesn?t have this entry updated.
> But I think you are correct, since Link entry has this alias mentioned,
> there is no harm in adding these entries to zidMap and aliasMap arrays.
> Here is the updated Webrev: http://cr.openjdk.java.net/~
> rpatil/8166875/webrev.01/
>
> Changes done:
>         - Updated src/java.base/share/classes/java/time/format/ZoneName.java
> to include "Yangon" entry.
>         - Removed unused imports from src/java.base/share/classes/
> java/time/format/ZoneName.java
>         - Updated ZoneName.java in the test package as well to include
> "Yangon". [test/java/time/test/java/time/format/ZoneName.java]
>         - Updated the bugID for test/java/time/test/java/time/format/TestZoneTextPrinterParser.java
> since this uses the "ZoneName.java" defined in test package.
>
> Also, looks like ZoneName.java is trying to maintain a comprehensive list
> of zone names. Though I found very few zone names are missing from this
> file like: "Europe/Busingen", "America/Fort_Nelson", "Antarctica/Troll"
> etc...
>
>
> Regards,
> Ramanand.
>
> From: Martin Buchholz [mailto:martinrb at google.com]
> Sent: Monday, October 03, 2016 8:55 PM
> To: Ramanand Patil <ramanand.patil at oracle.com>
> Cc: i18n-dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.
> net>
> Subject: Re: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
>
> Hi Ramanand,
> Pleased to meet you!
>
> I expected to see Yangon added to ZoneName, because of the existing
> reference to Rangoon
>
> java/time/test/java/time/format/ZoneName.java:179:        "Asia/Rangoon",
> "Myanmar", "Asia/Rangoon",
>
> Is ZoneName.java trying to maintain a comprehensive list of zone names?
>
> """Yangon (???????) is a combination of the two words yan (???) and koun
> (????), which mean "enemies" and "run out of", respectively. It is also
> translated as "End of Strife"."""
>
>
> On Mon, Oct 3, 2016 at 5:27 AM, Ramanand Patil <mailto:
> ramanand.patil at oracle.com> wrote:
> HI all,
> Please review the latest TZDATA integration (tzdata2016g) to JDK9.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
> Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/
>
> All the TimeZone related tests are passed after integration.
> [BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since
> they verify the renamed TZID "Asia/Yangon"].
>
> Regards,
> Ramanand.
>

From stuart.marks at oracle.com  Tue Oct  4 22:27:27 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 4 Oct 2016 15:27:27 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87pqoBtRQwPSvR-54e_sX33VgmcJWtmMJrZsBKVjWi1rg@mail.gmail.com>
	<2F084007-7660-4B2F-8B11-7B99EC0B9718@reini.net>
	<CAPMys851GThU6XiUsZY+2j=3WRqtYYHFFJPauQbrHvjMxoj-Lg@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
Message-ID: <a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>

On 9/30/16 6:57 AM, Jonathan Bluett-Duncan wrote:
> 1) It actually didn't occur to me that there was a potential TOCTOU problem in 
> ModuleFinder.compose, despite the fact that I found a potential problem in 
> FileTreeIterator - it completely missed me, so thank you for finding it! I'll 
> see if I can put a similar comment there to what I wrote in FileTreeIterator.
OK. A simple comment such as

     // make a copy since it's captured by the ModuleFinder instance belowfinal List<ModuleFinder> finderList = List.of(finders);

should be sufficient.
> 2) ... I decided just now to do a search on Grepcode for usages of 
> CookieManager.get 
> <http://grepcode.com/search/usages?type=method&id=repository.grepcode.com%24java%24root at jdk%24openjdk at 8u40-b25@java%24net at CookieManager@get%28java.net.URI%2Cjava.util.Map%29&k=u> and 
> CookieHandler.get 
> <http://grepcode.com/search/usages?id=repository.grepcode.com$java$root at jdk$openjdk at 8u40-b25@java$net at CookieHandler@get%28java.net.URI,java.util.Map%29&start=0&r=&type=method&k=u>, 
> and curiously it looks like they're only used in sun classes in the JDK. So 
> this change seems safe to me, unless I've missed something?
While we're looking at CookieManager.java, please fix this typo 
"NetscapeCookieSotre".

   84  *     by a more sophisticated CookieStore implementation, e.g. a NetscapeCookieSotre.

So, grepcode is kind of hard to work with. I've found that when doing "Find 
Usages" for an API, the results are relative to the JDK version you're looking 
at. Apparently at the time grepcode's indexes were run, most of those other 
libraries weren't building against any version of JDK 8. But if you start the 
search at (say) JDK6 b27, and then go to java.net.CookieHandler.get() and do 
"Find usages", you get a bunch of hits:

http://grepcode.com/search/usages?type=method&id=repository.grepcode.com%24java%24root at jdk%24openjdk at 6-b27@java%24net at CookieHandler@get%28java.net.URI%2Cjava.util.Map%29&k=u

Same with 6-b27 CookieManager.get():

http://grepcode.com/search/usages?type=method&id=repository.grepcode.com%24java%24root at jdk%24openjdk at 6-b27@java%24net at CookieManager@get%28java.net.URI%2Cjava.util.Map%29&k=u

7u40-b43 CookieHandler.get():

http://grepcode.com/search/usages?type=method&id=repository.grepcode.com%24java%24root at jdk%24openjdk at 7u40-b43@java%24net at CookieHandler@get%28java.net.URI%2Cjava.util.Map%29&k=u

7u40-b43 CookieManager.get():

http://grepcode.com/search/usages?type=method&id=repository.grepcode.com%24java%24root at jdk%24openjdk at 7u40-b43@java%24net at CookieManager@get%28java.net.URI%2Cjava.util.Map%29&k=u

I don't know a way to search for the usage of an API across all versions of the 
JDK. It's moderately painful to have to do the same searches for several 
different JDK versions.

Anyway, I spent some time looking through usage hits from the links above. It's 
hard to know how many there are; you just keep hitting "Next" until there aren't 
any more. (Is there a way to get statistics instead of just page after page of 
matches? I don't know.) There are lots of duplicates -- different versions of 
the same library -- so you can skip past those pretty quickly.

After a while some patterns begin to emerge. For this API, the Map that's 
returned is almost always iterated over looking for "Cookie" and "Cookie2" 
entries, or being probed directly for those entries, and is then thrown away. In 
one case it was stored in a private field of a non-serializable class, but the 
only usage of that field was to iterate over it in a manner similar to other cases.

I encourage you to look through these usages as well.

In any case, it's fairly clear to me that the most common use of the returned 
Map is to read stuff out of it immediately. This isn't definitive, obviously, 
but it does look like this is the most common usage pattern. Based on this I 
think the serialization difference won't be a problem, and that it's OK to 
proceed with this change.
> 3) In my local copy of jdk9, I've removed the TOCTOU comment in 
> FileTreeIterator and changed List.of back to Arrays.asList, as your 
> explanation regarding FileTreeWalker has convinced me that TOCTOU is not a 
> real problem here....
OK
> 4) The 'resolverFields' problem/comment was regarding DateTimeFormatter (see 
> this part of latest webrev 
> <http://cr.openjdk.java.net/%7Ereinhapa/reviews/8134373/webrev.01/src/java.base/share/classes/java/time/format/DateTimeFormatter.java.cdiff.html>), 
> where I realised I couldn't use Set.of instead of 
> Collections.unmodifiableSet(new HashSet<>(Arrays.asList(*))), because I 
> noticed that one of the java.time tests was testing whether 
> DateTimeFormatter.withResolverFields(TemporalField...) could accept null 
> parameters, which made using Set.of impossible since it's null-hostile (as in 
> it throws NullPointerException upon being constructed with null element(s)).
Hm, yes, it's odd that there's a test for an array containing a null, in 
addition to an empty array and a null array. See:

jdk/test/java/time/tck/java/time/format/TCKDateTimeFormatter.java
test_resolverFields_listOfOneNull()

I'm not entirely sure what's intended here. In any case, let's wait until we 
hear from Mr. Colebourne.

I looked at the rest of the files and the only questions I have are other places 
in the java.time files.

5) For the changes in the Chronology classes, the era() method now returns an 
immutable array where it didn't before. (The List returned by Arrays.asList() 
can have individual elements modified but its size can't be changed.) The spec 
for eras() says "may be immutable" so presumably this is OK. But note, since the 
returned List is now immutable, a new instance needn't be created each time. I'm 
not sure whether it's worth keeping around a cached copy in case someone calls 
eras() again though.

It would be good if we could hear from Stephen on this one as well.

s'marks


From scolebourne at joda.org  Tue Oct  4 23:10:21 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 5 Oct 2016 00:10:21 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87pqoBtRQwPSvR-54e_sX33VgmcJWtmMJrZsBKVjWi1rg@mail.gmail.com>
	<2F084007-7660-4B2F-8B11-7B99EC0B9718@reini.net>
	<CAPMys851GThU6XiUsZY+2j=3WRqtYYHFFJPauQbrHvjMxoj-Lg@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
Message-ID: <CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>

On 4 October 2016 at 23:27, Stuart Marks <stuart.marks at oracle.com> wrote:
> 4) The 'resolverFields' problem/comment was regarding DateTimeFormatter (see
> this part of latest webrev), where I realised I couldn't use Set.of instead
> of Collections.unmodifiableSet(new HashSet<>(Arrays.asList(*))), because I
> noticed that one of the java.time tests was testing whether
> DateTimeFormatter.withResolverFields(TemporalField...) could accept null
> parameters, which made using Set.of impossible since it's null-hostile (as
> in it throws NullPointerException upon being constructed with null
> element(s)).
>
> Hm, yes, it's odd that there's a test for an array containing a null, in
> addition to an empty array and a null array. See:
>
> jdk/test/java/time/tck/java/time/format/TCKDateTimeFormatter.java
> test_resolverFields_listOfOneNull()
>
> I'm not entirely sure what's intended here. In any case, let's wait until we
> hear from Mr. Colebourne.

I don't think that is a sensible test. AFAICT, null in the set has no
meaning. Its not documented to have meaning, so it is really a bug
with a test.

> 5) For the changes in the Chronology classes, the era() method now returns
> an immutable array where it didn't before. (The List returned by
> Arrays.asList() can have individual elements modified but its size can't be
> changed.) The spec for eras() says "may be immutable" so presumably this is
> OK. But note, since the returned List is now immutable, a new instance
> needn't be created each time. I'm not sure whether it's worth keeping around
> a cached copy in case someone calls eras() again though.
>
> It would be good if we could hear from Stephen on this one as well.

eras() will rarely be used, so no point caching. Changing to List.of() is fine.

Stephen

From jonathan.gibbons at oracle.com  Tue Oct  4 23:39:47 2016
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Tue, 04 Oct 2016 16:39:47 -0700
Subject: RFR: 8159855
Message-ID: <57F43DC3.8040905@oracle.com>

Core-libs folk,

Please review the following change to add a new service provider class
     java.util.spi.ToolProvider

which can be used provide simple "command-line" access to select JDK
tools, without starting a new JVM.

The following tools are updated to provide access through the new SPI:
     javac, javadoc, javap, jdeps

It is expected that additional tools will also be updated to provide access,
but that will be done separately.

Compiler-dev folk may wish to review the changes to the langtools 
repository.

JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
API: 
http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html

-- Jon

From jonathan.gibbons at oracle.com  Tue Oct  4 23:46:11 2016
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Tue, 04 Oct 2016 16:46:11 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <57F43DC3.8040905@oracle.com>
References: <57F43DC3.8040905@oracle.com>
Message-ID: <57F43F43.3080707@oracle.com>

Resend with non-mostly-empty subject line!

-- Jon

On 10/04/2016 04:39 PM, Jonathan Gibbons wrote:
> Core-libs folk,
>
> Please review the following change to add a new service provider class
>     java.util.spi.ToolProvider
>
> which can be used provide simple "command-line" access to select JDK
> tools, without starting a new JVM.
>
> The following tools are updated to provide access through the new SPI:
>     javac, javadoc, javap, jdeps
>
> It is expected that additional tools will also be updated to provide 
> access,
> but that will be done separately.
>
> Compiler-dev folk may wish to review the changes to the langtools 
> repository.
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
> API: 
> http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
>
> -- Jon


From claes.redestad at oracle.com  Wed Oct  5 00:00:01 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 5 Oct 2016 02:00:01 +0200
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <CAPMys849wiV07AvrLqNBNPbo5kaOgLcA53rxX8znpsSkB0__Kw@mail.gmail.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
	<66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>
	<CAPMys849wiV07AvrLqNBNPbo5kaOgLcA53rxX8znpsSkB0__Kw@mail.gmail.com>
Message-ID: <57F44281.8060502@oracle.com>

Hi Jonathan,

the aim isn't to add an in-depth explanation to the code about exactly
the circumstance that led to this constructor and comment being added,
but to put a clear message that it was simply, in fact, deliberate, so
even the proposed comment might be going further than strictly necessary.

I'm also not convinced of the value of putting explicit links to the
bug actually pushed, since there's an implicit link in the commit
itself anyhow.

Thanks!

/Claes

On 2016-10-04 23:20, Jonathan Bluett-Duncan wrote:
> The explanation which Stuart gives for this change in
> https://bugs.openjdk.java.net/browse/JDK-8167005 seems to describe why
> this constructor is needed much better than the comment itself does. So
> I wonder if it's worth adding the link to the bug report in the comment.
> E.g.
>
> // prevent generation of synthetic class required for access to private
> // constructor. See: https://bugs.openjdk.java.net/browse/JDK-8167005
>
> Kind regards,
> Jonathan
>
> On 4 October 2016 at 21:28, Stuart Marks <stuart.marks at oracle.com
> <mailto:stuart.marks at oracle.com>> wrote:
>
>
>
>     On 10/4/16 3:55 AM, Claes Redestad wrote:
>
>
>         On 2016-10-04 12:52, Aleksey Shipilev wrote:
>
>             On 10/04/2016 12:50 PM, Claes Redestad wrote:
>
>                 Webrev:
>                 http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>                 <http://cr.openjdk.java.net/~redestad/8167005/webrev.01/>
>
>
>             OK.
>
>         Thanks for the speedy review! :-)
>
>
>     Thanks for looking at this. The shorter text in the bug report is ok
>     too.
>
>     s'marks
>
>

From brian.burkhalter at oracle.com  Wed Oct  5 00:06:06 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Tue, 4 Oct 2016 17:06:06 -0700
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
Message-ID: <45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>

Hi Lucy,

On Oct 4, 2016, at 10:24 AM, Lu, Yingqi <yingqi.lu at intel.com> wrote:

> Anyone else has any feedback/comments?


I looked over the 8164900-2 patch and did not see any conceptual problems per se. I did not however scrutinize all the detailed changes, especially the more extensive ones in FileDispatcherImpl.c. I did however run the patch through our regression tests for IO and NIO and encountered some problems. Firstly the patch breaks the build on OS X [1], Solaris [2], and Windows [3]. Note that the code under src/java.base/unix is built on all Unix systems including Linux, OS X, and Solaris. Secondly but of lesser note there were some warnings during the Linux build [4]. Otherwise the IO and NIO tests succeeded on Linux, but that only suggests that pre-existing functionality is preserved as those tests of course do not exercise the new code for which I suppose tests will be provided in the final version of the patch.

Regards,

Brian

[1] OS X
build/macosx-x64/support/gensrc/java.base/sun/nio/fs/UnixConstants.java:41: error: self-reference in initializer
    static final int O_DIRECT = O_DIRECT;
                                ^
[2] Solaris
build/solaris-x64/support/gensrc/java.base/sun/nio/fs/UnixConstants.java:41: error: self-reference in initializer
    static final int O_DIRECT = O_DIRECT;
                                ^
[3] Windows
jdk\src\java.base\windows\classes\java\io\FileDescriptor.java:76: error: <anonymous java.io.FileDescriptor$1> is not abstract and does not override abstract method getDirect(FileDescriptor) in JavaIOFileDescriptorAccess
            new JavaIOFileDescriptorAccess() {
                                             ^
[4] Linux (Ubuntu 16.04 amd64 desktop)

jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function ?readDirect?:
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:104:5: warning: ignoring return value of ?posix_memalign?, declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, newLen);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function ?writeDirect?:
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:131:5: warning: ignoring return value of ?posix_memalign?, declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, len);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function ?Java_sun_nio_ch_FileDispatcherImpl_readvDirect0?:
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:250:5: warning: ignoring return value of ?posix_memalign?, declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, newLen);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function ?Java_sun_nio_ch_FileDispatcherImpl_writevDirect0?:
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:393:5: warning: ignoring return value of ?posix_memalign?, declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, totalLen)


From yingqi.lu at intel.com  Wed Oct  5 00:35:04 2016
From: yingqi.lu at intel.com (Lu, Yingqi)
Date: Wed, 5 Oct 2016 00:35:04 +0000
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>

Hi Brain,

Thank you very much for reviewing the patch and providing the detailed error/warning messages. They are very helpful!

We will look into the issues and solve them in the next version of the patch. Test cases for O_DIRECT and modified copyright year will be added in next version too.

Thanks,
Lucy


From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com]
Sent: Tuesday, October 04, 2016 5:06 PM
To: Lu, Yingqi <yingqi.lu at intel.com>
Cc: Langer, Christoph <christoph.langer at sap.com>; Alan Bateman <Alan.Bateman at oracle.com>; core-libs-dev at openjdk.java.net; nio-dev at openjdk.java.net; Kharbas, Kishor <kishor.kharbas at intel.com>; Kaczmarek, Eric <eric.kaczmarek at intel.com>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9

Hi Lucy,

On Oct 4, 2016, at 10:24 AM, Lu, Yingqi <yingqi.lu at intel.com<mailto:yingqi.lu at intel.com>> wrote:


Anyone else has any feedback/comments?

I looked over the 8164900-2 patch and did not see any conceptual problems per se. I did not however scrutinize all the detailed changes, especially the more extensive ones in FileDispatcherImpl.c. I did however run the patch through our regression tests for IO and NIO and encountered some problems. Firstly the patch breaks the build on OS X [1], Solaris [2], and Windows [3]. Note that the code under src/java.base/unix is built on all Unix systems including Linux, OS X, and Solaris. Secondly but of lesser note there were some warnings during the Linux build [4]. Otherwise the IO and NIO tests succeeded on Linux, but that only suggests that pre-existing functionality is preserved as those tests of course do not exercise the new code for which I suppose tests will be provided in the final version of the patch.

Regards,

Brian

[1] OS X

build/macosx-x64/support/gensrc/java.base/sun/nio/fs/UnixConstants.java:41: error: self-reference in initializer

    static final int O_DIRECT = O_DIRECT;

                                ^
[2] Solaris

build/solaris-x64/support/gensrc/java.base/sun/nio/fs/UnixConstants.java:41: error: self-reference in initializer

    static final int O_DIRECT = O_DIRECT;

                                ^
[3] Windows

jdk\src\java.base\windows\classes\java\io\FileDescriptor.java:76: error: <anonymous java.io.FileDescriptor$1> is not abstract and does not override abstract method getDirect(FileDescriptor) in JavaIOFileDescriptorAccess

            new JavaIOFileDescriptorAccess() {

                                             ^
[4] Linux (Ubuntu 16.04 amd64 desktop)

jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function 'readDirect':
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:104:5: warning: ignoring return value of 'posix_memalign', declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, newLen);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function 'writeDirect':
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:131:5: warning: ignoring return value of 'posix_memalign', declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, len);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function 'Java_sun_nio_ch_FileDispatcherImpl_readvDirect0':
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:250:5: warning: ignoring return value of 'posix_memalign', declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, newLen);
     ^
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c: In function 'Java_sun_nio_ch_FileDispatcherImpl_writevDirect0':
jdk/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c:393:5: warning: ignoring return value of 'posix_memalign', declared with attribute warn_unused_result [-Wunused-result]
     posix_memalign(&bytes, pageSize, totalLen)


From stuart.marks at oracle.com  Wed Oct  5 00:57:23 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 4 Oct 2016 17:57:23 -0700
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <57F44281.8060502@oracle.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
	<66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>
	<CAPMys849wiV07AvrLqNBNPbo5kaOgLcA53rxX8znpsSkB0__Kw@mail.gmail.com>
	<57F44281.8060502@oracle.com>
Message-ID: <50062775-6261-5811-6a8c-e3274180c8f7@oracle.com>

Right, the main point of the comment is to tell the reader the constructor isn't 
superfluous, to prevent it from being cleaned up and accidentally causing a 
regression. Full history can reside in the commit comment, the bug database, and 
in these email logs.

I'd put in a link to a bug only when there's some action on this code associated 
with that bug, e.g., "don't remove this code until bug XXXXXXX has been fixed."

s'marks

On 10/4/16 5:00 PM, Claes Redestad wrote:
> Hi Jonathan,
>
> the aim isn't to add an in-depth explanation to the code about exactly
> the circumstance that led to this constructor and comment being added,
> but to put a clear message that it was simply, in fact, deliberate, so
> even the proposed comment might be going further than strictly necessary.
>
> I'm also not convinced of the value of putting explicit links to the
> bug actually pushed, since there's an implicit link in the commit
> itself anyhow.
>
> Thanks!
>
> /Claes
>
> On 2016-10-04 23:20, Jonathan Bluett-Duncan wrote:
>> The explanation which Stuart gives for this change in
>> https://bugs.openjdk.java.net/browse/JDK-8167005 seems to describe why
>> this constructor is needed much better than the comment itself does. So
>> I wonder if it's worth adding the link to the bug report in the comment.
>> E.g.
>>
>> // prevent generation of synthetic class required for access to private
>> // constructor. See: https://bugs.openjdk.java.net/browse/JDK-8167005
>>
>> Kind regards,
>> Jonathan
>>
>> On 4 October 2016 at 21:28, Stuart Marks <stuart.marks at oracle.com
>> <mailto:stuart.marks at oracle.com>> wrote:
>>
>>
>>
>>     On 10/4/16 3:55 AM, Claes Redestad wrote:
>>
>>
>>         On 2016-10-04 12:52, Aleksey Shipilev wrote:
>>
>>             On 10/04/2016 12:50 PM, Claes Redestad wrote:
>>
>>                 Webrev:
>>                 http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>>                 <http://cr.openjdk.java.net/~redestad/8167005/webrev.01/>
>>
>>
>>             OK.
>>
>>         Thanks for the speedy review! :-)
>>
>>
>>     Thanks for looking at this. The shorter text in the bug report is ok
>>     too.
>>
>>     s'marks
>>
>>

From david.holmes at oracle.com  Wed Oct  5 02:16:53 2016
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 5 Oct 2016 12:16:53 +1000
Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port.
In-Reply-To: <c70a1c0fe0df4bb0bd84a6a3c13f446a@DEWDFE13DE50.global.corp.sap>
References: <b384c60b06394f5aaf9a11376899a0aa@DEWDFE13DE50.global.corp.sap>
	<e3963e6f-2dec-f00b-0489-67c4d7822959@oracle.com>
	<CA+3eh13Ca5O=f_+CUHdguP35ZMqWsuEVum+mtbZ5+9xPJDaO3w@mail.gmail.com>
	<4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap>
	<41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com>
	<c70a1c0fe0df4bb0bd84a6a3c13f446a@DEWDFE13DE50.global.corp.sap>
Message-ID: <bd7e9d68-1da5-9b92-3b70-71876bdfc2c1@oracle.com>

On 5/10/2016 2:52 AM, Lindenmaier, Goetz wrote:
> Hi David,
>
> I fixed the change accordingly:
> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/
>
> But do you really want me to incorporate a bugfix for ARM in the port?

Sure - it is only an error message and we've probably never encountered 
it. All the other code has the case for Aarch64 first so this is no 
different - and no need to comment it as it isn't commented upon elsewhere.

Thanks,
David

> Best regards,
>   Goetz.
>
>> -----Original Message-----
>> From: David Holmes [mailto:david.holmes at oracle.com]
>> Sent: Dienstag, 4. Oktober 2016 09:56
>> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Volker Simonis
>> <volker.simonis at gmail.com>
>> Cc: hotspot-dev at openjdk.java.net; core-libs-dev <core-libs-
>> dev at openjdk.java.net>
>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port.
>>
>> Hi Goetz,
>>
>> On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote:
>>> Hi,
>>>
>>> here a new webrev for this change:
>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>> basic_s390/hotspot.wr02/
>>>
>>> I reverted the major reorderings in macros.hpp and os_linux.cpp.
>>> David asked me to do so, and I guess it makes reviewing more simple.
>>
>> Thanks. That said ...
>>
>>> Also this fixes the issue spotted by David which actually was wrong.
>>> The other renaming of ARM to ARM32 was correct, though, as
>>> AARCH64 is defined in both ARM 64-bit ports, and is checked before.
>>> So the existing case checking ARM is only reached if !LP64.
>>> I documented this ...
>>
>> ... We actually have a bug with the Elf32_* defines in os_linux.cpp
>>
>> 1769 #elif  (defined ARM) // ARM must come before AARCH64 because the
>> closed 64-bit port requires so.
>> 1770   static  Elf32_Half running_arch_code=EM_ARM;
>>
>> Aarch64 should come first otherwise we will set the wrong value. I've
>> been told it would only affect an error message but still ... can I ask
>> you to fix this please. Thanks.
>>
>> ---
>> src/share/vm/code/codeCache.cpp
>>
>> I'd prefer to just see something like:
>>
>> S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is
>> used
>>
>> ---
>>
>> Otherwise seems okay.
>>
>> Thanks,
>> David
>>
>>> Also I removed the change in mutex.hpp.  Maybe we contribute
>>> the full change this was part of, but independent of the s390 port.
>>>
>>> I withdraw the part of the change to jdk introducing jvm.cfg.  Volker wants
>> to
>>> do a separate change for this.
>>>
>>> Best regards,
>>>   Goetz.
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Volker Simonis [mailto:volker.simonis at gmail.com]
>>>> Sent: Dienstag, 27. September 2016 19:58
>>>> To: David Holmes <david.holmes at oracle.com>
>>>> Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; hotspot-
>>>> dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.net>
>>>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port.
>>>>
>>>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes
>> <david.holmes at oracle.com>
>>>> wrote:
>>>>> Hi Goetz,
>>>>>
>>>>> I see a change not related directly to S390 ie change from ARM to ARM32
>> in
>>>>> src/os/linux/vm/os_linux.cpp
>>>>>
>>>>
>>>> The change looks a little confusing because Goetz reordered the ifdef
>>>> cascades alphabetically (which I think is good).
>>>>
>>>> Besides that, the only real change not related to s390 is indeed the
>>>> change from ARM to ARM32 which happend two times in the file.
>>>>
>>>> @Goetz: have you done this intentionally?
>>>>
>>>>> It will be a while before I can go through this in any detail.
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> This change is part of the s390 port. It contains some basic adaptions
>>>>>> needed for a full hotspot port for linux s390x.
>>>>>>
>>>>>> It defines the required macros, platform names and includes.
>>>>>>
>>>>>> The s390 port calles CodeCache::contains() in current_frame(), which is
>>>>>> used in NMT. As NMT already collects stack traces before the
>> CodeCache
>>>> is
>>>>>> initialized, contains() needs a check for this.
>>>>>>
>>>>>>  Wherever a row of platforms are listed, I sorted them alphabetically.
>>>>>>
>>>>>>  The jdk requires the file jvm.cfg.
>>>>>>
>>>>>> Please review. I please need a sponsor.
>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>>>> basic_s390/hotspot.wr01/
>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>> basic_s390/jdk.wr01/
>>>>>>
>>>>>> Best regards,
>>>>>>   Goetz.
>>>>>>
>>>>>

From david.holmes at oracle.com  Wed Oct  5 02:27:46 2016
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 5 Oct 2016 12:27:46 +1000
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
Message-ID: <62610596-ca1e-7d75-95c9-6836d0076aea@oracle.com>

Hi Peter,

Without making any comment on what you have actually done, does this 
account for private interface methods correctly?

Thanks,
David

On 4/10/2016 11:40 PM, Peter Levart wrote:
> Hi,
>
> I have a fix for conformance (P2) bug (8062389
> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a
> conformance (P3) bug (8029459
> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance
> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/
>
>
> As Daniel Smith writes in 8029459
> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following
> situation is as expected:
>
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends J {}
> K.class.getMethods() = [ J.m ]
>
> But the following has a different result although it should probably
> have the same:
>
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends I, J {}
> K.class.getMethods() = [ I.m, J.m ]
>
> He then suggests the following algorithm:
>
> An implementation of getMethods consistent with JLS 8 would include the
> following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
> 1) The class's/interface's declared (public) methods
> 2) The getMethods() of the superclass (if this is a class), minus any
> that have a match in (1)
> 3) The getMethods() of each superinterface, minus any that have a match
> in (1) or a _concrete_ match in (2) or a match from a more-specific
> class/interface in (2) or (3)
>
> But even that is not sufficient for the following situation:
>
> interface E { void m(); }
> interface F extends E { void m(); }
> abstract class G implements E {}
> abstract class H extends G implements F {}
> H.class.getMethods() = [ E.m, F.m ]
>
> The desired result of H.class.getMethods() = [ F.m ]
>
> So a more precise definition is required which is implemented in the fix.
>
> The getMethods() implementation partitions the union of the following
> methods:
>
> 1) The class's/interface's declared public methods
> 2) The getMethods() of the superclass (if this is a class)
> 3) The non-static getMethods() of each direct superinterface
>
> ...into equivalence classes (sets) of methods with same signature
> (return type, name, parameter types). Within each such set, only the
> "most specific" methods are kept and others are discarded. The union of
> the kept methods is the result.
>
> The "more specific" is defined as a partial order within a set of
> methods of same signature:
>
> Method A is more specific than method B if:
> - A is declared by a class and B is declared by an interface; or
> - A is declared by the same type as or a subtype of B's declaring type
> and both are either declared by classes or both by interfaces (clearly
> if A and B are declared by the same type and have the same signature,
> they are the same method)
>
> If A and B are distinct elements from the set of methods with same
> signature, then either:
> - A is more specific than B; or
> - B is more specific than A; or
> - A and B are incomparable
>
> A sub-set of "most specific" methods are the methods from the set where
> for each such method M, there is no method N != M such that N is "more
> specific" than M.
>
> There can be more than one "most specific" method for a particular
> signature as they can be inherited from multiple unrelated interfaces, but:
> - javac prevents compilation when among multiply inherited methods with
> same signature there is at least one default method, so in practice,
> getMethods() will only return multiple methods with same signature if
> they are abstract interface methods. With one exception: bridge methods
> that are created by javac for co-variant overrides are default methods
> in interfaces. For example:
>
>     interface I { Object m(); }
>     interface J1 extends I { Map m(); }
>     interface J2 extends I { HashMap m(); }
>     interface K extends J1, J2 {}
>
> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, default
> Object j2.m, abstract HashMap j2.m ]
>
> But this is an extreme case where one can still expect multiple
> non-abstract methods with same signature, but different declaring type,
> returned from getMethods().
>
> In order to also fix 8062389
> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and
> getMethods() share the same consolidation logic that results in a set of
> "most specific" methods. The partitioning in getMethods() is partially
> performed by collecting Methods into a HashMap where the keys are (name,
> parameter types) tuples and values are linked lists of Method objects
> with possibly varying return and declaring types. The consolidation,
> i.e. keeping only the set of most specific methods as new methods are
> encountered, is performed only among methods in the list that share same
> return type and also removes duplicates. getMethod() uses only one such
> list, consolidates methods that match given name and parameter types and
> returns the 1st method from the resulting list that has the most
> specific return type.
>
> That's it for algorithms used. As partitioning partially happens using
> HashMap with (name, parameter types) keys, lists of methods that form
> values are typically kept short with most of them containing only a
> single method, so this fix also fixes performance issue 8061950
> <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>
> The patch also contains some optimizations around redundant copying of
> arrays and reflective objects.
>
> The FilterNotMostSpecific jtreg test has been updated to accommodate for
> changed behavior. Both of the above extreme cases have been added to the
> test.
>
> So, comments, please.
>
> Regards, Peter
>

From mandy.chung at oracle.com  Wed Oct  5 02:50:11 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 4 Oct 2016 19:50:11 -0700
Subject: Review Request JDK-8166846: jdeps fails to generate module info if
	there is any class in unnamed package
Message-ID: <C3C8A61B-035C-4098-97D3-5AB40ED21189@oracle.com>

http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8166846/webrev.00/

This improves jdeps error message to indicate that a JAR file containing unnamed package is not valid to generate module-info.java.

Mandy

From mandy.chung at oracle.com  Wed Oct  5 02:52:36 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 4 Oct 2016 19:52:36 -0700
Subject: Review Request JDK-8167014: jdeps: Missing message: warn.skipped.entry
Message-ID: <BDD25366-BCFE-4EE1-B2AE-69338A617C38@oracle.com>

http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167014/webrev.00/index.html

This fixes the missing key in the resource bundle and also include the exception message of the invalid entry.  I verified with a JAR file with bad entries.

Mandy

From masayoshi.okutsu at oracle.com  Wed Oct  5 03:18:50 2016
From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu)
Date: Wed, 5 Oct 2016 12:18:50 +0900
Subject: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
In-Reply-To: <68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>
References: <44b5d97a-222a-4d46-8896-8a2115108081@default>
	<CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>
	<68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>
Message-ID: <b5377a46-9a54-be4c-eca6-aed98b6a821f@oracle.com>

Hi Ramanand,

I don't think it's appropriate to add the bug ID to 
test/sun/util/resources/cldr/Bug8134384.java. This test doesn't verify 
the TimeZoneNames*.java changes of this fix. Otherwise, the change looks 
OK to me.

Thanks,
Masayoshi


On 10/4/2016 7:22 PM, Ramanand Patil wrote:
> Hi Martin,
> Thank you for your review and explanation of "Yangon". I liked the translation "End of Strife".
>
> Looking at the description of the ZoneNames.java:
> * The zid<->metazone mappings are based on CLDR metaZones.xml.
>   * The alias mappings are based on Link entries in tzdb data files.
>
> I had thought to not update this file because the CLDR metaZones.xml file doesn?t have this entry updated.
> But I think you are correct, since Link entry has this alias mentioned, there is no harm in adding these entries to zidMap and aliasMap arrays.
> Here is the updated Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.01/
>
> Changes done:
> 	- Updated src/java.base/share/classes/java/time/format/ZoneName.java to include "Yangon" entry.
> 	- Removed unused imports from src/java.base/share/classes/java/time/format/ZoneName.java
> 	- Updated ZoneName.java in the test package as well to include "Yangon". [test/java/time/test/java/time/format/ZoneName.java]
> 	- Updated the bugID for test/java/time/test/java/time/format/TestZoneTextPrinterParser.java since this uses the "ZoneName.java" defined in test package.
>
> Also, looks like ZoneName.java is trying to maintain a comprehensive list of zone names. Though I found very few zone names are missing from this file like: "Europe/Busingen", "America/Fort_Nelson", "Antarctica/Troll" etc...
>
>
> Regards,
> Ramanand.
>
> From: Martin Buchholz [mailto:martinrb at google.com]
> Sent: Monday, October 03, 2016 8:55 PM
> To: Ramanand Patil <ramanand.patil at oracle.com>
> Cc: i18n-dev at openjdk.java.net; core-libs-dev <core-libs-dev at openjdk.java.net>
> Subject: Re: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
>
> Hi Ramanand,
> Pleased to meet you!
>
> I expected to see Yangon added to ZoneName, because of the existing reference to Rangoon
>
> java/time/test/java/time/format/ZoneName.java:179:        "Asia/Rangoon", "Myanmar", "Asia/Rangoon",
>
> Is ZoneName.java trying to maintain a comprehensive list of zone names?
>
> """Yangon (???????) is a combination of the two words yan (???) and koun (????), which mean "enemies" and "run out of", respectively. It is also translated as "End of Strife"."""
>
>
> On Mon, Oct 3, 2016 at 5:27 AM, Ramanand Patil <mailto:ramanand.patil at oracle.com> wrote:
> HI all,
> Please review the latest TZDATA integration (tzdata2016g) to JDK9.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
> Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/
>
> All the TimeZone related tests are passed after integration.
> [BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since they verify the renamed TZID "Asia/Yangon"].
>
> Regards,
> Ramanand.


From mandy.chung at oracle.com  Wed Oct  5 04:19:48 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 4 Oct 2016 21:19:48 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <57F43F43.3080707@oracle.com>
References: <57F43DC3.8040905@oracle.com> <57F43F43.3080707@oracle.com>
Message-ID: <9C464F0F-5295-45E4-972D-2511EC937534@oracle.com>

This SPI is useful and provides as a replacement to existing use of internal APIs to launch some of our tools.  We will get jar, jmod, jlink and possibly other tools to convert to this SPI.

ToolProvider::findFirst(String name) can find tool providers on classpath.  I think it needs to wrap the for-loop (specifically iterating on providers) with doPrivileged due to the stack-based permission check.

Otherwise, looks good.

Mandy

> On Oct 4, 2016, at 4:46 PM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> Resend with non-mostly-empty subject line!
> 
> -- Jon
> 
> On 10/04/2016 04:39 PM, Jonathan Gibbons wrote:
>> Core-libs folk,
>> 
>> Please review the following change to add a new service provider class
>>    java.util.spi.ToolProvider
>> 
>> which can be used provide simple "command-line" access to select JDK
>> tools, without starting a new JVM.
>> 
>> The following tools are updated to provide access through the new SPI:
>>    javac, javadoc, javap, jdeps
>> 
>> It is expected that additional tools will also be updated to provide access,
>> but that will be done separately.
>> 
>> Compiler-dev folk may wish to review the changes to the langtools repository.
>> 
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
>> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
>> API: http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
>> 
>> -- Jon
> 


From mandy.chung at oracle.com  Wed Oct  5 04:19:48 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 4 Oct 2016 21:19:48 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <57F43F43.3080707@oracle.com>
References: <57F43DC3.8040905@oracle.com> <57F43F43.3080707@oracle.com>
Message-ID: <250C708F-AF40-45C0-A075-5BBDFA41F0A9@oracle.com>

This SPI is useful and provides as a replacement to existing use of internal APIs to launch some of our tools.  We will get jar, jmod, jlink and possibly other tools to convert to this SPI.

ToolProvider::findFirst(String name) can find tool providers on classpath.  I think it needs to wrap the for-loop (specifically iterating on providers) with doPrivileged due to the stack-based permission check.

Otherwise, looks good.

Mandy

> On Oct 4, 2016, at 4:46 PM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> Resend with non-mostly-empty subject line!
> 
> -- Jon
> 
> On 10/04/2016 04:39 PM, Jonathan Gibbons wrote:
>> Core-libs folk,
>> 
>> Please review the following change to add a new service provider class
>>    java.util.spi.ToolProvider
>> 
>> which can be used provide simple "command-line" access to select JDK
>> tools, without starting a new JVM.
>> 
>> The following tools are updated to provide access through the new SPI:
>>    javac, javadoc, javap, jdeps
>> 
>> It is expected that additional tools will also be updated to provide access,
>> but that will be done separately.
>> 
>> Compiler-dev folk may wish to review the changes to the langtools repository.
>> 
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
>> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
>> API: http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
>> 
>> -- Jon
> 


From ramanand.patil at oracle.com  Wed Oct  5 07:33:08 2016
From: ramanand.patil at oracle.com (Ramanand Patil)
Date: Wed, 5 Oct 2016 00:33:08 -0700 (PDT)
Subject: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
In-Reply-To: <b5377a46-9a54-be4c-eca6-aed98b6a821f@oracle.com>
References: <44b5d97a-222a-4d46-8896-8a2115108081@default>
	<CA+kOe0-PEEY+PcSUS1wQDT4uw5co0LJBP+ZC5HE8Dq93s12qbA@mail.gmail.com>
	<68b8173e-5fca-42f0-bc0e-d0066acd76bb@default>
	<b5377a46-9a54-be4c-eca6-aed98b6a821f@oracle.com>
Message-ID: <48e1f954-204e-4a7a-a163-7fee6ea08f86@default>

Hi Masayoshi,
Thank you for pointing that small miss. During testing phase, I had actually renamed "Asia/Rangoon" to "Asia/Yangon" (Asia/Rangoon entry was deleted) in TimeZoneNames*.java and this test case was failing along with other test cases, hence the bug ID tag was added there. But after correcting the TimeZoneNames*.java with correct changes I forgot to remove the bug ID from this bug.

Here is the updated Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.02/

Regards,
Ramanand.

-----Original Message-----
From: Masayoshi Okutsu 
Sent: Wednesday, October 05, 2016 8:49 AM
To: Ramanand Patil <ramanand.patil at oracle.com>; Martin Buchholz <martinrb at google.com>
Cc: core-libs-dev <core-libs-dev at openjdk.java.net>; i18n-dev at openjdk.java.net
Subject: Re: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g

Hi Ramanand,

I don't think it's appropriate to add the bug ID to test/sun/util/resources/cldr/Bug8134384.java. This test doesn't verify the TimeZoneNames*.java changes of this fix. Otherwise, the change looks OK to me.

Thanks,
Masayoshi


On 10/4/2016 7:22 PM, Ramanand Patil wrote:
> Hi Martin,
> Thank you for your review and explanation of "Yangon". I liked the translation "End of Strife".
>
> Looking at the description of the ZoneNames.java:
> * The zid<->metazone mappings are based on CLDR metaZones.xml.
>   * The alias mappings are based on Link entries in tzdb data files.
>
> I had thought to not update this file because the CLDR metaZones.xml file doesn?t have this entry updated.
> But I think you are correct, since Link entry has this alias mentioned, there is no harm in adding these entries to zidMap and aliasMap arrays.
> Here is the updated Webrev: 
> http://cr.openjdk.java.net/~rpatil/8166875/webrev.01/
>
> Changes done:
> 	- Updated src/java.base/share/classes/java/time/format/ZoneName.java to include "Yangon" entry.
> 	- Removed unused imports from src/java.base/share/classes/java/time/format/ZoneName.java
> 	- Updated ZoneName.java in the test package as well to include "Yangon". [test/java/time/test/java/time/format/ZoneName.java]
> 	- Updated the bugID for test/java/time/test/java/time/format/TestZoneTextPrinterParser.java since this uses the "ZoneName.java" defined in test package.
>
> Also, looks like ZoneName.java is trying to maintain a comprehensive list of zone names. Though I found very few zone names are missing from this file like: "Europe/Busingen", "America/Fort_Nelson", "Antarctica/Troll" etc...
>
>
> Regards,
> Ramanand.
>
> From: Martin Buchholz [mailto:martinrb at google.com]
> Sent: Monday, October 03, 2016 8:55 PM
> To: Ramanand Patil <ramanand.patil at oracle.com>
> Cc: i18n-dev at openjdk.java.net; core-libs-dev 
> <core-libs-dev at openjdk.java.net>
> Subject: Re: <i18n dev> RFR: 8166875: (tz) Support tzdata2016g
>
> Hi Ramanand,
> Pleased to meet you!
>
> I expected to see Yangon added to ZoneName, because of the existing 
> reference to Rangoon
>
> java/time/test/java/time/format/ZoneName.java:179:        "Asia/Rangoon", "Myanmar", "Asia/Rangoon",
>
> Is ZoneName.java trying to maintain a comprehensive list of zone names?
>
> """Yangon (???????) is a combination of the two words yan (???) and koun (????), which mean "enemies" and "run out of", respectively. It is also translated as "End of Strife"."""
>
>
> On Mon, Oct 3, 2016 at 5:27 AM, Ramanand Patil <mailto:ramanand.patil at oracle.com> wrote:
> HI all,
> Please review the latest TZDATA integration (tzdata2016g) to JDK9.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8166875
> Webrev: http://cr.openjdk.java.net/~rpatil/8166875/webrev.00/
>
> All the TimeZone related tests are passed after integration.
> [BugID is updated for tests TimeZoneTest.java and Bug8134384.java, since they verify the renamed TZID "Asia/Yangon"].
>
> Regards,
> Ramanand.


From peter.levart at gmail.com  Wed Oct  5 07:50:43 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 5 Oct 2016 09:50:43 +0200
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <62610596-ca1e-7d75-95c9-6836d0076aea@oracle.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<62610596-ca1e-7d75-95c9-6836d0076aea@oracle.com>
Message-ID: <0a7a94f6-c2e3-9429-377c-0cf34fb458b1@gmail.com>

Hi David,


On 10/05/2016 04:27 AM, David Holmes wrote:
> Hi Peter,
>
> Without making any comment on what you have actually done, does this 
> account for private interface methods correctly?

In short, yes. Class.getMethods() and Class.getMethod() only ever return 
public methods. The source of Method objects is in both cases the 
following invocation:

     privateGetDeclaredMethods(/* publicOnly */ true)

in lines 2981 and 3087, respectively.

Regards, Peter

>
> Thanks,
> David
>
> On 4/10/2016 11:40 PM, Peter Levart wrote:
>> Hi,
>>
>> I have a fix for conformance (P2) bug (8062389
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a
>> conformance (P3) bug (8029459
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance
>> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/ 
>>
>>
>>
>> As Daniel Smith writes in 8029459
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following
>> situation is as expected:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends J {}
>> K.class.getMethods() = [ J.m ]
>>
>> But the following has a different result although it should probably
>> have the same:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends I, J {}
>> K.class.getMethods() = [ I.m, J.m ]
>>
>> He then suggests the following algorithm:
>>
>> An implementation of getMethods consistent with JLS 8 would include the
>> following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
>> 1) The class's/interface's declared (public) methods
>> 2) The getMethods() of the superclass (if this is a class), minus any
>> that have a match in (1)
>> 3) The getMethods() of each superinterface, minus any that have a match
>> in (1) or a _concrete_ match in (2) or a match from a more-specific
>> class/interface in (2) or (3)
>>
>> But even that is not sufficient for the following situation:
>>
>> interface E { void m(); }
>> interface F extends E { void m(); }
>> abstract class G implements E {}
>> abstract class H extends G implements F {}
>> H.class.getMethods() = [ E.m, F.m ]
>>
>> The desired result of H.class.getMethods() = [ F.m ]
>>
>> So a more precise definition is required which is implemented in the 
>> fix.
>>
>> The getMethods() implementation partitions the union of the following
>> methods:
>>
>> 1) The class's/interface's declared public methods
>> 2) The getMethods() of the superclass (if this is a class)
>> 3) The non-static getMethods() of each direct superinterface
>>
>> ...into equivalence classes (sets) of methods with same signature
>> (return type, name, parameter types). Within each such set, only the
>> "most specific" methods are kept and others are discarded. The union of
>> the kept methods is the result.
>>
>> The "more specific" is defined as a partial order within a set of
>> methods of same signature:
>>
>> Method A is more specific than method B if:
>> - A is declared by a class and B is declared by an interface; or
>> - A is declared by the same type as or a subtype of B's declaring type
>> and both are either declared by classes or both by interfaces (clearly
>> if A and B are declared by the same type and have the same signature,
>> they are the same method)
>>
>> If A and B are distinct elements from the set of methods with same
>> signature, then either:
>> - A is more specific than B; or
>> - B is more specific than A; or
>> - A and B are incomparable
>>
>> A sub-set of "most specific" methods are the methods from the set where
>> for each such method M, there is no method N != M such that N is "more
>> specific" than M.
>>
>> There can be more than one "most specific" method for a particular
>> signature as they can be inherited from multiple unrelated 
>> interfaces, but:
>> - javac prevents compilation when among multiply inherited methods with
>> same signature there is at least one default method, so in practice,
>> getMethods() will only return multiple methods with same signature if
>> they are abstract interface methods. With one exception: bridge methods
>> that are created by javac for co-variant overrides are default methods
>> in interfaces. For example:
>>
>>     interface I { Object m(); }
>>     interface J1 extends I { Map m(); }
>>     interface J2 extends I { HashMap m(); }
>>     interface K extends J1, J2 {}
>>
>> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, default
>> Object j2.m, abstract HashMap j2.m ]
>>
>> But this is an extreme case where one can still expect multiple
>> non-abstract methods with same signature, but different declaring type,
>> returned from getMethods().
>>
>> In order to also fix 8062389
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and
>> getMethods() share the same consolidation logic that results in a set of
>> "most specific" methods. The partitioning in getMethods() is partially
>> performed by collecting Methods into a HashMap where the keys are (name,
>> parameter types) tuples and values are linked lists of Method objects
>> with possibly varying return and declaring types. The consolidation,
>> i.e. keeping only the set of most specific methods as new methods are
>> encountered, is performed only among methods in the list that share same
>> return type and also removes duplicates. getMethod() uses only one such
>> list, consolidates methods that match given name and parameter types and
>> returns the 1st method from the resulting list that has the most
>> specific return type.
>>
>> That's it for algorithms used. As partitioning partially happens using
>> HashMap with (name, parameter types) keys, lists of methods that form
>> values are typically kept short with most of them containing only a
>> single method, so this fix also fixes performance issue 8061950
>> <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>>
>> The patch also contains some optimizations around redundant copying of
>> arrays and reflective objects.
>>
>> The FilterNotMostSpecific jtreg test has been updated to accommodate for
>> changed behavior. Both of the above extreme cases have been added to the
>> test.
>>
>> So, comments, please.
>>
>> Regards, Peter
>>


From aph at redhat.com  Wed Oct  5 08:19:26 2016
From: aph at redhat.com (Andrew Haley)
Date: Wed, 5 Oct 2016 09:19:26 +0100
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
Message-ID: <148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>

On 04/10/16 22:19, Martin Buchholz wrote:
> VarHandle is a reasonable replacement for FieldUpdaters, but it's not yet
> complete (where is accumulateAndGet?), and when do you deprecate something
> when the replacement won't be ubiquitous for 10 years?

Surely you have to start somewhere: deprecation is no more than saying
to programmers "Don't use this, use that."  And if you were leaning
over someone's shoulder that's what you would say.

Andrew.



From scolebourne at joda.org  Wed Oct  5 09:54:16 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 5 Oct 2016 10:54:16 +0100
Subject: RFR: 8159855
In-Reply-To: <57F43DC3.8040905@oracle.com>
References: <57F43DC3.8040905@oracle.com>
Message-ID: <CACzrW9Ak7fxKrWYH_5HivN7LNDi-EtzoGHRG4ayyhUcz64WBJA@mail.gmail.com>

Interesting.

How likely is it that there will be more than one tool of a given name
available? The method name findFirst() seems relatively odd for the
lookup operation.

I'd also note that the name string to pass in are "magic". There are
no constants defined for callers to use. Since there is no obvious way
in the API to find the vendor, this could get tricky across JDKs. Plus
how would a caller know what arguments to pass if each vendors tool
differs?

Just being cautious about the use case being solved.

Stephen

On 5 October 2016 at 00:39, Jonathan Gibbons
<jonathan.gibbons at oracle.com> wrote:
> Core-libs folk,
>
> Please review the following change to add a new service provider class
>     java.util.spi.ToolProvider
>
> which can be used provide simple "command-line" access to select JDK
> tools, without starting a new JVM.
>
> The following tools are updated to provide access through the new SPI:
>     javac, javadoc, javap, jdeps
>
> It is expected that additional tools will also be updated to provide access,
> but that will be done separately.
>
> Compiler-dev folk may wish to review the changes to the langtools
> repository.
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
> API:
> http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
>
> -- Jon

From jbluettduncan at gmail.com  Wed Oct  5 10:27:52 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Wed, 5 Oct 2016 11:27:52 +0100
Subject: RFR: 8167005: Comment on the need for an empty constructor in
	ArrayList$Itr
In-Reply-To: <50062775-6261-5811-6a8c-e3274180c8f7@oracle.com>
References: <57F38993.6050505@oracle.com>
	<b1ecbcb6-9237-b7e7-085b-ed20a9e150ec@redhat.com>
	<57F38AB9.70506@oracle.com>
	<66577b61-05e9-1a02-dca7-df822bb519e5@oracle.com>
	<CAPMys849wiV07AvrLqNBNPbo5kaOgLcA53rxX8znpsSkB0__Kw@mail.gmail.com>
	<57F44281.8060502@oracle.com>
	<50062775-6261-5811-6a8c-e3274180c8f7@oracle.com>
Message-ID: <CAPMys87PszN31ZQAUGAX=fd3aLXCUOTOSosCNKmf_zVtOtxiNQ@mail.gmail.com>

Okay, that makes sense to me! Thank you for your explanations Claes and
Stuart.

Kind regards,
Jonathan

On 5 October 2016 at 01:57, Stuart Marks <stuart.marks at oracle.com> wrote:

> Right, the main point of the comment is to tell the reader the constructor
> isn't superfluous, to prevent it from being cleaned up and accidentally
> causing a regression. Full history can reside in the commit comment, the
> bug database, and in these email logs.
>
> I'd put in a link to a bug only when there's some action on this code
> associated with that bug, e.g., "don't remove this code until bug XXXXXXX
> has been fixed."
>
> s'marks
>
>
> On 10/4/16 5:00 PM, Claes Redestad wrote:
>
>> Hi Jonathan,
>>
>> the aim isn't to add an in-depth explanation to the code about exactly
>> the circumstance that led to this constructor and comment being added,
>> but to put a clear message that it was simply, in fact, deliberate, so
>> even the proposed comment might be going further than strictly necessary.
>>
>> I'm also not convinced of the value of putting explicit links to the
>> bug actually pushed, since there's an implicit link in the commit
>> itself anyhow.
>>
>> Thanks!
>>
>> /Claes
>>
>> On 2016-10-04 23:20, Jonathan Bluett-Duncan wrote:
>>
>>> The explanation which Stuart gives for this change in
>>> https://bugs.openjdk.java.net/browse/JDK-8167005 seems to describe why
>>> this constructor is needed much better than the comment itself does. So
>>> I wonder if it's worth adding the link to the bug report in the comment.
>>> E.g.
>>>
>>> // prevent generation of synthetic class required for access to private
>>> // constructor. See: https://bugs.openjdk.java.net/browse/JDK-8167005
>>>
>>> Kind regards,
>>> Jonathan
>>>
>>> On 4 October 2016 at 21:28, Stuart Marks <stuart.marks at oracle.com
>>> <mailto:stuart.marks at oracle.com>> wrote:
>>>
>>>
>>>
>>>     On 10/4/16 3:55 AM, Claes Redestad wrote:
>>>
>>>
>>>         On 2016-10-04 12:52, Aleksey Shipilev wrote:
>>>
>>>             On 10/04/2016 12:50 PM, Claes Redestad wrote:
>>>
>>>                 Webrev:
>>>                 http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>>>                 <http://cr.openjdk.java.net/~redestad/8167005/webrev.01/
>>> >
>>>
>>>
>>>             OK.
>>>
>>>         Thanks for the speedy review! :-)
>>>
>>>
>>>     Thanks for looking at this. The shorter text in the bug report is ok
>>>     too.
>>>
>>>     s'marks
>>>
>>>
>>>

From Alan.Bateman at oracle.com  Wed Oct  5 10:34:17 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 5 Oct 2016 11:34:17 +0100
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <CACzrW9Ak7fxKrWYH_5HivN7LNDi-EtzoGHRG4ayyhUcz64WBJA@mail.gmail.com>
References: <57F43DC3.8040905@oracle.com>
	<CACzrW9Ak7fxKrWYH_5HivN7LNDi-EtzoGHRG4ayyhUcz64WBJA@mail.gmail.com>
Message-ID: <54f7ed03-11b5-72fe-45bc-f6e7019014a5@oracle.com>

On 05/10/2016 10:54, Stephen Colebourne wrote:

> Interesting.
>
> How likely is it that there will be more than one tool of a given name
> available? The method name findFirst() seems relatively odd for the
> lookup operation.
>
> I'd also note that the name string to pass in are "magic". There are
> no constants defined for callers to use. Since there is no obvious way
> in the API to find the vendor, this could get tricky across JDKs. Plus
> how would a caller know what arguments to pass if each vendors tool
> differs?
>
> Just being cautious about the use case being solved.
Many command line tools don't define an API and so not unheard of to 
find code that uses sun.tools.jar.Main.main or the main method of other 
tools. Also common to see code using ProcessBuilder or Runtime.exec to 
launch tools like jar. If the commonly used tools (jar, jdeps, ...) 
document their tool names in their module description then it will make 
it easier to run these tools in the same VM. It does mean that some 
tools will have both an API and a documented tool name but that 
shouldn't be an issue. I'm sure Jon has a lot to say on this topic, esp. 
with linking usage documentation and man pages.

As regards constants then not clear where something like this would live 
as it amounts to a registry. Also many of the tools are JDK or library 
specific and so wouldn't have a place in the standard/Java SE docs.

The firstFirst(String) method limits itself to tools that are visible 
via the system class loader. Sure, someone might decide to deploy lots 
of libraries that all claim to be the "hammer" tool but this is no 
different to many of the service providers mechanisms. One could have 
the ToolProvider define methods with the tool capabilities that would 
aid selection but that would complicate the API.

-Alan

From forax at univ-mlv.fr  Wed Oct  5 11:10:13 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 5 Oct 2016 13:10:13 +0200 (CEST)
Subject: RFR: 8159855
In-Reply-To: <57F43DC3.8040905@oracle.com>
References: <57F43DC3.8040905@oracle.com>
Message-ID: <489906237.1277782.1475665813824.JavaMail.zimbra@u-pem.fr>

Very nice,
it will greatly help gradle, maven, etc.

in com/sun/tools/javac/main/Main.java,
the other constructor should delegate its initialization to the constructor you just add,
     public Main(String name, PrintWriter out) {
         this(name, out, out);
     }

I wonder if findFirst() in ToolProvider should not take a ClassLoader as parameter instead of being restricted to the system classloader.

regards,
R?mi

----- Mail original -----
> De: "Jonathan Gibbons" <jonathan.gibbons at oracle.com>
> ?: "OpenJDK Dev list" <core-libs-dev at openjdk.java.net>
> Cc: compiler-dev at openjdk.java.net
> Envoy?: Mercredi 5 Octobre 2016 01:39:47
> Objet: RFR: 8159855

> Core-libs folk,
> 
> Please review the following change to add a new service provider class
>     java.util.spi.ToolProvider
> 
> which can be used provide simple "command-line" access to select JDK
> tools, without starting a new JVM.
> 
> The following tools are updated to provide access through the new SPI:
>     javac, javadoc, javap, jdeps
> 
> It is expected that additional tools will also be updated to provide access,
> but that will be done separately.
> 
> Compiler-dev folk may wish to review the changes to the langtools
> repository.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
> API:
> http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
> 
> -- Jon

From lance.andersen at oracle.com  Wed Oct  5 11:17:45 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Wed, 5 Oct 2016 07:17:45 -0400
Subject: Review Request JDK-8167014: jdeps: Missing message:
	warn.skipped.entry
In-Reply-To: <BDD25366-BCFE-4EE1-B2AE-69338A617C38@oracle.com>
References: <BDD25366-BCFE-4EE1-B2AE-69338A617C38@oracle.com>
Message-ID: <D0229F87-3723-4743-9908-E0682A63D1C5@oracle.com>

Looks Ok Mandy
> On Oct 4, 2016, at 10:52 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167014/webrev.00/index.html
> 
> This fixes the missing key in the resource bundle and also include the exception message of the invalid entry.  I verified with a JAR file with bad entries.
> 
> Mandy

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From Alan.Bateman at oracle.com  Wed Oct  5 11:39:02 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 5 Oct 2016 12:39:02 +0100
Subject: RFR: 8159855
In-Reply-To: <489906237.1277782.1475665813824.JavaMail.zimbra@u-pem.fr>
References: <57F43DC3.8040905@oracle.com>
	<489906237.1277782.1475665813824.JavaMail.zimbra@u-pem.fr>
Message-ID: <b2dcf1fa-a8d1-838a-3afe-ac8039e4acf2@oracle.com>



On 05/10/2016 12:10, Remi Forax wrote:
> Very nice,
> it will greatly help gradle, maven, etc.
>
> in com/sun/tools/javac/main/Main.java,
> the other constructor should delegate its initialization to the constructor you just add,
>       public Main(String name, PrintWriter out) {
>           this(name, out, out);
>       }
>
> I wonder if findFirst() in ToolProvider should not take a ClassLoader as parameter instead of being restricted to the system classloader.
>
The findFirst(tool) method just a convenience, the intention is that if 
you want to use other contexts then you just invoke ServiceLoader to 
locate it.

-Alan.

From forax at univ-mlv.fr  Wed Oct  5 12:20:09 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 5 Oct 2016 14:20:09 +0200 (CEST)
Subject: RFR: 8159855
In-Reply-To: <b2dcf1fa-a8d1-838a-3afe-ac8039e4acf2@oracle.com>
References: <57F43DC3.8040905@oracle.com>
	<489906237.1277782.1475665813824.JavaMail.zimbra@u-pem.fr>
	<b2dcf1fa-a8d1-838a-3afe-ac8039e4acf2@oracle.com>
Message-ID: <1167030149.1309621.1475670009250.JavaMail.zimbra@u-pem.fr>

It seems to be a little to JDK centric to me.

R?mi

----- Mail original -----
> De: "Alan Bateman" <Alan.Bateman at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>, "Jonathan Gibbons" <jonathan.gibbons at oracle.com>
> Cc: compiler-dev at openjdk.java.net, "OpenJDK Dev list" <core-libs-dev at openjdk.java.net>
> Envoy?: Mercredi 5 Octobre 2016 13:39:02
> Objet: Re: RFR: 8159855

> On 05/10/2016 12:10, Remi Forax wrote:
>> Very nice,
>> it will greatly help gradle, maven, etc.
>>
>> in com/sun/tools/javac/main/Main.java,
>> the other constructor should delegate its initialization to the constructor you
>> just add,
>>       public Main(String name, PrintWriter out) {
>>           this(name, out, out);
>>       }
>>
>> I wonder if findFirst() in ToolProvider should not take a ClassLoader as
>> parameter instead of being restricted to the system classloader.
>>
> The findFirst(tool) method just a convenience, the intention is that if
> you want to use other contexts then you just invoke ServiceLoader to
> locate it.
> 
> -Alan.

From Alan.Bateman at oracle.com  Wed Oct  5 14:10:50 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 5 Oct 2016 15:10:50 +0100
Subject: RFR: 8159855
In-Reply-To: <1167030149.1309621.1475670009250.JavaMail.zimbra@u-pem.fr>
References: <57F43DC3.8040905@oracle.com>
	<489906237.1277782.1475665813824.JavaMail.zimbra@u-pem.fr>
	<b2dcf1fa-a8d1-838a-3afe-ac8039e4acf2@oracle.com>
	<1167030149.1309621.1475670009250.JavaMail.zimbra@u-pem.fr>
Message-ID: <e423c6e5-a50e-e0d4-27b6-71b73627c8b5@oracle.com>

On 05/10/2016 13:20, forax at univ-mlv.fr wrote:

> It seems to be a little to JDK centric to me.
>
It supports the common case, making the advanced cases possible. Also 
it's consistent with the javax.tools API. So overall then I think Jon 
has this right.

-Alan

From srinivas.dama at oracle.com  Wed Oct  5 14:21:10 2016
From: srinivas.dama at oracle.com (Srinivas Dama)
Date: Wed, 5 Oct 2016 07:21:10 -0700 (PDT)
Subject: RFR:8055033: Shell tests for jrunscript don't pass through VM options
Message-ID: <89ec1fe4-5e9c-4013-bc15-cfc142aebd2a@default>

Hi,

Please review http://cr.openjdk.java.net/~sdama/8055033/webrev.00/ 
for https://bugs.openjdk.java.net/browse/JDK-8055033 

When launching java and javac from shell script, ${TESTVMOPTS}, ${TESTJAVAOPTS} and ${TESTTOOLVMOPTS}, ${TESTJAVACOPTS} should be 
passed in respectively since jtreg sets these environment variables.

Regards,
Srinivas


From Alan.Bateman at oracle.com  Wed Oct  5 14:34:26 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 5 Oct 2016 15:34:26 +0100
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <9C464F0F-5295-45E4-972D-2511EC937534@oracle.com>
References: <57F43DC3.8040905@oracle.com> <57F43F43.3080707@oracle.com>
	<9C464F0F-5295-45E4-972D-2511EC937534@oracle.com>
Message-ID: <a4fbba01-ef17-3a2a-33a4-deb964855f0f@oracle.com>

On 05/10/2016 05:19, Mandy Chung wrote:

> :
>
> ToolProvider::findFirst(String name) can find tool providers on classpath.  I think it needs to wrap the for-loop (specifically iterating on providers) with doPrivileged due to the stack-based permission check.
>
Yes, that will be needed, otherwise the caller will need permission to 
get the system class loader and whatever other permissions are needed to 
locate and and load the tools.

The rest looks okay to me. Personally I would space out the javadoc tags 
more, and indent the second/third lines more than one space but that is 
just minor stuff.

-Alan

From forax at univ-mlv.fr  Wed Oct  5 14:39:07 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 05 Oct 2016 14:39:07 +0000
Subject: Deprecate all java.util.concurrent.*FieldUpdater
In-Reply-To: <CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
Message-ID: <73C87358-90FB-4C00-9CC8-78C43360FFE5@univ-mlv.fr>

Hi Martin,

On October 4, 2016 11:19:33 PM GMT+02:00, Martin Buchholz <martinrb at google.com> wrote:
>VarHandle is a reasonable replacement for FieldUpdaters, but it's not
>yet
>complete (where is accumulateAndGet?), 

Seems to be a feature for me :)
I've never liked the methods that takes a lambda in FieldUpdater. If you're using a FieldUpdater, you are in the basement, you want a reliable performance model. The JLS does not guarantee that a side effect free lambda will be always inlined.

>and when do you deprecate
>something
>when the replacement won't be ubiquitous for 10 years?

The right answer is the one from Andrew but i can not resisrt, here is my answer:
when you hope that it will not take 10 years to be ubiquitous.

regards,
Remi

>
>On Tue, Oct 4, 2016 at 1:32 PM, Remi Forax <forax at univ-mlv.fr> wrote:
>
>> Given that Java 9 introduces a faster way to emit things like
>> compareAndSet by using the VarHandke API and that AtomiReference (and
>> likes) are now rewritten to use VarHandles directly,
>> i think it's time to deprecate all *FieldUpdater with something
>saying
>> that they have been superseded by the VarHandle API.
>>
>> R?mi
>> substitute dr deprecator
>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From volker.simonis at gmail.com  Wed Oct  5 14:43:19 2016
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 5 Oct 2016 16:43:19 +0200
Subject: RFR(XS): 8166800: [s390] Top-level build changes required for
	Linux/s390x
Message-ID: <CA+3eh10LTXZXE89SkbKBgQhsE71tQVS=tEosJSthGtz2WGUM0A@mail.gmail.com>

Hi,

can I please have a review and sponsor for the following tiny
top-level build change required for building the OpenJDK on
Linux/s390:

http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/
https://bugs.openjdk.java.net/browse/JDK-8166800

All this change does is to add some s390-specifc C and C++ flags for
the Hotspot and the class-library build which do not affect any of the
existing platforms.

Thank you and best regards,
Volker

From volker.simonis at gmail.com  Wed Oct  5 14:43:24 2016
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 5 Oct 2016 16:43:24 +0200
Subject: RFR(XXS): 8166801: [s390] Add jvm.cfg file for Linux/s390x
Message-ID: <CA+3eh12sxgq6t1LTPregAeEsJ-8qbnF-hbWAgZxWTZ7HrLVjMA@mail.gmail.com>

Hi,

can somebody please review the following trivial change which simply
adds a new jvm.cfg file for Linux/s390x:

http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/
https://bugs.openjdk.java.net/browse/JDK-8166801

This is so far the only change we need in the jdk-repository for our
Linux/s390x port.

Thank you and best regards,
Volker

From david.lloyd at redhat.com  Wed Oct  5 14:45:59 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 5 Oct 2016 09:45:59 -0500
Subject: Deprecate all java.util.concurrent.*FieldUpdater
In-Reply-To: <CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
Message-ID: <96854dcc-d476-72a5-b096-678fe6f76eb4@redhat.com>

I'm sure I recall an email from the past few months which proposed that 
*FieldUpdater are still going to be recommended in many cases over 
VarHandle because the latter is probably too low-level for casual uses. 
It was (IIRC) an argument in favor of more advanced fence methods or 
something like that.

Am I imagining it?

On 10/04/2016 04:19 PM, Martin Buchholz wrote:
> VarHandle is a reasonable replacement for FieldUpdaters, but it's not yet
> complete (where is accumulateAndGet?), and when do you deprecate something
> when the replacement won't be ubiquitous for 10 years?
>
> On Tue, Oct 4, 2016 at 1:32 PM, Remi Forax <forax at univ-mlv.fr> wrote:
>
>> Given that Java 9 introduces a faster way to emit things like
>> compareAndSet by using the VarHandke API and that AtomiReference (and
>> likes) are now rewritten to use VarHandles directly,
>> i think it's time to deprecate all *FieldUpdater with something saying
>> that they have been superseded by the VarHandle API.
>>
>> R?mi
>> substitute dr deprecator
>>

-- 
- DML

From shade at redhat.com  Wed Oct  5 14:47:57 2016
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 5 Oct 2016 16:47:57 +0200
Subject: RFR(XXS): 8166801: [s390] Add jvm.cfg file for Linux/s390x
In-Reply-To: <CA+3eh12sxgq6t1LTPregAeEsJ-8qbnF-hbWAgZxWTZ7HrLVjMA@mail.gmail.com>
References: <CA+3eh12sxgq6t1LTPregAeEsJ-8qbnF-hbWAgZxWTZ7HrLVjMA@mail.gmail.com>
Message-ID: <0191f32a-9552-46cf-4e6e-62b0447ba8b6@redhat.com>

On 10/05/2016 04:43 PM, Volker Simonis wrote:
> can somebody please review the following trivial change which simply
> adds a new jvm.cfg file for Linux/s390x:
> 
> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/
> https://bugs.openjdk.java.net/browse/JDK-8166801

This looks okay (and consistent with other arches) to me.

Thanks,
-Aleksey


From erik.joelsson at oracle.com  Wed Oct  5 15:06:14 2016
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 5 Oct 2016 17:06:14 +0200
Subject: RFR(XXS): 8166801: [s390] Add jvm.cfg file for Linux/s390x
In-Reply-To: <CA+3eh12sxgq6t1LTPregAeEsJ-8qbnF-hbWAgZxWTZ7HrLVjMA@mail.gmail.com>
References: <CA+3eh12sxgq6t1LTPregAeEsJ-8qbnF-hbWAgZxWTZ7HrLVjMA@mail.gmail.com>
Message-ID: <f9cfe16c-5fd7-5b99-6402-a16924c23533@oracle.com>

Looks good to me.

/Erik


On 2016-10-05 16:43, Volker Simonis wrote:
> Hi,
>
> can somebody please review the following trivial change which simply
> adds a new jvm.cfg file for Linux/s390x:
>
> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/
> https://bugs.openjdk.java.net/browse/JDK-8166801
>
> This is so far the only change we need in the jdk-repository for our
> Linux/s390x port.
>
> Thank you and best regards,
> Volker


From erik.joelsson at oracle.com  Wed Oct  5 15:07:10 2016
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 5 Oct 2016 17:07:10 +0200
Subject: RFR(XS): 8166800: [s390] Top-level build changes required for
	Linux/s390x
In-Reply-To: <CA+3eh10LTXZXE89SkbKBgQhsE71tQVS=tEosJSthGtz2WGUM0A@mail.gmail.com>
References: <CA+3eh10LTXZXE89SkbKBgQhsE71tQVS=tEosJSthGtz2WGUM0A@mail.gmail.com>
Message-ID: <6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com>

Looks ok to me. I can sponsor this tomorrow.

/Erik


On 2016-10-05 16:43, Volker Simonis wrote:
> Hi,
>
> can I please have a review and sponsor for the following tiny
> top-level build change required for building the OpenJDK on
> Linux/s390:
>
> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/
> https://bugs.openjdk.java.net/browse/JDK-8166800
>
> All this change does is to add some s390-specifc C and C++ flags for
> the Hotspot and the class-library build which do not affect any of the
> existing platforms.
>
> Thank you and best regards,
> Volker


From jonathan.gibbons at oracle.com  Wed Oct  5 15:19:51 2016
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Wed, 5 Oct 2016 08:19:51 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <54f7ed03-11b5-72fe-45bc-f6e7019014a5@oracle.com>
References: <57F43DC3.8040905@oracle.com>
	<CACzrW9Ak7fxKrWYH_5HivN7LNDi-EtzoGHRG4ayyhUcz64WBJA@mail.gmail.com>
	<54f7ed03-11b5-72fe-45bc-f6e7019014a5@oracle.com>
Message-ID: <8de4a83d-18e3-1cd4-ee7b-a54c413ce3a7@oracle.com>

On 10/5/16 3:34 AM, Alan Bateman wrote:
> On 05/10/2016 10:54, Stephen Colebourne wrote:
>
>> Interesting.
>>
>> How likely is it that there will be more than one tool of a given name
>> available? The method name findFirst() seems relatively odd for the
>> lookup operation.
>>
>> I'd also note that the name string to pass in are "magic". There are
>> no constants defined for callers to use. Since there is no obvious way
>> in the API to find the vendor, this could get tricky across JDKs. Plus
>> how would a caller know what arguments to pass if each vendors tool
>> differs?
>>
>> Just being cautious about the use case being solved.
> Many command line tools don't define an API and so not unheard of to 
> find code that uses sun.tools.jar.Main.main or the main method of 
> other tools. Also common to see code using ProcessBuilder or 
> Runtime.exec to launch tools like jar. If the commonly used tools 
> (jar, jdeps, ...) document their tool names in their module 
> description then it will make it easier to run these tools in the same 
> VM. It does mean that some tools will have both an API and a 
> documented tool name but that shouldn't be an issue. I'm sure Jon has 
> a lot to say on this topic, esp. with linking usage documentation and 
> man pages.
>
> As regards constants then not clear where something like this would 
> live as it amounts to a registry. Also many of the tools are JDK or 
> library specific and so wouldn't have a place in the standard/Java SE 
> docs.
>
> The firstFirst(String) method limits itself to tools that are visible 
> via the system class loader. Sure, someone might decide to deploy lots 
> of libraries that all claim to be the "hammer" tool but this is no 
> different to many of the service providers mechanisms. One could have 
> the ToolProvider define methods with the tool capabilities that would 
> aid selection but that would complicate the API.
>
> -Alan

Informally, I think the common use case for this API is to get "same VM" 
access to tools in the bin/ directory of the product image. That implies 
there is an obvious name for each tool, and that there will be 
documentation for the arguments that each tool that is available.

Also, in general, the doc comments for a module should contain details 
of any services for general use, including details of how to access the 
service (e.g. what name to use, in this case) and how to use it (info 
about arguments, etc, in this case.)  We are working on updating the 
javadoc tool to better support the provision of such information.

-- Jon

From volker.simonis at gmail.com  Wed Oct  5 15:22:15 2016
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 5 Oct 2016 17:22:15 +0200
Subject: RFR(XS): 8166800: [s390] Top-level build changes required for
	Linux/s390x
In-Reply-To: <6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com>
References: <CA+3eh10LTXZXE89SkbKBgQhsE71tQVS=tEosJSthGtz2WGUM0A@mail.gmail.com>
	<6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com>
Message-ID: <CA+3eh12Uf79MoKXX0ehL_eEeeMrQG0qAhD4_6R+f7sJxtg-zWQ@mail.gmail.com>

That would be really great!

Could you please do it in the jdk9/hs forest as that's the place where
we will need it first (i.e. when integrating the whole s390 hotspot
platform files).

Thanks,
Volker

On Wed, Oct 5, 2016 at 5:07 PM, Erik Joelsson <erik.joelsson at oracle.com> wrote:
> Looks ok to me. I can sponsor this tomorrow.
>
> /Erik
>
>
>
> On 2016-10-05 16:43, Volker Simonis wrote:
>>
>> Hi,
>>
>> can I please have a review and sponsor for the following tiny
>> top-level build change required for building the OpenJDK on
>> Linux/s390:
>>
>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/
>> https://bugs.openjdk.java.net/browse/JDK-8166800
>>
>> All this change does is to add some s390-specifc C and C++ flags for
>> the Hotspot and the class-library build which do not affect any of the
>> existing platforms.
>>
>> Thank you and best regards,
>> Volker
>
>

From aph at redhat.com  Wed Oct  5 15:38:22 2016
From: aph at redhat.com (Andrew Haley)
Date: Wed, 5 Oct 2016 16:38:22 +0100
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
	<148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
	<BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>
Message-ID: <ca8a9ae0-fede-1385-6618-271aad22c193@redhat.com>

On 05/10/16 15:39, Justin Sampson wrote:

> Deprecation is stronger than that -- it says "we're going to remove
> this, or we wish we could remove it because it's broken, so you'd
> better change your code a.s.a.p." -- e.g. Thread.stop().

We're moving Java to support some new ways of working, and these
changes will inevitably mean that some parts of the project will be
obsolescent.  We need to be able to flag the old ways of doing things
in some way.

Deprecation is a way to do that.  I don't think that Thread.stop() is
a typical case.

> My interpretation of Martin's comment is that using deprecation for
> things that aren't actually broken just means that people will live
> with lots of deprecation warnings in their code for years to
> come. This could be a perfect case for introducing a weaker
> alternative to deprecation, saying "here's something better that you
> should really be using for new code" -- e.g. ArrayList vs. Vector. I
> remember the Guava team talking about that a lot. Don't know if they
> ever implemented it.

OK.  But we really do need a way to do that.

Andrew.

From xueming.shen at oracle.com  Wed Oct  5 15:53:11 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Wed, 5 Oct 2016 08:53:11 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods) throws
	PatternSyntaxException
Message-ID: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>

Hi

Please help review

issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
webre: http://cr.openjdk.java.net/~sherman/8166261/webrev

The radix sanity check are missing  from 
hasNextByte/Short/Int/Long/BigInteger().
The only method we are doing now is useRadix(). The proposed change here 
is to
add the check into all above methods that take a radix.

Arguably it's an incompatible api change, but I don't expect it really 
breaks anyone's
code.  Need go through ccc if approved.

Thanks,
Sherman


From jsampson at guidewire.com  Wed Oct  5 14:39:10 2016
From: jsampson at guidewire.com (Justin Sampson)
Date: Wed, 5 Oct 2016 14:39:10 +0000
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
	<148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
Message-ID: <BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>

Deprecation is stronger than that -- it says "we're going to remove this, or we wish we could remove it because it's broken, so you'd better change your code a.s.a.p." -- e.g. Thread.stop(). My interpretation of Martin's comment is that using deprecation for things that aren't actually broken just means that people will live with lots of deprecation warnings in their code for years to come. This could be a perfect case for introducing a weaker alternative to deprecation, saying "here's something better that you should really be using for new code" -- e.g. ArrayList vs. Vector. I remember the Guava team talking about that a lot. Don't know if they ever implemented it.

Cheers,
Justin


-----Original Message-----
From: Concurrency-interest [mailto:concurrency-interest-bounces at cs.oswego.edu] On Behalf Of Andrew Haley
Sent: Wednesday, October 05, 2016 1:19 AM
To: Martin Buchholz; Remi Forax; Paul Sandoz
Cc: core-libs-dev; concurrency-interest
Subject: Re: [concurrency-interest] Deprecate all java.util.concurrent.*FieldUpdater

On 04/10/16 22:19, Martin Buchholz wrote:
> VarHandle is a reasonable replacement for FieldUpdaters, but it's not yet
> complete (where is accumulateAndGet?), and when do you deprecate something
> when the replacement won't be ubiquitous for 10 years?

Surely you have to start somewhere: deprecation is no more than saying
to programmers "Don't use this, use that."  And if you were leaning
over someone's shoulder that's what you would say.

Andrew.


_______________________________________________
Concurrency-interest mailing list
Concurrency-interest at cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

From jbluettduncan at gmail.com  Wed Oct  5 16:07:34 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Wed, 5 Oct 2016 17:07:34 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87pqoBtRQwPSvR-54e_sX33VgmcJWtmMJrZsBKVjWi1rg@mail.gmail.com>
	<2F084007-7660-4B2F-8B11-7B99EC0B9718@reini.net>
	<CAPMys851GThU6XiUsZY+2j=3WRqtYYHFFJPauQbrHvjMxoj-Lg@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
Message-ID: <CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>

Stuart, thank you very much for your continued review of this changeset,
and for finding those usages of CookieManager::get in Grepcode for me. I've
applied the comment you suggested for ModuleFinder and I've also fixed the
NetscapeCookieStore typo.

Stephen, thank you for getting back about DateTimeFormatter. It's not clear
to me what should be done with
TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. Do I
delete it; or do I change it to test that a null TemporalField param causes
a NullPointerException to be thrown; or do I do something else? May I have
your continued thoughts on this?

Kind regards,
Jonathan

From vladimir.x.ivanov at oracle.com  Wed Oct  5 16:55:00 2016
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 5 Oct 2016 19:55:00 +0300
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <ca8a9ae0-fede-1385-6618-271aad22c193@redhat.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
	<148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
	<BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>
	<ca8a9ae0-fede-1385-6618-271aad22c193@redhat.com>
Message-ID: <5dab5b50-bf80-0566-9a4a-f09b5523a092@oracle.com>

>> My interpretation of Martin's comment is that using deprecation for
>> things that aren't actually broken just means that people will live
>> with lots of deprecation warnings in their code for years to
>> come. This could be a perfect case for introducing a weaker
>> alternative to deprecation, saying "here's something better that you
>> should really be using for new code" -- e.g. ArrayList vs. Vector. I
>> remember the Guava team talking about that a lot. Don't know if they
>> ever implemented it.
>
> OK.  But we really do need a way to do that.
Doesn't enhanced deprecation [1] in 9 cover that?

Best regards,
Vladimir Ivanov

[1] http://openjdk.java.net/jeps/277

From aph at redhat.com  Wed Oct  5 17:04:25 2016
From: aph at redhat.com (Andrew Haley)
Date: Wed, 5 Oct 2016 18:04:25 +0100
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <5dab5b50-bf80-0566-9a4a-f09b5523a092@oracle.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
	<148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
	<BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>
	<ca8a9ae0-fede-1385-6618-271aad22c193@redhat.com>
	<5dab5b50-bf80-0566-9a4a-f09b5523a092@oracle.com>
Message-ID: <64dfab7d-a4d7-e096-a53a-dd9f4cc603e0@redhat.com>

On 05/10/16 17:55, Vladimir Ivanov wrote:
>>> My interpretation of Martin's comment is that using deprecation for
>>> things that aren't actually broken just means that people will live
>>> with lots of deprecation warnings in their code for years to
>>> come. This could be a perfect case for introducing a weaker
>>> alternative to deprecation, saying "here's something better that you
>>> should really be using for new code" -- e.g. ArrayList vs. Vector. I
>>> remember the Guava team talking about that a lot. Don't know if they
>>> ever implemented it.
>>
>> OK.  But we really do need a way to do that.
> Doesn't enhanced deprecation [1] in 9 cover that?

I can't tell.  As far as I can see there is no annotation which
covers exactly our case:

    the API is flawed and is impractical to fix,

    usage of the API is likely to lead to errors,

    the API has been superseded by another API,

    the API is obsolete,

    the API is experimental and is subject to incompatible changes,

    or any combination of the above.

we'd perhaps need to add

    this method over here is better in most cases

...which perhaps implies obsolescent rather than obsolete.

Andrew.


From naoto.sato at oracle.com  Wed Oct  5 17:25:19 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Wed, 5 Oct 2016 10:25:19 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods)
	throws PatternSyntaxException
In-Reply-To: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
References: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
Message-ID: <62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>

Looks good to me.

The test case could use IntStream.rangeClosed(Character.MIN_RADIX, 
Character.MAX_RADIX) for the good radixes, instead of hard coding ints.

Naoto

On 10/5/16 8:53 AM, Xueming Shen wrote:
> Hi
>
> Please help review
>
> issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
> webre: http://cr.openjdk.java.net/~sherman/8166261/webrev
>
> The radix sanity check are missing  from
> hasNextByte/Short/Int/Long/BigInteger().
> The only method we are doing now is useRadix(). The proposed change here
> is to
> add the check into all above methods that take a radix.
>
> Arguably it's an incompatible api change, but I don't expect it really
> breaks anyone's
> code.  Need go through ccc if approved.
>
> Thanks,
> Sherman
>

From patrick at reini.net  Wed Oct  5 17:56:47 2016
From: patrick at reini.net (Patrick Reinhart)
Date: Wed, 5 Oct 2016 19:56:47 +0200
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87pqoBtRQwPSvR-54e_sX33VgmcJWtmMJrZsBKVjWi1rg@mail.gmail.com>
	<2F084007-7660-4B2F-8B11-7B99EC0B9718@reini.net>
	<CAPMys851GThU6XiUsZY+2j=3WRqtYYHFFJPauQbrHvjMxoj-Lg@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a1...
Message-ID: <211A44D6-3279-435F-BC0D-EB119EAA0A5C@reini.net>

Hi Jonathan,

As soon you got all the changes together, we can go thru them and I will recreate the webrev?

-Patrick

> Am 05.10.2016 um 18:07 schrieb Jonathan Bluett-Duncan <jbluettduncan at gmail.com>:
> 
> Stuart, thank you very much for your continued review of this changeset,
> and for finding those usages of CookieManager::get in Grepcode for me. I've
> applied the comment you suggested for ModuleFinder and I've also fixed the
> NetscapeCookieStore typo.
> 
> Stephen, thank you for getting back about DateTimeFormatter. It's not clear
> to me what should be done with
> TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. Do I
> delete it; or do I change it to test that a null TemporalField param causes
> a NullPointerException to be thrown; or do I do something else? May I have
> your continued thoughts on this?
> 
> Kind regards,
> Jonathan


From lance.andersen at oracle.com  Wed Oct  5 18:29:55 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Wed, 5 Oct 2016 14:29:55 -0400
Subject: Review Request JDK-8166846: jdeps fails to generate module info
	if there is any class in unnamed package
In-Reply-To: <C3C8A61B-035C-4098-97D3-5AB40ED21189@oracle.com>
References: <C3C8A61B-035C-4098-97D3-5AB40ED21189@oracle.com>
Message-ID: <317FE4ED-14FD-41F3-87DA-C8265CECDE18@oracle.com>

Hi Mandy

The code changes look good.

A couple of minor suggestions

This looks good.  Only thought I had is 

err.genmoduleinfo.unnamed.package={0} contains unnamed package; not a valid module to generate module-info.java
perhaps change it to:

	{0} contains an unnamed package that is not allowed in a module.

err.genmoduleinfo.not.jarfile={0} is a modular JAR file; not valid for --generate-module-info option
I might tweak this to be:

	{0} is a modular JAR file and may not be specified with the ?generate-module-info option


HTH,

Best,
Lance
> On Oct 4, 2016, at 10:50 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8166846/webrev.00/
> 
> This improves jdeps error message to indicate that a JAR file containing unnamed package is not valid to generate module-info.java.
> 
> Mandy

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From Alan.Bateman at oracle.com  Wed Oct  5 18:32:08 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 5 Oct 2016 19:32:08 +0100
Subject: Review Request JDK-8167014: jdeps: Missing message:
	warn.skipped.entry
In-Reply-To: <BDD25366-BCFE-4EE1-B2AE-69338A617C38@oracle.com>
References: <BDD25366-BCFE-4EE1-B2AE-69338A617C38@oracle.com>
Message-ID: <803865ab-d9c1-b85c-f9d5-a4d07ca1ba43@oracle.com>

On 05/10/2016 03:52, Mandy Chung wrote:

> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167014/webrev.00/index.html
>
> This fixes the missing key in the resource bundle and also include the exception message of the invalid entry.  I verified with a JAR file with bad entries.
>
This looks okay to me.

-Alan

From mandy.chung at oracle.com  Wed Oct  5 18:49:32 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 5 Oct 2016 11:49:32 -0700
Subject: Review Request JDK-8166846: jdeps fails to generate module info
	if there is any class in unnamed package
In-Reply-To: <317FE4ED-14FD-41F3-87DA-C8265CECDE18@oracle.com>
References: <C3C8A61B-035C-4098-97D3-5AB40ED21189@oracle.com>
	<317FE4ED-14FD-41F3-87DA-C8265CECDE18@oracle.com>
Message-ID: <7124B3B1-681D-4F6B-A21A-BCB54CF05AEA@oracle.com>

Lance,

Thanks for the review. jdeps.properties updated:

err.genmoduleinfo.not.jarfile={0} is a modular JAR file that cannot be specified with the --generate-module-info option
err.genmoduleinfo.unnamed.package={0} contains an unnamed package that is not allowed in a module

Mandy

> On Oct 5, 2016, at 11:29 AM, Lance Andersen <lance.andersen at oracle.com> wrote:
> 
> Hi Mandy
> 
> The code changes look good.
> 
> A couple of minor suggestions
> 
> This looks good.  Only thought I had is 
> 
> err.genmoduleinfo.unnamed.package={0} contains unnamed package; not a valid module to generate module-info.java
> perhaps change it to:
> 
> 	{0} contains an unnamed package that is not allowed in a module.
> 
> err.genmoduleinfo.not.jarfile={0} is a modular JAR file; not valid for --generate-module-info option
> I might tweak this to be:
> 
> 	{0} is a modular JAR file and may not be specified with the ?generate-module-info option
> 
> 
> HTH,
> 
> Best,
> Lance
>> On Oct 4, 2016, at 10:50 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
>> 
>> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8166846/webrev.00/
>> 
>> This improves jdeps error message to indicate that a JAR file containing unnamed package is not valid to generate module-info.java.
>> 
>> Mandy
> 
> <oracle_sig_logo.gif>
> 
> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering 
> 1 Network Drive 
> Burlington, MA 01803
> Lance.Andersen at oracle.com
> 
> 
> 


From steve.drach at oracle.com  Wed Oct  5 19:27:25 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 5 Oct 2016 12:27:25 -0700
Subject: RFR: 8166460: jdk/internal/util/jar/TestVersionedStream gets
	Assertion error
Message-ID: <1EC05ECD-D562-459E-BA86-E7D755B650B0@oracle.com>

Hi,

Please review the following changeset that creates a replacement for the TestVersionedStream test.  The previous test occasionally failed on Linux hotspot nightly testing due to jar tool sometimes changing the order of the entries.  The new test specifically sets the order of the entries and tests both possible orders.

issue: https://bugs.openjdk.java.net/browse/JDK-8166460 <https://bugs.openjdk.java.net/browse/JDK-8166460>
webrev: http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/>

Thanks,
Steve


From scolebourne at joda.org  Wed Oct  5 19:32:59 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 5 Oct 2016 20:32:59 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87pqoBtRQwPSvR-54e_sX33VgmcJWtmMJrZsBKVjWi1rg@mail.gmail.com>
	<2F084007-7660-4B2F-8B11-7B99EC0B9718@reini.net>
	<CAPMys851GThU6XiUsZY+2j=3WRqtYYHFFJPauQbrHvjMxoj-Lg@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
Message-ID: <CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>

On 5 October 2016 at 17:07, Jonathan Bluett-Duncan
<jbluettduncan at gmail.com> wrote:
> Stephen, thank you for getting back about DateTimeFormatter. It's not clear
> to me what should be done with
> TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. Do I
> delete it; or do I change it to test that a null TemporalField param causes
> a NullPointerException to be thrown; or do I do something else? May I have
> your continued thoughts on this?

I think you should perform no change to DateTimeFormatter (other than
a comment) as part of this changeset. The behaviour of that
DateTimeFormatter method is subtle, and I now suspect that what we
have there might be the best option.

Stephen

From brent.christian at oracle.com  Wed Oct  5 19:38:10 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Wed, 5 Oct 2016 12:38:10 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
Message-ID: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>

Hi,

Please review my fix for 8151486, "Class.forName causes memory leak".

Bug:
https://bugs.openjdk.java.net/browse/JDK-8151486
Webrev:
http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/

The test case reproduces the bug as such:

With a SecurityManager enabled, a class ("ClassForName") is loaded from 
a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from 
that class makes a call to Class.forName() on the system classloader. 
(The specific class name isn't so important, I just used 
java.util.List).  The result is that the user's classloader is never 
collected.

The leak occurs because of the following:

Class.forName0() is passed the "caller" class, ClassForName.

JVM_FindClassFromCaller will "find a class with this name 
(java.util.List) in this loader (the system classloader), using the 
caller's (ClassForName's) protection domain.  A ProtectionDomain is 
created for ClassForName, with ProtectionDomain.classloader referring to 
LeakedClassLoader.

This PD is passed to ClassLoader.checkPackageAccess(), and is added to 
'domains' of the system classloader (ClassLoader.java line 643).  Thus, 
the system classloader holds a reference to the user's classloader via 
'domains'.

Nothing is ever removed from 'domains'.  In fact, besides being added 
to, I found no other uses of 'domains' - not in library code, not in 
hotspot.  (From my research, it's questionable if 'domains' was ever used).

Removal of 'domains' fixes the leak, and cleans out a little bit of 
unused code.

Automated building and testing (JPRT, RBT) looks fine.

Thanks!
-Brent


From Roger.Riggs at Oracle.com  Wed Oct  5 20:08:14 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 5 Oct 2016 16:08:14 -0400
Subject: [9] RFR of JDK-8085192: java/rmi/activation/Activatable tests
	fail intermittently due to "Port already in use"
In-Reply-To: <02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>
References: <3c0bdf63-f55c-5b93-6d74-58e26916cb3b@oracle.com>
	<f881b76f-2a76-5abd-3118-7f6fe12d9205@Oracle.com>
	<e3b084d3-de5e-700f-2f69-22f91a0f80bf@oracle.com>
	<02147682-541f-9746-b6da-2cc2800165b5@Oracle.com>
	<3E39A1E9-0764-404D-9824-85CFB75113FF@oracle.com>
	<E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
	<02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>
Message-ID: <2e13f9fa-cfeb-94bb-0ed5-6319c32b3e1f@Oracle.com>

Hi Chris,

Some comments,

BTW, there is a newer version of Webrev that provides convenient next 
and prev links...


* sun/rmi/server/Activation.java: 1973
  - I'd stick to the normal set of values for a boolean system property: 
use java.lang.Boolean.getProperty("sun.rmi....").

* With so many dependencies, how about adding the modules = to an 
appropriate TEST.properties file.

* With the extra security permissions and breaking into package private 
utilities
    this test is going to be more fragile and harder to work with.

* 
test/java/rmi/activation/Activatable/checkAnnotations/CheckAnnotations.java
  - line 184, leftover println...  (or incorrectly indented).
  - line 226 ditto?

* test/java/rmi/activation/Activatable/extLoadedImpl/ExtLoadedImplTest.java
+ 
test/java/rmi/activation/ActivateFailedException/activateFails/ActivateFails.java
+ 
test/java/rmi/activation/ActivationGroup/downloadActivationGroup/DownloadActivationGroup.java
+ 
test/java/rmi/activation/ActivationSystem/activeGroup/IdempotentActiveGroup.java
+ 
test/java/rmi/activation/ActivationSystem/modifyDescriptor/ModifyDescriptor.java
+ 
test/java/rmi/activation/ActivationSystem/stubClassesPermitted/StubClassesPermitted.java
+ 
test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java
+ test/java/rmi/activation/CommandEnvironment/SetChildEnv.java
+ 
test/java/rmi/activation/rmidViaInheritedChannel/InheritedChannelNotServerSocket.java
+ 
test/java/rmi/activation/rmidViaInheritedChannel/RmidViaInheritedChannel.java
+
  - Do these tests also need @module java.base/sun.nio.ch

* I wonder of the security.policy and rmid.security.policy files can be 
(mostly) shared.
   (but that's not really this issue)

* test/java/rmi/testlibrary/RMID.java
  - 139: typo "do no" -> "do not"

I'm vaguely not very comfortable with scraping the port number off stdout
and the inherited channel pieces seem like a lot of moving parts.

Roger

p.s.  Anyother idea
I assume not all platforms can allow separate processes to open server 
sockets to the same port.
If so, we would just have the client allocate a port (0), mark it 
non-exclusive and keep it open
while passing the port number to RMID. Only after RMID is started close 
the allocating socket.






On 10/3/2016 11:42 AM, Chris Hegarty wrote:
> Here is an updated version of this ( ready for review ):
>
>   http://cr.openjdk.java.net/~chegar/8085192_webrev.02/
>
> Changes from previous:
>
> 1) Updated Activation/rmid to NOT redirect stderr, if an
>    implementation specific system property is used ( we can
>    discuss the name )
>
> 2) For now, I added createRMIDOnEphemeralPort, rather than
>    change the current implementation, as it is being used in
>    other places. We can revert this once other usages are
>    updated and verified.
>
> -Chris.
>
> On 29/09/16 20:09, Chris Hegarty wrote:
>> On 29 Sep 2016, at 16:25, Chris Hegarty <chris.hegarty at oracle.com> 
>> wrote:
>>>
>>> I have asked Hamlin to hold off on this for a day or so.  I have an
>>> alternative proposal that eliminates the free port anti-pattern.
>>
>> It is possible to use the inheritedChannel mechanism to have the rmid
>> process create the server channel on an ephemeral port and report it
>> back to the test, i.e. remove the free port pattern.
>>
>> http://cr.openjdk.java.net/~chegar/8085192_webrev/
>>
>> 1) The port number is reported from rmid to the test over stdout.
>>
>> 2) All tests pass except CheckAnnotations.java, which looks for stderr
>>     ( see 3 below ). I think the stderr check can be removed, and the
>>     just check stdout.
>>
>> 3)  rmid, when using inheritChannel, redirects stderr to a tmp file, so
>>      it is not possible to get the processes stderr over the processes
>>      stderr pipe. I did?t find that this was an issue when testing ( 
>> other
>>      than having to clear out /tmp )
>>
>> 4) This could be expanded to other tests, outside of activation, to
>>      remove more usages of free port.
>>
>> This is not yet complete, I just want to share the idea to see if it 
>> is a
>> runner, or not.
>>
>> -Chris.
>>


From mandy.chung at oracle.com  Wed Oct  5 20:19:32 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 5 Oct 2016 13:19:32 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
Message-ID: <BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>


> On Oct 5, 2016, at 12:38 PM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> Hi,
> 
> Please review my fix for 8151486, "Class.forName causes memory leak".
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8151486
> Webrev:
> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/
> 

Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me.

Nit in the new test: a few long lines.  Good to have line breaks.

  64         ClassLoader classLoader = new URLClassLoader(new URL[]{jarFilePath.toFile().toURI().toURL()}) {

You can call Path.toUri().toURL()

  78         Path testClassesDir = FileSystems.getDefault().getPath(

You can replace this with Paths.get(?)

Otherwise looks good.

Mandy

From stuart.marks at oracle.com  Wed Oct  5 20:45:27 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Wed, 5 Oct 2016 13:45:27 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<eda7db40710b4552e359e6b7fcea7b27@reini.net>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
Message-ID: <c535426e-0fa1-0678-e4d3-6b23b8189ff1@oracle.com>

Stephen,

Thanks for the quick followup clarifications. I'm not entirely sure how you'd 
like to proceed; see discussion below.

Jonathan, also see below.


On 10/5/16 9:07 AM, Jonathan Bluett-Duncan wrote:
> Stuart, thank you very much for your continued review of this changeset, and for
> finding those usages of CookieManager::get in Grepcode for me. I've applied the
> comment you suggested for ModuleFinder and I've also fixed the
> NetscapeCookieStore typo.

Great!

> Stephen, thank you for getting back about DateTimeFormatter. It's not clear to
> me what should be done with
> TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. Do I
> delete it; or do I change it to test that a null TemporalField param causes a
> NullPointerException to be thrown; or do I do something else? May I have your
> continued thoughts on this?

OK, this is kind of subtle. This is a TCK (conformance) test, so it probably 
cannot simply be removed; it may need a specification change to clarify this 
case. I've filed JDK-8167222 to cover these issues. I've made a note in this bug 
regarding the potential change in DateTimeFormatter.withResolverFields() to use 
Set.of().

(There's an additional wrinkle with Set.of() aside from rejecting nulls; it also 
rejects duplicates by throwing IAE.)

In any case that code can't be changed to use Set.of() until the test/spec issue 
is resolved, so for the purposes of this changeset, I'd suggest simply removing 
the withResolverFields() comment from the webrev. We can revisit this during or 
after the resolution of JDK-8167222.

I think this clears all the issues, so you can probably go ahead and work with 
Patrick to update the webrev. And Patrick, thanks once again for hosting the webrev!

s'marks

From Roger.Riggs at Oracle.com  Wed Oct  5 21:05:21 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 5 Oct 2016 17:05:21 -0400
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <c535426e-0fa1-0678-e4d3-6b23b8189ff1@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<9BBC10F6-B4AE-400F-B202-8A7F2DD550C3@reini.net>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<c535426e-0fa1-0678-e4d3-6b23b8189ff1@oracle.com>
Message-ID: <227973cc-5d97-1b60-ab3c-8211e9e9e71a@Oracle.com>

Hi,

I would leave the test and code as is for the purposes of 8133273
and leave it to later to resolve 8167222.  There is no compelling need 
to change it.

$.02, Roger

On 10/5/2016 4:45 PM, Stuart Marks wrote:
> Stephen,
>
> Thanks for the quick followup clarifications. I'm not entirely sure 
> how you'd like to proceed; see discussion below.
>
> Jonathan, also see below.
>
>
> On 10/5/16 9:07 AM, Jonathan Bluett-Duncan wrote:
>> Stuart, thank you very much for your continued review of this 
>> changeset, and for
>> finding those usages of CookieManager::get in Grepcode for me. I've 
>> applied the
>> comment you suggested for ModuleFinder and I've also fixed the
>> NetscapeCookieStore typo.
>
> Great!
>
>> Stephen, thank you for getting back about DateTimeFormatter. It's not 
>> clear to
>> me what should be done with
>> TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. 
>> Do I
>> delete it; or do I change it to test that a null TemporalField param 
>> causes a
>> NullPointerException to be thrown; or do I do something else? May I 
>> have your
>> continued thoughts on this?
>
> OK, this is kind of subtle. This is a TCK (conformance) test, so it 
> probably cannot simply be removed; it may need a specification change 
> to clarify this case. I've filed JDK-8167222 to cover these issues. 
> I've made a note in this bug regarding the potential change in 
> DateTimeFormatter.withResolverFields() to use Set.of().
>
> (There's an additional wrinkle with Set.of() aside from rejecting 
> nulls; it also rejects duplicates by throwing IAE.)
>
> In any case that code can't be changed to use Set.of() until the 
> test/spec issue is resolved, so for the purposes of this changeset, 
> I'd suggest simply removing the withResolverFields() comment from the 
> webrev. We can revisit this during or after the resolution of 
> JDK-8167222.
>
> I think this clears all the issues, so you can probably go ahead and 
> work with Patrick to update the webrev. And Patrick, thanks once again 
> for hosting the webrev!
>
> s'marks


From naoto.sato at oracle.com  Wed Oct  5 21:08:20 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Wed, 5 Oct 2016 14:08:20 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
Message-ID: <d706c24c-4bcf-f426-ead7-091227201b67@oracle.com>

Typo in ClassForNameLeak.java? At line 103, "change" -> "chance"?

Naoto

On 10/5/16 12:38 PM, Brent Christian wrote:
> Hi,
>
> Please review my fix for 8151486, "Class.forName causes memory leak".
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8151486
> Webrev:
> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/
>
> The test case reproduces the bug as such:
>
> With a SecurityManager enabled, a class ("ClassForName") is loaded from
> a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from
> that class makes a call to Class.forName() on the system classloader.
> (The specific class name isn't so important, I just used
> java.util.List).  The result is that the user's classloader is never
> collected.
>
> The leak occurs because of the following:
>
> Class.forName0() is passed the "caller" class, ClassForName.
>
> JVM_FindClassFromCaller will "find a class with this name
> (java.util.List) in this loader (the system classloader), using the
> caller's (ClassForName's) protection domain.  A ProtectionDomain is
> created for ClassForName, with ProtectionDomain.classloader referring to
> LeakedClassLoader.
>
> This PD is passed to ClassLoader.checkPackageAccess(), and is added to
> 'domains' of the system classloader (ClassLoader.java line 643).  Thus,
> the system classloader holds a reference to the user's classloader via
> 'domains'.
>
> Nothing is ever removed from 'domains'.  In fact, besides being added
> to, I found no other uses of 'domains' - not in library code, not in
> hotspot.  (From my research, it's questionable if 'domains' was ever used).
>
> Removal of 'domains' fixes the leak, and cleans out a little bit of
> unused code.
>
> Automated building and testing (JPRT, RBT) looks fine.
>
> Thanks!
> -Brent
>

From david.holmes at oracle.com  Wed Oct  5 21:26:18 2016
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 6 Oct 2016 07:26:18 +1000
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
Message-ID: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>

On 6/10/2016 6:19 AM, Mandy Chung wrote:
>
>> On Oct 5, 2016, at 12:38 PM, Brent Christian <brent.christian at oracle.com> wrote:
>>
>> Hi,
>>
>> Please review my fix for 8151486, "Class.forName causes memory leak".
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8151486
>> Webrev:
>> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/
>>
>
> Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me.

What then is intended to keep PD alive? Or do we not need to keep them 
alive?

I have always assumed that domains was used to keep the PD alive for as 
long as the classloader is alive.

Thanks,
David
-----


> Nit in the new test: a few long lines.  Good to have line breaks.
>
>   64         ClassLoader classLoader = new URLClassLoader(new URL[]{jarFilePath.toFile().toURI().toURL()}) {
>
> You can call Path.toUri().toURL()
>
>   78         Path testClassesDir = FileSystems.getDefault().getPath(
>
> You can replace this with Paths.get(?)
>
> Otherwise looks good.
>
> Mandy
>

From brent.christian at oracle.com  Wed Oct  5 21:48:37 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Wed, 5 Oct 2016 14:48:37 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <d706c24c-4bcf-f426-ead7-091227201b67@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<d706c24c-4bcf-f426-ead7-091227201b67@oracle.com>
Message-ID: <a3bf1541-7885-a1c2-f4d1-3f33b231fb69@oracle.com>

Yes!  Good catch, thanks.
-Brent

On 10/5/16 2:08 PM, Naoto Sato wrote:
> Typo in ClassForNameLeak.java? At line 103, "change" -> "chance"?
>
> Naoto
>
> On 10/5/16 12:38 PM, Brent Christian wrote:
>> Hi,
>>
>> Please review my fix for 8151486, "Class.forName causes memory leak".
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8151486
>> Webrev:
>> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/
>>
>> The test case reproduces the bug as such:
>>
>> With a SecurityManager enabled, a class ("ClassForName") is loaded from
>> a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from
>> that class makes a call to Class.forName() on the system classloader.
>> (The specific class name isn't so important, I just used
>> java.util.List).  The result is that the user's classloader is never
>> collected.
>>
>> The leak occurs because of the following:
>>
>> Class.forName0() is passed the "caller" class, ClassForName.
>>
>> JVM_FindClassFromCaller will "find a class with this name
>> (java.util.List) in this loader (the system classloader), using the
>> caller's (ClassForName's) protection domain.  A ProtectionDomain is
>> created for ClassForName, with ProtectionDomain.classloader referring to
>> LeakedClassLoader.
>>
>> This PD is passed to ClassLoader.checkPackageAccess(), and is added to
>> 'domains' of the system classloader (ClassLoader.java line 643).  Thus,
>> the system classloader holds a reference to the user's classloader via
>> 'domains'.
>>
>> Nothing is ever removed from 'domains'.  In fact, besides being added
>> to, I found no other uses of 'domains' - not in library code, not in
>> hotspot.  (From my research, it's questionable if 'domains' was ever
>> used).
>>
>> Removal of 'domains' fixes the leak, and cleans out a little bit of
>> unused code.
>>
>> Automated building and testing (JPRT, RBT) looks fine.
>>
>> Thanks!
>> -Brent
>>

From mandy.chung at oracle.com  Wed Oct  5 22:55:20 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 5 Oct 2016 15:55:20 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
Message-ID: <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>


> On Oct 5, 2016, at 2:26 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 6/10/2016 6:19 AM, Mandy Chung wrote:
>> 
>> 
>> Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me.
> 
> What then is intended to keep PD alive? Or do we not need to keep them alive?
> 
> I have always assumed that domains was used to keep the PD alive for as long as the classloader is alive.


Note that PD is the protection domain of an initiating class that resolves a target type T.  PD is kept in T?s class loader L.  It?s not the protection domains of the classes defined by L.  VM keeps its own cache of protection domains that have been validated with a resolved type.  VM does not depend on ClassLoader::domains field to keep it alive.  IIUC, it?s kept as a weak reference in VM which is what we want here.

Brent did the archeology and found that the initial fix introduced the domains field attempted to use it.  But the initial fix was back out and reimplemented and domains is not used.  In any case, this field is not used and depended by VM. 

Mandy

From stuart.marks at oracle.com  Wed Oct  5 23:00:00 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Wed, 5 Oct 2016 16:00:00 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
Message-ID: <058f6ac5-21f4-00f3-76da-7378901000f4@oracle.com>



On 10/5/16 12:32 PM, Stephen Colebourne wrote:
> On 5 October 2016 at 17:07, Jonathan Bluett-Duncan
> <jbluettduncan at gmail.com> wrote:
>> Stephen, thank you for getting back about DateTimeFormatter. It's not clear
>> to me what should be done with
>> TCKDateTimeFormatter::test_resolverFields_listOfOneNull in this case. Do I
>> delete it; or do I change it to test that a null TemporalField param causes
>> a NullPointerException to be thrown; or do I do something else? May I have
>> your continued thoughts on this?
>
> I think you should perform no change to DateTimeFormatter (other than
> a comment) as part of this changeset. The behaviour of that
> DateTimeFormatter method is subtle, and I now suspect that what we
> have there might be the best option.

I had recommended removing the comment from DateTimeFormatter, but if Stephen 
wants the comment in, that's fine with me. In fact I'll defer to what Stephen 
(and Roger Riggs) want with this code, since they're the maintainers.

s'marks

From david.holmes at oracle.com  Wed Oct  5 23:02:56 2016
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 6 Oct 2016 09:02:56 +1000
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
	<1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
Message-ID: <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>

On 6/10/2016 8:55 AM, Mandy Chung wrote:
>
>> On Oct 5, 2016, at 2:26 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 6/10/2016 6:19 AM, Mandy Chung wrote:
>>>
>>>
>>> Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me.
>>
>> What then is intended to keep PD alive? Or do we not need to keep them alive?
>>
>> I have always assumed that domains was used to keep the PD alive for as long as the classloader is alive.
>
>
> Note that PD is the protection domain of an initiating class that resolves a target type T.  PD is kept in T?s class loader L.  It?s not the protection domains of the classes defined by L.  VM keeps its own cache of protection domains that have been validated with a resolved type.  VM does not depend on ClassLoader::domains field to keep it alive.  IIUC, it?s kept as a weak reference in VM which is what we want here.

Okay but this will still affect the lifecycle of the PDs because without 
the strong reference in L, those weak references in the VM will quickly 
be cleared. I'm unclear what the implications of this might be, but I 
doubt regular short-lived tests would expose any issues.

David
-----

> Brent did the archeology and found that the initial fix introduced the domains field attempted to use it.  But the initial fix was back out and reimplemented and domains is not used.  In any case, this field is not used and depended by VM.
>
> Mandy
>

From mandy.chung at oracle.com  Wed Oct  5 23:19:20 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 5 Oct 2016 16:19:20 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
	<1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
	<300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>
Message-ID: <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com>


> On Oct 5, 2016, at 4:02 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 6/10/2016 8:55 AM, Mandy Chung wrote:
>> 
>> 
>> Note that PD is the protection domain of an initiating class that resolves a target type T.  PD is kept in T?s class loader L.  It?s not the protection domains of the classes defined by L.  VM keeps its own cache of protection domains that have been validated with a resolved type.  VM does not depend on ClassLoader::domains field to keep it alive.  IIUC, it?s kept as a weak reference in VM which is what we want here.
> 
> Okay but this will still affect the lifecycle of the PDs because without the strong reference in L, those weak references in the VM will quickly be cleared. I'm unclear what the implications of this might be, but I doubt regular short-lived tests would expose any issues.

There is no issue there.

PD will be GC?ed together when the classes along with its class loader are unloaded. These class loaders may be one or more and they are L1, L2 ?. != L.

Mandy

From brian.burkhalter at oracle.com  Wed Oct  5 23:31:33 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Wed, 5 Oct 2016 16:31:33 -0700
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
Message-ID: <DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>

Hi Lucy,

On Oct 4, 2016, at 5:35 PM, Lu, Yingqi <yingqi.lu at intel.com> wrote:

> Thank you very much for reviewing the patch and providing the detailed error/warning messages. They are very helpful!

You are welcome.

> We will look into the issues and solve them in the next version of the patch. Test cases for O_DIRECT and modified copyright year will be added in next version too.

Given that the functionality of O_DIRECT on Linux appears to be supported by other interfaces on OS X, Solaris, and Windows, I wonder whether the patch will need to be refactored in some way to accommodate these other operating systems? For reference it looks as if direct I/O on OS X uses the F_NOCACHE command of fcntl(2) [1] (although per some online comments this might have some problems), Solaris uses the advice argument of directio(3c) [2], and Windows uses a combination of flags passed to CreateFile() [3, 4].

Thanks,

Brian

[1] https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man2/fcntl.2.html
[2] https://docs.oracle.com/cd/E26505_01/html/816-5168/directio-3c.html
[3] https://support.microsoft.com/en-us/kb/99794
[4] https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx


From david.holmes at oracle.com  Wed Oct  5 23:43:21 2016
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 6 Oct 2016 09:43:21 +1000
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
	<1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
	<300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>
	<56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com>
Message-ID: <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com>

On 6/10/2016 9:19 AM, Mandy Chung wrote:
>
>> On Oct 5, 2016, at 4:02 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 6/10/2016 8:55 AM, Mandy Chung wrote:
>>>
>>>
>>> Note that PD is the protection domain of an initiating class that resolves a target type T.  PD is kept in T?s class loader L.  It?s not the protection domains of the classes defined by L.  VM keeps its own cache of protection domains that have been validated with a resolved type.  VM does not depend on ClassLoader::domains field to keep it alive.  IIUC, it?s kept as a weak reference in VM which is what we want here.
>>
>> Okay but this will still affect the lifecycle of the PDs because without the strong reference in L, those weak references in the VM will quickly be cleared. I'm unclear what the implications of this might be, but I doubt regular short-lived tests would expose any issues.
>
> There is no issue there.
>
> PD will be GC?ed together when the classes along with its class loader are unloaded. These class loaders may be one or more and they are L1, L2 ?. != L.

Okay, thanks for walking me through this.

David

> Mandy
>

From scolebourne at joda.org  Thu Oct  6 08:38:46 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Thu, 6 Oct 2016 09:38:46 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <058f6ac5-21f4-00f3-76da-7378901000f4@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<058f6ac5-21f4-00f3-76da-7378901000f4@oracle.com>
Message-ID: <CACzrW9CixpACPdqng58AaEcsfaqFDE8OqS+e=0reyckLsV3Rcg@mail.gmail.com>

On 6 October 2016 at 00:00, Stuart Marks <stuart.marks at oracle.com> wrote:
>> I think you should perform no change to DateTimeFormatter (other than
>> a comment) as part of this changeset. The behaviour of that
>> DateTimeFormatter method is subtle, and I now suspect that what we
>> have there might be the best option.
>
> I had recommended removing the comment from DateTimeFormatter, but if
> Stephen wants the comment in, that's fine with me. In fact I'll defer to
> what Stephen (and Roger Riggs) want with this code, since they're the
> maintainers.

I think it makes sense to leave the new comment in. All further change
should be under 8167222.

Stephen

From stanislav.smirnov at oracle.com  Thu Oct  6 12:19:47 2016
From: stanislav.smirnov at oracle.com (Stanislav Smirnov)
Date: Thu, 6 Oct 2016 15:19:47 +0300
Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under
	test/lib
In-Reply-To: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
Message-ID: <4F4B0746-C6D5-400A-95BA-EFDD211C5DD5@oracle.com>

adding core-libs-dev mailing list, since my change affect some files for testing APIs in java.util.function

Best regards,
Stanislav Smirnov


> On 05 Oct 2016, at 19:44, Stanislav Smirnov <stanislav.smirnov at oracle.com> wrote:
> 
> Hi,
> 
> Please review this fix for JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>.
> This one is similar to another one, I have sent earlier.
> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements.
> 
> There are two files from test/lib/jdk do I need to include jdk9_dev mailing list?
> 
> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>
> Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ <http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/>
> Best regards,
> Stanislav Smirnov
> 
> 
> 
> 
> 


From denis.kononenko at oracle.com  Thu Oct  6 16:37:31 2016
From: denis.kononenko at oracle.com (Denis Kononenko)
Date: Thu, 6 Oct 2016 19:37:31 +0300
Subject: [9] RFR: 8167240: Write new tests to cover functionality of existing
	'jimage' options
Message-ID: <6b4570ad-e700-4f38-df8d-c91ea8130bad@oracle.com>

Hi,

Could someone please review these new tests for jimage utility.

There're 5 new files containing tests to cover use cases for 'info', 
'list', 'extract' and other options. No new tests for 'verify'.

BUGURL: https://bugs.openjdk.java.net/browse/JDK-8167240
WEBREV: http://cr.openjdk.java.net/~dkononenko/8167240/webrev.00/


Thank you,
Denis.

From alexandre.iline at oracle.com  Thu Oct  6 18:18:09 2016
From: alexandre.iline at oracle.com (Alexandre (Shura) Iline)
Date: Thu, 6 Oct 2016 11:18:09 -0700
Subject: [9] RFR: 8167240: Write new tests to cover functionality of
	existing 'jimage' options
In-Reply-To: <6b4570ad-e700-4f38-df8d-c91ea8130bad@oracle.com>
References: <6b4570ad-e700-4f38-df8d-c91ea8130bad@oracle.com>
Message-ID: <FCFE0E4D-35D0-47FA-87CC-A30C13DB6DFB@oracle.com>

Hi, Denis,

By an agreement, @modules should go before any action tag. You use it after @build.

Shura

> On Oct 6, 2016, at 9:37 AM, Denis Kononenko <denis.kononenko at oracle.com> wrote:
> 
> Hi,
> 
> Could someone please review these new tests for jimage utility.
> 
> There're 5 new files containing tests to cover use cases for 'info', 'list', 'extract' and other options. No new tests for 'verify'.
> 
> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8167240
> WEBREV: http://cr.openjdk.java.net/~dkononenko/8167240/webrev.00/
> 
> 
> Thank you,
> Denis.


From stuart.marks at oracle.com  Thu Oct  6 19:39:18 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Thu, 6 Oct 2016 12:39:18 -0700
Subject: [concurrency-interest] Deprecate all
	java.util.concurrent.*FieldUpdater
In-Reply-To: <64dfab7d-a4d7-e096-a53a-dd9f4cc603e0@redhat.com>
References: <1761941304.1037912.1475613121083.JavaMail.zimbra@u-pem.fr>
	<CA+kOe090sCv5v8MukaESSj75ZtaDfQVUXk+XGUFuaPfjPXfVCw@mail.gmail.com>
	<148dc564-1632-ecb1-8f65-05de407cfbfd@redhat.com>
	<BN3PR05MB25162EBB74D923EF9EA05884D1C40@BN3PR05MB2516.namprd05.prod.outlook.com>
	<ca8a9ae0-fede-1385-6618-271aad22c193@redhat.com>
	<5dab5b50-bf80-0566-9a4a-f09b5523a092@oracle.com>
	<64dfab7d-a4d7-e096-a53a-dd9f4cc603e0@redhat.com>
Message-ID: <5e95d017-5446-d223-2806-975b2d39c675@oracle.com>



On 10/5/16 10:04 AM, Andrew Haley wrote:
> On 05/10/16 17:55, Vladimir Ivanov wrote:
>>> On 10/5/16 7:39 AM, Justin Sampson wrote:
>>>> My interpretation of Martin's comment is that using deprecation for
>>>> things that aren't actually broken just means that people will live
>>>> with lots of deprecation warnings in their code for years to
>>>> come. This could be a perfect case for introducing a weaker
>>>> alternative to deprecation, saying "here's something better that you
>>>> should really be using for new code" -- e.g. ArrayList vs. Vector.
>> Doesn't enhanced deprecation [1] in 9 cover that?
>
> I can't tell.  As far as I can see there is no annotation which
> covers exactly our case:
>
>     the API is flawed and is impractical to fix,
>
>     usage of the API is likely to lead to errors,
>
>     the API has been superseded by another API,
>
>     the API is obsolete,
>
>     the API is experimental and is subject to incompatible changes,
>
>     or any combination of the above.
>
> we'd perhaps need to add
>
>     this method over here is better in most cases
>
> ...which perhaps implies obsolescent rather than obsolete.


(Vladimir, thanks for mentioning JEP 277. [1])

Andrew, all,

The list of criteria from JEP 277 isn't intended to be restrictive. It turns out 
that the criteria for deprecating something are quite subjective, and it's 
really hard to boil them down to specific reasons or attributes or whatever.

To my eye, "has been superseded by" includes "this method over here is better in 
most cases" but I'm sure we could find differences if we went looking for them.

The main contribution of JEP 277 in this area is the addition of the boolean 
forRemoval element to the @Deprecated annotation. If true, it means there is a 
plan to remove the annotated API -- for the JDK, most likely in the next major 
release. If false, it means there are no plans to remove the API at present.

To get back to this specific case, if you're looking over the shoulder of 
someone who's using *FieldUpdater, and your reaction is "you should be using 
VarHandles instead" then that's an argument for deprecating the FieldUpdaters 
with forRemoval=false.

On the other hand, if there are still things you can do with FieldUpdaters that 
you cannot (yet) do with VarHandles or some other construct, that's an argument 
for *not* deprecating the FieldUpdaters (yet).

I'm not familiar enough with the current state of things in this area to make an 
informed recommendation here.

s'marks


[1] http://openjdk.java.net/jeps/277

From steve.drach at oracle.com  Thu Oct  6 20:50:30 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Thu, 6 Oct 2016 13:50:30 -0700
Subject: RFR: 8167237: Jar tool can not correctly find/process the --release
	option if it occurs before the file list
Message-ID: <D299E831-210E-42DD-BBC8-3F8107B416ED@oracle.com>

Hi,

Please review the changeset that fixes the problem of not ?seeing? the jar tool ?release  option if it is not preceded by anything other than gnu-style options.  It?s a simple one line change to process ?release the same way as -C.

issue: https://bugs.openjdk.java.net/browse/JDK-8167237 <https://bugs.openjdk.java.net/browse/JDK-8167237>webrev: http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/>

Thanks,
Steve


From sundararajan.athijegannathan at oracle.com  Fri Oct  7 03:27:19 2016
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Fri, 7 Oct 2016 08:57:19 +0530
Subject: RFR:8055033: Shell tests for jrunscript don't pass through VM
	options
In-Reply-To: <89ec1fe4-5e9c-4013-bc15-cfc142aebd2a@default>
References: <89ec1fe4-5e9c-4013-bc15-cfc142aebd2a@default>
Message-ID: <f140d458-fa36-c0d6-c148-2cbde5141efb@oracle.com>

+1

-Sundar


On 10/5/2016 7:51 PM, Srinivas Dama wrote:
> Hi,
>
> Please review http://cr.openjdk.java.net/~sdama/8055033/webrev.00/ 
> for https://bugs.openjdk.java.net/browse/JDK-8055033 
>
> When launching java and javac from shell script, ${TESTVMOPTS}, ${TESTJAVAOPTS} and ${TESTTOOLVMOPTS}, ${TESTJAVACOPTS} should be 
> passed in respectively since jtreg sets these environment variables.
>
> Regards,
> Srinivas
>


From anubhav.meena at oracle.com  Fri Oct  7 08:42:24 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Fri, 7 Oct 2016 14:12:24 +0530
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
Message-ID: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>

Hi all,

Please review.

Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>

Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.

Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>


-- 
Thanks and Regards,
Anubhav

From jbluettduncan at gmail.com  Fri Oct  7 10:00:52 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Fri, 7 Oct 2016 11:00:52 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CACzrW9CixpACPdqng58AaEcsfaqFDE8OqS+e=0reyckLsV3Rcg@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<058f6ac5-21f4-00f3-76da-7378901000f4@oracle.com>
	<CACzrW9CixpACPdqng58AaEcsfaqFDE8OqS+e=0reyckLsV3Rcg@mail.gmail.com>
Message-ID: <CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>

Hi all,

Sorry for the delayed response, I've been busy lately with university and
other things.

Thank you all for your comments. I'll leave the DateTimeFormatter comment
in, as you requested Stephen and Roger, and I'll work again with Patrick as
soon as I'm ready.

Kind regards,
Jonathan

On 6 October 2016 at 09:38, Stephen Colebourne <scolebourne at joda.org> wrote:

> On 6 October 2016 at 00:00, Stuart Marks <stuart.marks at oracle.com> wrote:
> >> I think you should perform no change to DateTimeFormatter (other than
> >> a comment) as part of this changeset. The behaviour of that
> >> DateTimeFormatter method is subtle, and I now suspect that what we
> >> have there might be the best option.
> >
> > I had recommended removing the comment from DateTimeFormatter, but if
> > Stephen wants the comment in, that's fine with me. In fact I'll defer to
> > what Stephen (and Roger Riggs) want with this code, since they're the
> > maintainers.
>
> I think it makes sense to leave the new comment in. All further change
> should be under 8167222.
>
> Stephen
>

From tim.bell at oracle.com  Fri Oct  7 13:40:25 2016
From: tim.bell at oracle.com (Tim Bell)
Date: Fri, 7 Oct 2016 06:40:25 -0700
Subject: RFR: JDK-8166648,,jib make run-test for langtools results in
	intermittent failures on windows-x86
Message-ID: <2b5596b5-8340-8936-f891-8220982a8f78@oracle.com>

Hello

The change of HotSpot runtimes on 32-bit Windows [1] has been a source 
of langtools test failures due to memory space exhaustion.

On our Windows build/test clients, the combination of the C2 runtime and 
parallel GC causes about 20 extra threads to be started in each VM. 
These added thread stacks, plus heap management in C2, result in too 
much memory pressure to fit the workload into 32-bit VMs.

This fix will run langtools jtreg tests using the serial garbage 
collector instead of using the default collector (parallel GC).

Testing:
    -5 out of 5 jobs (with the fix) passed langtools testing
    -4 out of 5 control jobs (without the fix) failed testing in 
windows_i586_6.3-product-c2-langtools_jtreg


Thanks in advance-

Tim

[1] JDK-8154209 "Remove client VM from default JIB profile on 
windows-x86 and linux-x86"
https://bugs.openjdk.java.net/browse/JDK-8154209

From erik.joelsson at oracle.com  Fri Oct  7 13:52:16 2016
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 7 Oct 2016 15:52:16 +0200
Subject: RFR: JDK-8166648,,jib make run-test for langtools results in
	intermittent failures on windows-x86
In-Reply-To: <2b5596b5-8340-8936-f891-8220982a8f78@oracle.com>
References: <2b5596b5-8340-8936-f891-8220982a8f78@oracle.com>
Message-ID: <72993b07-963a-ff6d-a01c-d11beb463cca@oracle.com>

Hello Tim,

Assuming this is the webrev: 
http://cr.openjdk.java.net/~tbell/8166648/webrev/

It looks ok to me.

/Erik

On 2016-10-07 15:40, Tim Bell wrote:
> Hello
>
> The change of HotSpot runtimes on 32-bit Windows [1] has been a source 
> of langtools test failures due to memory space exhaustion.
>
> On our Windows build/test clients, the combination of the C2 runtime 
> and parallel GC causes about 20 extra threads to be started in each 
> VM. These added thread stacks, plus heap management in C2, result in 
> too much memory pressure to fit the workload into 32-bit VMs.
>
> This fix will run langtools jtreg tests using the serial garbage 
> collector instead of using the default collector (parallel GC).
>
> Testing:
>    -5 out of 5 jobs (with the fix) passed langtools testing
>    -4 out of 5 control jobs (without the fix) failed testing in 
> windows_i586_6.3-product-c2-langtools_jtreg
>
>
> Thanks in advance-
>
> Tim
>
> [1] JDK-8154209 "Remove client VM from default JIB profile on 
> windows-x86 and linux-x86"
> https://bugs.openjdk.java.net/browse/JDK-8154209


From tim.bell at oracle.com  Fri Oct  7 14:15:33 2016
From: tim.bell at oracle.com (Tim Bell)
Date: Fri, 7 Oct 2016 07:15:33 -0700
Subject: RFR: JDK-8166648,,jib make run-test for langtools results in
	intermittent failures on windows-x86
In-Reply-To: <2b5596b5-8340-8936-f891-8220982a8f78@oracle.com>
References: <2b5596b5-8340-8936-f891-8220982a8f78@oracle.com>
Message-ID: <79be1fa5-7ef7-fb2d-de6e-c7c8d3b16759@oracle.com>

Again with the review link .. see below:

On 10/07/16 06:40, Tim Bell wrote:
> Hello
>
> The change of HotSpot runtimes on 32-bit Windows [1] has been a source
> of langtools test failures due to memory space exhaustion.
>
> On our Windows build/test clients, the combination of the C2 runtime and
> parallel GC causes about 20 extra threads to be started in each VM.
> These added thread stacks, plus heap management in C2, result in too
> much memory pressure to fit the workload into 32-bit VMs.
>
> This fix will run langtools jtreg tests using the serial garbage
> collector instead of using the default collector (parallel GC).

http://cr.openjdk.java.net/~tbell/8166648/

>
> Testing:
>    -5 out of 5 jobs (with the fix) passed langtools testing
>    -4 out of 5 control jobs (without the fix) failed testing in
> windows_i586_6.3-product-c2-langtools_jtreg
>
>
> Thanks in advance-
>
> Tim
>
> [1] JDK-8154209 "Remove client VM from default JIB profile on
> windows-x86 and linux-x86"
> https://bugs.openjdk.java.net/browse/JDK-8154209


From huizhe.wang at oracle.com  Fri Oct  7 16:10:01 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Fri, 07 Oct 2016 09:10:01 -0700
Subject: RFR (JAXP) 8139584: XMLStreamWriterImpl does not write 'standalone'
	property
Message-ID: <57F7C8D9.8010705@oracle.com>

Hi,

Please review a fix for declaring the 'standalone' attribute.

JBS: https://bugs.openjdk.java.net/browse/JDK-8139584
webrev: http://cr.openjdk.java.net/~joehw/jdk9/8139584/webrev/

Thanks,
Joe

From lance.andersen at oracle.com  Fri Oct  7 17:07:30 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 7 Oct 2016 13:07:30 -0400
Subject: RFR (JAXP) 8139584: XMLStreamWriterImpl does not write
	'standalone' property
In-Reply-To: <57F7C8D9.8010705@oracle.com>
References: <57F7C8D9.8010705@oracle.com>
Message-ID: <B4DC8E27-E8D3-4144-8F76-4C87D31BA6D8@oracle.com>

Hi Joe,

Looks fine

Best
Lance
> On Oct 7, 2016, at 12:10 PM, Joe Wang <huizhe.wang at oracle.com> wrote:
> 
> Hi,
> 
> Please review a fix for declaring the 'standalone' attribute.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8139584
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8139584/webrev/
> 
> Thanks,
> Joe

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From stuart.marks at oracle.com  Fri Oct  7 17:20:28 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 7 Oct 2016 10:20:28 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and flatMap()
Message-ID: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>

Hi all,

Please review this small API adjustment to Optional.or and flatMap to add 
wildcards. This provides a bit more flexibility to callers about the types of 
functions they can provide to these methods.

The or() method is new in 9 so there is no compatibility issue.

The flatMap() method was introduced in 8, so this is a change to an existing 
API. There shouldn't be a binary compatibility issue, since the method's erasure 
doesn't change. I *think* it is source compatible, as anything that was accepted 
by the old signature:

     flatMap(Function<? super T, Optional<U>> mapper)

should also be accepted by the new signature:

     flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)

But there may be some subtle issues of which I'm unaware.

Bug report:

	https://bugs.openjdk.java.net/browse/JDK-8152617

Webrev:

	http://cr.openjdk.java.net/~smarks/reviews/8152617/webrev.0/

Thanks,

s'marks

From huizhe.wang at oracle.com  Fri Oct  7 17:52:01 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Fri, 07 Oct 2016 10:52:01 -0700
Subject: RFR (JAXP) 8139584: XMLStreamWriterImpl does not write
	'standalone' property
In-Reply-To: <B4DC8E27-E8D3-4144-8F76-4C87D31BA6D8@oracle.com>
References: <57F7C8D9.8010705@oracle.com>
	<B4DC8E27-E8D3-4144-8F76-4C87D31BA6D8@oracle.com>
Message-ID: <57F7E0C1.1070208@oracle.com>

Thanks Lance!

On 10/7/16, 10:07 AM, Lance Andersen wrote:
> Hi Joe,
>
> Looks fine
>
> Best
> Lance
>> On Oct 7, 2016, at 12:10 PM, Joe Wang <huizhe.wang at oracle.com 
>> <mailto:huizhe.wang at oracle.com>> wrote:
>>
>> Hi,
>>
>> Please review a fix for declaring the 'standalone' attribute.
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8139584
>> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8139584/webrev/ 
>> <http://cr.openjdk.java.net/%7Ejoehw/jdk9/8139584/webrev/>
>>
>> Thanks,
>> Joe
>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif><http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance 
> Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>
>
>
>

From paul.sandoz at oracle.com  Fri Oct  7 18:23:41 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 7 Oct 2016 11:23:41 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
Message-ID: <85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>


> On 7 Oct 2016, at 10:20, Stuart Marks <stuart.marks at oracle.com> wrote:
> 
> Hi all,
> 
> Please review this small API adjustment to Optional.or and flatMap to add wildcards. This provides a bit more flexibility to callers about the types of functions they can provide to these methods.
> 
> The or() method is new in 9 so there is no compatibility issue.
> 
> The flatMap() method was introduced in 8, so this is a change to an existing API. There shouldn't be a binary compatibility issue, since the method's erasure doesn't change. I *think* it is source compatible, as anything that was accepted by the old signature:

Source computability should be ok IICU because Optional is final and there are no overriding methods.

> 
>    flatMap(Function<? super T, Optional<U>> mapper)
> 
> should also be accepted by the new signature:
> 
>    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
> 
> But there may be some subtle issues of which I'm unaware.
> 

Optional is final so why do you need to express ?? extends Optional? ?

Paul.

> Bug report:
> 
> 	https://bugs.openjdk.java.net/browse/JDK-8152617
> 
> Webrev:
> 
> 	http://cr.openjdk.java.net/~smarks/reviews/8152617/webrev.0/
> 
> Thanks,
> 
> s'marks


From brent.christian at oracle.com  Fri Oct  7 18:25:53 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Fri, 7 Oct 2016 11:25:53 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
	<1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
	<300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>
	<56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com>
	<508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com>
Message-ID: <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com>

On 10/5/16 4:43 PM, David Holmes wrote:
>>
>>> Okay but this will still affect the lifecycle of the PDs because
>>> without the strong reference in L, those weak references in the VM
>>> will quickly be cleared.

There's also a strong reference held by the Class object itself (on the 
VM side [1]).

Thanks for having a look, folks.  I've applied Naoto and Mandy's 
suggestions for the test case and updated the webrev in place with what 
I plan to push.

http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/

-Brent

1. 
http://hg.openjdk.java.net/jdk9/dev/hotspot/file/fec31089c2ef/src/share/vm/classfile/javaClasses.cpp#l917

From mandy.chung at oracle.com  Fri Oct  7 18:42:11 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 7 Oct 2016 11:42:11 -0700
Subject: RFR 8151486 : Class.forName causes memory leak
In-Reply-To: <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com>
References: <bcdeeebe-535d-57f5-5574-e7b013a2400c@oracle.com>
	<BF641F06-FA74-4CCD-8059-6BA7A7103D36@oracle.com>
	<70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com>
	<1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com>
	<300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com>
	<56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com>
	<508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com>
	<582a2109-b3bb-984d-0090-00d19ad14940@oracle.com>
Message-ID: <D67063F7-DED7-4D3C-A66B-85AE408019E7@oracle.com>


> On Oct 7, 2016, at 11:25 AM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> On 10/5/16 4:43 PM, David Holmes wrote:
>>> 
>>>> Okay but this will still affect the lifecycle of the PDs because
>>>> without the strong reference in L, those weak references in the VM
>>>> will quickly be cleared.
> 
> There's also a strong reference held by the Class object itself (on the VM side [1]).
> 

Yes this is the protection domain of the Class itself (i.e. PD associated with its code source) that was set when the class is defined.

> Thanks for having a look, folks.  I've applied Naoto and Mandy's suggestions for the test case and updated the webrev in place with what I plan to push.
> 
> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/

It might be good to refactor System.getProperty("test.classes", ".?) as a static final field.  Otherwise looks good.

No need for a new webrev.

Mandy

From stuart.marks at oracle.com  Fri Oct  7 19:22:09 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 7 Oct 2016 12:22:09 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
Message-ID: <9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>



On 10/7/16 11:23 AM, Paul Sandoz wrote:
>>    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>
> Optional is final so why do you need to express ?? extends Optional? ?

The short answer is, it doesn't work if you don't have it. :-)

The theoretical answer is that in this context, "? extends P<Q>" means "some 
subtype of P<Q>" and not necessarily just a subclass of P.

(And even though Optional is final, it's not "permanently final" in that a 
hypothetical future version of the class library might change it to non-final, 
allowing subclasses.)

This issue is covered in Angelika Langer's Generics FAQ entry, "What do 
multi-level (i.e., nested) wildcards mean?" [1] Note that the answer begins, "It 
depends." :-) I also note in passing that I've read this about five times and 
I'm still not quite sure I understand it entirely.

For me, the best explanation comes from looking at examples. First, the history 
is that the signature in Java 8 is:

   #1    flatMap(Function<? super T, Optional<U>> mapper)

I believe R?mi originally proposed something like this (although it was about 
or(), the same issue applies to flatMap()):

   #2    flatMap(Function<? super T, ? extends Optional<U>> mapper)

The suggested fix that ended up in bug report was this:

   #3    flatMap(Function<? super T, Optional<? extends U>> mapper)

But this doesn't work for reasons I explain below. Finally, I'm proposing:

   #4    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)

In researching old email threads and chasing links, I ran across an example from 
Stefan Zobel [2] that fails with #3. He had an alternative that didn't seem 
quite right to me, so I ended up with #4.

I've adapted Stefan's example as follows:

	Optional<Integer> oi = Optional.empty();
	Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
	Optional<CharSequence> ocs = oi.flatMap(fm);

The flatmapper function 'fm' returns Optional<StringBuilder>. In the assignment 
on the last line, U is CharSequence, so we can compare this to the various 
signatures shown above with U filled in.

Case #1 fails because Optional<StringBuilder> is incompatible with 
Optional<CharSequence>. This is the usual "generics are invariant thing". Even 
though SB is a subtype of CS, Optional<SB> isn't a subtype of Optional<CS>.

Case #2 fails because adding a wildcard there doesn't help matters, since 
Optional<SB> is still unrelated to Optional<CS>.

Now for the tricky part. :-)

Surely case #3 should work, because adding an inner wildcard provides a 
subtyping relationship, so Optional<SB> is a subtype of Optional<? extends CS>.

That much is true, but these are nested within the Function<> generic type, so 
the "generics are invariant" rule applies again. Thus,

     Function<..., Optional<StringBuilder>>

is not a subtype of

     Function<..., Optional<? extends CharSequence>>

To get around this, we have to add the outer wildcard as well, so that

     Function<..., Optional<StringBuilder>>

is a subtype of

     Function<..., ? extends Optional<? extends CharSequence>>

So that's what ended up in the signature.

Similar analysis applies to the or() case.

Now awaiting a message from R?mi telling me my explanation is incorrect in 3... 
2... 1... :-) :-)

s'marks



[1] 
http://angelikalanger.com/GenericsFAQ/FAQSections/TypeArguments.html#What%20do%20multilevel%20wildcards%20mean?

[2] https://sourceforge.net/p/streamsupport/tickets/125/#2d90


From spliterator at gmail.com  Fri Oct  7 19:40:16 2016
From: spliterator at gmail.com (Stefan Zobel)
Date: Fri, 7 Oct 2016 21:40:16 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
Message-ID: <CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>

2016-10-07 21:22 GMT+02:00 Stuart Marks <stuart.marks at oracle.com>:
>
>
> On 10/7/16 11:23 AM, Paul Sandoz wrote:
>>>
>>>    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>>
>>
>> Optional is final so why do you need to express ?? extends Optional? ?
>
>
> The short answer is, it doesn't work if you don't have it. :-)
>
> The theoretical answer is that in this context, "? extends P<Q>" means "some
> subtype of P<Q>" and not necessarily just a subclass of P.
>
> (And even though Optional is final, it's not "permanently final" in that a
> hypothetical future version of the class library might change it to
> non-final, allowing subclasses.)
>
> This issue is covered in Angelika Langer's Generics FAQ entry, "What do
> multi-level (i.e., nested) wildcards mean?" [1] Note that the answer begins,
> "It depends." :-) I also note in passing that I've read this about five
> times and I'm still not quite sure I understand it entirely.
>
> For me, the best explanation comes from looking at examples. First, the
> history is that the signature in Java 8 is:
>
>   #1    flatMap(Function<? super T, Optional<U>> mapper)
>
> I believe R?mi originally proposed something like this (although it was
> about or(), the same issue applies to flatMap()):
>
>   #2    flatMap(Function<? super T, ? extends Optional<U>> mapper)
>
> The suggested fix that ended up in bug report was this:
>
>   #3    flatMap(Function<? super T, Optional<? extends U>> mapper)
>
> But this doesn't work for reasons I explain below. Finally, I'm proposing:
>
>   #4    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>
> In researching old email threads and chasing links, I ran across an example
> from Stefan Zobel [2] that fails with #3. He had an alternative that didn't
> seem quite right to me, so I ended up with #4.
>
> I've adapted Stefan's example as follows:
>
>         Optional<Integer> oi = Optional.empty();
>         Function<Number, Optional<StringBuilder>> fm = n ->
> Optional.empty();
>         Optional<CharSequence> ocs = oi.flatMap(fm);
>
> The flatmapper function 'fm' returns Optional<StringBuilder>. In the
> assignment on the last line, U is CharSequence, so we can compare this to
> the various signatures shown above with U filled in.
>
> Case #1 fails because Optional<StringBuilder> is incompatible with
> Optional<CharSequence>. This is the usual "generics are invariant thing".
> Even though SB is a subtype of CS, Optional<SB> isn't a subtype of
> Optional<CS>.
>
> Case #2 fails because adding a wildcard there doesn't help matters, since
> Optional<SB> is still unrelated to Optional<CS>.
>
> Now for the tricky part. :-)
>
> Surely case #3 should work, because adding an inner wildcard provides a
> subtyping relationship, so Optional<SB> is a subtype of Optional<? extends
> CS>.
>
> That much is true, but these are nested within the Function<> generic type,
> so the "generics are invariant" rule applies again. Thus,
>
>     Function<..., Optional<StringBuilder>>
>
> is not a subtype of
>
>     Function<..., Optional<? extends CharSequence>>
>
> To get around this, we have to add the outer wildcard as well, so that
>
>     Function<..., Optional<StringBuilder>>
>
> is a subtype of
>
>     Function<..., ? extends Optional<? extends CharSequence>>
>
> So that's what ended up in the signature.
>
> Similar analysis applies to the or() case.
>
> Now awaiting a message from R?mi telling me my explanation is incorrect in
> 3... 2... 1... :-) :-)
>
> s'marks
>
>
>
> [1]
> http://angelikalanger.com/GenericsFAQ/FAQSections/TypeArguments.html#What%20do%20multilevel%20wildcards%20mean?
>
> [2] https://sourceforge.net/p/streamsupport/tickets/125/#2d90
>



Hhm, that's really mind-boggling!

What's wrong with the alternative "additional bounded type parameter" approach?

Couldn't we also get by with something like

<V, U extends V> Optional<V> flatMap(Function<? super T, Optional<U>> mapper)


and


<S extends T> Optional<T> or(Supplier<Optional<S>> supplier)


Personally, I find that much easier to digest. But that's only me, of course.


Regards,
Stefan

From misterm at gmail.com  Fri Oct  7 19:42:07 2016
From: misterm at gmail.com (Michael Nascimento)
Date: Fri, 7 Oct 2016 16:42:07 -0300
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
Message-ID: <CAAWA2oXyiK4THYZh91JSRUxauca0NgZ=zPsxdnTRkdUwDtxtAw@mail.gmail.com>

Doesn't work, as Stuart has noted (s'marks, as far as I know your
explanation is 100% correct). Nested generics == pain :-(

Regards,
Michael

On Fri, Oct 7, 2016 at 4:40 PM, Stefan Zobel <spliterator at gmail.com> wrote:

> 2016-10-07 21:22 GMT+02:00 Stuart Marks <stuart.marks at oracle.com>:
> >
> >
> > On 10/7/16 11:23 AM, Paul Sandoz wrote:
> >>>
> >>>    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
> >>
> >>
> >> Optional is final so why do you need to express ?? extends Optional? ?
> >
> >
> > The short answer is, it doesn't work if you don't have it. :-)
> >
> > The theoretical answer is that in this context, "? extends P<Q>" means
> "some
> > subtype of P<Q>" and not necessarily just a subclass of P.
> >
> > (And even though Optional is final, it's not "permanently final" in that
> a
> > hypothetical future version of the class library might change it to
> > non-final, allowing subclasses.)
> >
> > This issue is covered in Angelika Langer's Generics FAQ entry, "What do
> > multi-level (i.e., nested) wildcards mean?" [1] Note that the answer
> begins,
> > "It depends." :-) I also note in passing that I've read this about five
> > times and I'm still not quite sure I understand it entirely.
> >
> > For me, the best explanation comes from looking at examples. First, the
> > history is that the signature in Java 8 is:
> >
> >   #1    flatMap(Function<? super T, Optional<U>> mapper)
> >
> > I believe R?mi originally proposed something like this (although it was
> > about or(), the same issue applies to flatMap()):
> >
> >   #2    flatMap(Function<? super T, ? extends Optional<U>> mapper)
> >
> > The suggested fix that ended up in bug report was this:
> >
> >   #3    flatMap(Function<? super T, Optional<? extends U>> mapper)
> >
> > But this doesn't work for reasons I explain below. Finally, I'm
> proposing:
> >
> >   #4    flatMap(Function<? super T, ? extends Optional<? extends U>>
> mapper)
> >
> > In researching old email threads and chasing links, I ran across an
> example
> > from Stefan Zobel [2] that fails with #3. He had an alternative that
> didn't
> > seem quite right to me, so I ended up with #4.
> >
> > I've adapted Stefan's example as follows:
> >
> >         Optional<Integer> oi = Optional.empty();
> >         Function<Number, Optional<StringBuilder>> fm = n ->
> > Optional.empty();
> >         Optional<CharSequence> ocs = oi.flatMap(fm);
> >
> > The flatmapper function 'fm' returns Optional<StringBuilder>. In the
> > assignment on the last line, U is CharSequence, so we can compare this to
> > the various signatures shown above with U filled in.
> >
> > Case #1 fails because Optional<StringBuilder> is incompatible with
> > Optional<CharSequence>. This is the usual "generics are invariant thing".
> > Even though SB is a subtype of CS, Optional<SB> isn't a subtype of
> > Optional<CS>.
> >
> > Case #2 fails because adding a wildcard there doesn't help matters, since
> > Optional<SB> is still unrelated to Optional<CS>.
> >
> > Now for the tricky part. :-)
> >
> > Surely case #3 should work, because adding an inner wildcard provides a
> > subtyping relationship, so Optional<SB> is a subtype of Optional<?
> extends
> > CS>.
> >
> > That much is true, but these are nested within the Function<> generic
> type,
> > so the "generics are invariant" rule applies again. Thus,
> >
> >     Function<..., Optional<StringBuilder>>
> >
> > is not a subtype of
> >
> >     Function<..., Optional<? extends CharSequence>>
> >
> > To get around this, we have to add the outer wildcard as well, so that
> >
> >     Function<..., Optional<StringBuilder>>
> >
> > is a subtype of
> >
> >     Function<..., ? extends Optional<? extends CharSequence>>
> >
> > So that's what ended up in the signature.
> >
> > Similar analysis applies to the or() case.
> >
> > Now awaiting a message from R?mi telling me my explanation is incorrect
> in
> > 3... 2... 1... :-) :-)
> >
> > s'marks
> >
> >
> >
> > [1]
> > http://angelikalanger.com/GenericsFAQ/FAQSections/
> TypeArguments.html#What%20do%20multilevel%20wildcards%20mean?
> >
> > [2] https://sourceforge.net/p/streamsupport/tickets/125/#2d90
> >
>
>
>
> Hhm, that's really mind-boggling!
>
> What's wrong with the alternative "additional bounded type parameter"
> approach?
>
> Couldn't we also get by with something like
>
> <V, U extends V> Optional<V> flatMap(Function<? super T, Optional<U>>
> mapper)
>
>
> and
>
>
> <S extends T> Optional<T> or(Supplier<Optional<S>> supplier)
>
>
> Personally, I find that much easier to digest. But that's only me, of
> course.
>
>
> Regards,
> Stefan
>

From paul.sandoz at oracle.com  Fri Oct  7 19:56:27 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 7 Oct 2016 12:56:27 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
Message-ID: <8E517395-C87C-4561-AA7D-E636FEA455D9@oracle.com>


> On 7 Oct 2016, at 12:22, Stuart Marks <stuart.marks at oracle.com> wrote:
> 
> 
> 
> On 10/7/16 11:23 AM, Paul Sandoz wrote:
>>>   flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>> 
>> Optional is final so why do you need to express ?? extends Optional? ?
> 
> The short answer is, it doesn't work if you don't have it. :-)
> 

I thought as much :-)


> The theoretical answer is that in this context, "? extends P<Q>" means "some subtype of P<Q>" and not necessarily just a subclass of P.
> 

Doh!, yes, that makes sense. Thanks for the detailed explanation. Fix looks good.

Paul.

From spliterator at gmail.com  Fri Oct  7 20:02:33 2016
From: spliterator at gmail.com (Stefan Zobel)
Date: Fri, 7 Oct 2016 22:02:33 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <CAAWA2oXyiK4THYZh91JSRUxauca0NgZ=zPsxdnTRkdUwDtxtAw@mail.gmail.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<CAAWA2oXyiK4THYZh91JSRUxauca0NgZ=zPsxdnTRkdUwDtxtAw@mail.gmail.com>
Message-ID: <CAPkkuSfDBaocLtX9MRW6YiSoyxo-4w7GADqTn7SdDPHmJz1MWA@mail.gmail.com>

2016-10-07 21:42 GMT+02:00 Michael Nascimento <misterm at gmail.com>:
> Doesn't work, as Stuart has noted (s'marks, as far as I know your
> explanation is 100% correct). Nested generics == pain :-(

Hi Michael,

sorry for being obtrusive. What exactly doesn't work?

Stuart's example

Optional<Integer> oi = Optional.empty();
Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
Optional<CharSequence> ocs = oi.flatMap(fm);
System.out.println(ocs.orElse("empty"));

work's for me (on 1.8.0_51). Sorry, I'm just trying to understand.

Kind regards,
Stefan


>
> Regards,
> Michael
>

From andrej.golovnin at gmail.com  Fri Oct  7 20:09:21 2016
From: andrej.golovnin at gmail.com (Andrej Golovnin)
Date: Fri, 7 Oct 2016 22:09:21 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
Message-ID: <0071033C-4185-4D8C-8852-262631C44AFF@gmail.com>

Hi Stuart,

> Webrev:
> 
> 	http://cr.openjdk.java.net/~smarks/reviews/8152617/webrev.0/


267     public<U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)

I think there should be a space between ?public? and ?<U>?.

Best regards,
Andrej Golovnin

From patrick at reini.net  Fri Oct  7 20:59:53 2016
From: patrick at reini.net (Patrick Reinhart)
Date: Fri, 7 Oct 2016 22:59:53 +0200
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<058f6ac5-21f4-00f3-76da-7378901000f...
Message-ID: <AD0DE602-CFC8-4507-8797-49D549095073@reini.net>

Here is the latest webrev:

http://cr.openjdk.java.net/~reinhapa/reviews/8134373/webrev.02

-Patrick



> Am 07.10.2016 um 12:00 schrieb Jonathan Bluett-Duncan <jbluettduncan at gmail.com>:
> 
> Hi all,
> 
> Sorry for the delayed response, I've been busy lately with university and other things.
> 
> Thank you all for your comments. I'll leave the DateTimeFormatter comment in, as you requested Stephen and Roger, and I'll work again with Patrick as soon as I'm ready.
> 
> Kind regards,
> Jonathan
> 
> On 6 October 2016 at 09:38, Stephen Colebourne <scolebourne at joda.org> wrote:
> On 6 October 2016 at 00:00, Stuart Marks <stuart.marks at oracle.com> wrote:
> >> I think you should perform no change to DateTimeFormatter (other than
> >> a comment) as part of this changeset. The behaviour of that
> >> DateTimeFormatter method is subtle, and I now suspect that what we
> >> have there might be the best option.
> >
> > I had recommended removing the comment from DateTimeFormatter, but if
> > Stephen wants the comment in, that's fine with me. In fact I'll defer to
> > what Stephen (and Roger Riggs) want with this code, since they're the
> > maintainers.
> 
> I think it makes sense to leave the new comment in. All further change
> should be under 8167222.
> 
> Stephen
> 


From stuart.marks at oracle.com  Fri Oct  7 21:10:28 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 7 Oct 2016 14:10:28 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <0071033C-4185-4D8C-8852-262631C44AFF@gmail.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<0071033C-4185-4D8C-8852-262631C44AFF@gmail.com>
Message-ID: <47e7f24a-e0c5-3f06-b10c-8318b3fbc900@oracle.com>



On 10/7/16 1:09 PM, Andrej Golovnin wrote:
> 267     public<U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>
> I think there should be a space between ?public? and ?<U>?.

Sure, I'll add this. There's also a space missing at a similar spot at the map() 
declaration; I'll fix that too.

thanks

s'marks


From stuart.marks at oracle.com  Fri Oct  7 21:38:57 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 7 Oct 2016 14:38:57 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
Message-ID: <aa611977-969c-f0d4-2644-10417c12db57@oracle.com>

On 10/7/16 12:40 PM, Stefan Zobel wrote:
> What's wrong with the alternative "additional bounded type parameter" approach?
>
> Couldn't we also get by with something like
>
> <V, U extends V> Optional<V> flatMap(Function<? super T, Optional<U>> mapper)
>
> and
>
> <S extends T> Optional<T> or(Supplier<Optional<S>> supplier)
>
> Personally, I find that much easier to digest. But that's only me, of course.

Yes, the additional type parameter seems to work for this example. I'm a bit 
surprised.

I think it's mostly a style thing. To me, a type parameter carries a certain 
amount of semantic weight, implying that it's a concern that the caller needs to 
pay attention to. (Note that type parameters are documented using the javadoc 
@param tag, and by convention every parameter must be documented.) By contrast, 
a wildcard says "anything that satisfies the constraints" but we don't care 
exactly what it is.

If a type parameter is used only once, it can (always, I think) be replaced by a 
wildcard. After having looked at lots of generic APIs, it seems to me that a 
style has emerged where wildcards are used whenever possible, and type 
parameters are used only when necessary, e.g. when the exact same type is 
required in multiple places. I don't know if this is actually written down 
anywhere though.

A difference *would* arise between the multiple-wildcards and 
additional-type-parameter approaches, if it were possible to have subclasses of 
Optional.
Of course, it doesn't apply to this case, since Optional is final, and we have 
no plans to make it non-final!

s'marks

From jonathan.gibbons at oracle.com  Fri Oct  7 21:40:11 2016
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Fri, 07 Oct 2016 14:40:11 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <250C708F-AF40-45C0-A075-5BBDFA41F0A9@oracle.com>
References: <57F43DC3.8040905@oracle.com> <57F43F43.3080707@oracle.com>
	<250C708F-AF40-45C0-A075-5BBDFA41F0A9@oracle.com>
Message-ID: <57F8163B.3030701@oracle.com>

Updated webrev with feedback from comments:

* use doPrivileged within ToolProvider.findFIrst  (includes adding new test)
* improve whitespace in doc comments

Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.05/

-- Jon



On 10/04/2016 09:19 PM, Mandy Chung wrote:
> This SPI is useful and provides as a replacement to existing use of internal APIs to launch some of our tools.  We will get jar, jmod, jlink and possibly other tools to convert to this SPI.
>
> ToolProvider::findFirst(String name) can find tool providers on classpath.  I think it needs to wrap the for-loop (specifically iterating on providers) with doPrivileged due to the stack-based permission check.
>
> Otherwise, looks good.
>
> Mandy
>
>> On Oct 4, 2016, at 4:46 PM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
>>
>> Resend with non-mostly-empty subject line!
>>
>> -- Jon
>>
>> On 10/04/2016 04:39 PM, Jonathan Gibbons wrote:
>>> Core-libs folk,
>>>
>>> Please review the following change to add a new service provider class
>>>     java.util.spi.ToolProvider
>>>
>>> which can be used provide simple "command-line" access to select JDK
>>> tools, without starting a new JVM.
>>>
>>> The following tools are updated to provide access through the new SPI:
>>>     javac, javadoc, javap, jdeps
>>>
>>> It is expected that additional tools will also be updated to provide access,
>>> but that will be done separately.
>>>
>>> Compiler-dev folk may wish to review the changes to the langtools repository.
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8159855
>>> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.03/
>>> API: http://cr.openjdk.java.net/~jjg/8159855/api.02/java/util/spi/ToolProvider.html
>>>
>>> -- Jon


From xueming.shen at oracle.com  Fri Oct  7 21:38:29 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Fri, 07 Oct 2016 14:38:29 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods) throws
	PatternSyntaxException
In-Reply-To: <62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>
References: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
	<62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>
Message-ID: <57F815D5.4030901@oracle.com>

thanks! updated, with biconsumer as well.

http://cr.openjdk.java.net/~sherman/8166261/webrev/


On 10/05/2016 10:25 AM, Naoto Sato wrote:
> Looks good to me.
>
> The test case could use IntStream.rangeClosed(Character.MIN_RADIX, Character.MAX_RADIX) for the good radixes, instead of hard coding ints.
>
> Naoto
>
> On 10/5/16 8:53 AM, Xueming Shen wrote:
>> Hi
>>
>> Please help review
>>
>> issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
>> webre: http://cr.openjdk.java.net/~sherman/8166261/webrev
>>
>> The radix sanity check are missing  from
>> hasNextByte/Short/Int/Long/BigInteger().
>> The only method we are doing now is useRadix(). The proposed change here
>> is to
>> add the check into all above methods that take a radix.
>>
>> Arguably it's an incompatible api change, but I don't expect it really
>> breaks anyone's
>> code.  Need go through ccc if approved.
>>
>> Thanks,
>> Sherman
>>


From mandy.chung at oracle.com  Fri Oct  7 22:00:39 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 7 Oct 2016 15:00:39 -0700
Subject: RFR: 8159855: Create an SPI for tools
In-Reply-To: <57F8163B.3030701@oracle.com>
References: <57F43DC3.8040905@oracle.com> <57F43F43.3080707@oracle.com>
	<250C708F-AF40-45C0-A075-5BBDFA41F0A9@oracle.com>
	<57F8163B.3030701@oracle.com>
Message-ID: <3DDACF4B-5E40-4D84-A0AB-7CDF186BC054@oracle.com>


> On Oct 7, 2016, at 2:40 PM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> Updated webrev with feedback from comments:
> 
> * use doPrivileged within ToolProvider.findFIrst  (includes adding new test)
> * improve whitespace in doc comments
> 
> Webrev: http://cr.openjdk.java.net/~jjg/8159855/webrev.05/

Looks good.  Thanks for adding the new test.

Nit: line 68: s/nonzero/non-zero

Mandy


From spliterator at gmail.com  Fri Oct  7 22:12:23 2016
From: spliterator at gmail.com (Stefan Zobel)
Date: Sat, 8 Oct 2016 00:12:23 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
Message-ID: <CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>

Hi Stuart,

thanks for explaining.

> ... After having looked at lots of generic APIs, it seems to me that a
> style has emerged where wildcards are used whenever possible, and type
> parameters are used only when necessary, ...

Yes, I'm familiar with that kind of reasoning (I think Josh Bloch stated
that principle in "Effective Java"). But, to be honest, I never thought
that it should apply as a strict rule in all circumstances.

Rhetorical question: do you really think that a signature employing 3
wildcards is easier to understand for the proverbial "average Joe" than
a bounded type parameter that expresses the sub-type relationship clearly?
I do not.

But anyway, you probably have to follow the established "style".

It's just too bad that most Java programmers I know won't understand the
proposed signature of 'flatMap'.

Kind regards,
Stefan

From forax at univ-mlv.fr  Fri Oct  7 22:35:26 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 8 Oct 2016 00:35:26 +0200 (CEST)
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
Message-ID: <2132515662.2552960.1475879726754.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Stuart Marks" <stuart.marks at oracle.com>
> ?: "Paul Sandoz" <paul.sandoz at oracle.com>
> Cc: "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoy?: Vendredi 7 Octobre 2016 21:22:09
> Objet: Re: RFR(s): 8152617 add missing wildcards to Optional or() and	flatMap()

> On 10/7/16 11:23 AM, Paul Sandoz wrote:
>>>    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
>>
>> Optional is final so why do you need to express ?? extends Optional? ?
> 
> The short answer is, it doesn't work if you don't have it. :-)
> 
> The theoretical answer is that in this context, "? extends P<Q>" means "some
> subtype of P<Q>" and not necessarily just a subclass of P.
> 
> (And even though Optional is final, it's not "permanently final" in that a
> hypothetical future version of the class library might change it to non-final,
> allowing subclasses.)
> 
> This issue is covered in Angelika Langer's Generics FAQ entry, "What do
> multi-level (i.e., nested) wildcards mean?" [1] Note that the answer begins, "It
> depends." :-) I also note in passing that I've read this about five times and
> I'm still not quite sure I understand it entirely.
> 
> For me, the best explanation comes from looking at examples. First, the history
> is that the signature in Java 8 is:
> 
>   #1    flatMap(Function<? super T, Optional<U>> mapper)
> 
> I believe R?mi originally proposed something like this (although it was about
> or(), the same issue applies to flatMap()):
> 
>   #2    flatMap(Function<? super T, ? extends Optional<U>> mapper)
> 
> The suggested fix that ended up in bug report was this:
> 
>   #3    flatMap(Function<? super T, Optional<? extends U>> mapper)
> 
> But this doesn't work for reasons I explain below. Finally, I'm proposing:
> 
>   #4    flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
> 
> In researching old email threads and chasing links, I ran across an example from
> Stefan Zobel [2] that fails with #3. He had an alternative that didn't seem
> quite right to me, so I ended up with #4.
> 
> I've adapted Stefan's example as follows:
> 
>	Optional<Integer> oi = Optional.empty();
>	Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
>	Optional<CharSequence> ocs = oi.flatMap(fm);
> 
> The flatmapper function 'fm' returns Optional<StringBuilder>. In the assignment
> on the last line, U is CharSequence, so we can compare this to the various
> signatures shown above with U filled in.
> 
> Case #1 fails because Optional<StringBuilder> is incompatible with
> Optional<CharSequence>. This is the usual "generics are invariant thing". Even
> though SB is a subtype of CS, Optional<SB> isn't a subtype of Optional<CS>.
> 
> Case #2 fails because adding a wildcard there doesn't help matters, since
> Optional<SB> is still unrelated to Optional<CS>.
> 
> Now for the tricky part. :-)
> 
> Surely case #3 should work, because adding an inner wildcard provides a
> subtyping relationship, so Optional<SB> is a subtype of Optional<? extends CS>.
> 
> That much is true, but these are nested within the Function<> generic type, so
> the "generics are invariant" rule applies again. Thus,
> 
>     Function<..., Optional<StringBuilder>>
> 
> is not a subtype of
> 
>     Function<..., Optional<? extends CharSequence>>
> 
> To get around this, we have to add the outer wildcard as well, so that
> 
>     Function<..., Optional<StringBuilder>>
> 
> is a subtype of
> 
>     Function<..., ? extends Optional<? extends CharSequence>>
> 
> So that's what ended up in the signature.
> 
> Similar analysis applies to the or() case.
> 
> Now awaiting a message from R?mi telling me my explanation is incorrect in 3...
> 2... 1... :-) :-)

no i think you are right :)

here is an example that shows that the signature of Optional.flatmap is not correct
  public static <T> Optional<T> first(List<T> list) {
    return list.stream().findFirst();
  }
  
  public static void bar(List<? extends CharSequence> list) {
    Function<String, Optional<? extends CharSequence>> fun = s -> first(list);
    Optional.of("foo").flatMap(fun );
  }
  
  public static void main(String[] args) {
    bar(List.of("hello"));
    bar(List.of(new StringBuilder("hello")));
  }

while Optional is final, you can still create an Optional<? extends CharSequence> when doing a capture, i.e. when you see a wildcard type as a parameterized type.
here, first() is called with a List<? extends CharSequence> so T = ? extends CharSequence so the return type of first() is an Optional<? extends CharSequence>.

> 
> s'marks

R?mi

> 
> 
> 
> [1]
> http://angelikalanger.com/GenericsFAQ/FAQSections/TypeArguments.html#What%20do%20multilevel%20wildcards%20mean?
> 
> [2] https://sourceforge.net/p/streamsupport/tickets/125/#2d90

From naoto.sato at oracle.com  Fri Oct  7 22:52:27 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Fri, 7 Oct 2016 15:52:27 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods)
	throws PatternSyntaxException
In-Reply-To: <57F815D5.4030901@oracle.com>
References: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
	<62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>
	<57F815D5.4030901@oracle.com>
Message-ID: <c4d6872f-45a8-50a1-0ec4-ea00e4c1d2bf@oracle.com>

The test case now looks good. Using biconsumer makes it more readable.

I see the additional paragraph with regard to IllegalArgumentException 
in each method description, however, it contradicts the @throws clause. 
Need to add @throws IllegalArgumentException for each method?

Naoto

On 10/7/16 2:38 PM, Xueming Shen wrote:
> thanks! updated, with biconsumer as well.
>
> http://cr.openjdk.java.net/~sherman/8166261/webrev/
>
>
> On 10/05/2016 10:25 AM, Naoto Sato wrote:
>> Looks good to me.
>>
>> The test case could use IntStream.rangeClosed(Character.MIN_RADIX,
>> Character.MAX_RADIX) for the good radixes, instead of hard coding ints.
>>
>> Naoto
>>
>> On 10/5/16 8:53 AM, Xueming Shen wrote:
>>> Hi
>>>
>>> Please help review
>>>
>>> issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
>>> webre: http://cr.openjdk.java.net/~sherman/8166261/webrev
>>>
>>> The radix sanity check are missing  from
>>> hasNextByte/Short/Int/Long/BigInteger().
>>> The only method we are doing now is useRadix(). The proposed change here
>>> is to
>>> add the check into all above methods that take a radix.
>>>
>>> Arguably it's an incompatible api change, but I don't expect it really
>>> breaks anyone's
>>> code.  Need go through ccc if approved.
>>>
>>> Thanks,
>>> Sherman
>>>
>

From xueming.shen at oracle.com  Fri Oct  7 23:08:24 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Fri, 07 Oct 2016 16:08:24 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods) throws
	PatternSyntaxException
In-Reply-To: <c4d6872f-45a8-50a1-0ec4-ea00e4c1d2bf@oracle.com>
References: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
	<62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>
	<57F815D5.4030901@oracle.com>
	<c4d6872f-45a8-50a1-0ec4-ea00e4c1d2bf@oracle.com>
Message-ID: <57F82AE8.4010408@oracle.com>

On 10/07/2016 03:52 PM, Naoto Sato wrote:
> The test case now looks good. Using biconsumer makes it more readable.
>
> I see the additional paragraph with regard to IllegalArgumentException in each method description, however, it contradicts the @throws clause. Need to add @throws IllegalArgumentException for each method?
>

thanks. updated. somehow I messed up with two versions there.


> Naoto
>
> On 10/7/16 2:38 PM, Xueming Shen wrote:
>> thanks! updated, with biconsumer as well.
>>
>> http://cr.openjdk.java.net/~sherman/8166261/webrev/
>>
>>
>> On 10/05/2016 10:25 AM, Naoto Sato wrote:
>>> Looks good to me.
>>>
>>> The test case could use IntStream.rangeClosed(Character.MIN_RADIX,
>>> Character.MAX_RADIX) for the good radixes, instead of hard coding ints.
>>>
>>> Naoto
>>>
>>> On 10/5/16 8:53 AM, Xueming Shen wrote:
>>>> Hi
>>>>
>>>> Please help review
>>>>
>>>> issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
>>>> webre: http://cr.openjdk.java.net/~sherman/8166261/webrev
>>>>
>>>> The radix sanity check are missing  from
>>>> hasNextByte/Short/Int/Long/BigInteger().
>>>> The only method we are doing now is useRadix(). The proposed change here
>>>> is to
>>>> add the check into all above methods that take a radix.
>>>>
>>>> Arguably it's an incompatible api change, but I don't expect it really
>>>> breaks anyone's
>>>> code.  Need go through ccc if approved.
>>>>
>>>> Thanks,
>>>> Sherman
>>>>
>>


From naoto.sato at oracle.com  Fri Oct  7 23:18:15 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Fri, 7 Oct 2016 16:18:15 -0700
Subject: RFR JDK-8166261: Scanner.nextInt(int) (and similar methods)
	throws PatternSyntaxException
In-Reply-To: <57F82AE8.4010408@oracle.com>
References: <461bf114-b1bf-64a9-e6a8-463125f03a45@oracle.com>
	<62cb46b6-b77a-07ab-8053-50a7b1252205@oracle.com>
	<57F815D5.4030901@oracle.com>
	<c4d6872f-45a8-50a1-0ec4-ea00e4c1d2bf@oracle.com>
	<57F82AE8.4010408@oracle.com>
Message-ID: <199eb3ae-dd71-7eec-2451-fb9208a85125@oracle.com>

+1

Naoto

On 10/7/16 4:08 PM, Xueming Shen wrote:
> On 10/07/2016 03:52 PM, Naoto Sato wrote:
>> The test case now looks good. Using biconsumer makes it more readable.
>>
>> I see the additional paragraph with regard to IllegalArgumentException
>> in each method description, however, it contradicts the @throws
>> clause. Need to add @throws IllegalArgumentException for each method?
>>
>
> thanks. updated. somehow I messed up with two versions there.
>
>
>> Naoto
>>
>> On 10/7/16 2:38 PM, Xueming Shen wrote:
>>> thanks! updated, with biconsumer as well.
>>>
>>> http://cr.openjdk.java.net/~sherman/8166261/webrev/
>>>
>>>
>>> On 10/05/2016 10:25 AM, Naoto Sato wrote:
>>>> Looks good to me.
>>>>
>>>> The test case could use IntStream.rangeClosed(Character.MIN_RADIX,
>>>> Character.MAX_RADIX) for the good radixes, instead of hard coding ints.
>>>>
>>>> Naoto
>>>>
>>>> On 10/5/16 8:53 AM, Xueming Shen wrote:
>>>>> Hi
>>>>>
>>>>> Please help review
>>>>>
>>>>> issue:  https://bugs.openjdk.java.net/browse/JDK-8166261
>>>>> webre: http://cr.openjdk.java.net/~sherman/8166261/webrev
>>>>>
>>>>> The radix sanity check are missing  from
>>>>> hasNextByte/Short/Int/Long/BigInteger().
>>>>> The only method we are doing now is useRadix(). The proposed change
>>>>> here
>>>>> is to
>>>>> add the check into all above methods that take a radix.
>>>>>
>>>>> Arguably it's an incompatible api change, but I don't expect it really
>>>>> breaks anyone's
>>>>> code.  Need go through ccc if approved.
>>>>>
>>>>> Thanks,
>>>>> Sherman
>>>>>
>>>
>

From stuart.marks at oracle.com  Sat Oct  8 00:28:27 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 7 Oct 2016 17:28:27 -0700
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
	<CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
Message-ID: <d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>



On 10/7/16 3:12 PM, Stefan Zobel wrote:
>> ... After having looked at lots of generic APIs, it seems to me that a
>> style has emerged where wildcards are used whenever possible, and type
>> parameters are used only when necessary, ...
>
> Yes, I'm familiar with that kind of reasoning (I think Josh Bloch stated
> that principle in "Effective Java"). But, to be honest, I never thought
> that it should apply as a strict rule in all circumstances.

Yep, it's in Effective Java, near the end of Item 28:

     ?As a rule, if a type parameter appears only once in a method
     declaration, replace it with a wildcard.?

> Rhetorical question: do you really think that a signature employing 3
> wildcards is easier to understand for the proverbial "average Joe" than
> a bounded type parameter that expresses the sub-type relationship clearly?
> I do not.
>
> But anyway, you probably have to follow the established "style".
>
> It's just too bad that most Java programmers I know won't understand the
> proposed signature of 'flatMap'.

Turns out that R?mi's example exposes the difference between the wildcard 
approach and the type-parameter approach. Returning to the example,

     Optional<Integer> oi = Optional.empty();
     Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
     Optional<CharSequence> ocs = oi.flatMap(fm);

If the flatMapper function itself has a wildcard type, for example,

     Function<Number, Optional<? extends CharSequence>> fm = n -> Optional.empty();

then this will still work with the wildcard approach but fail with the 
type-parameter approach. As R?mi also pointed out, a wildcarded type can result 
from the capture of a type with a wildcarded type parameter.

Based on this, I believe the nested wildcard approach to be the correct one.

s'marks


From spliterator at gmail.com  Sat Oct  8 09:04:59 2016
From: spliterator at gmail.com (Stefan Zobel)
Date: Sat, 8 Oct 2016 11:04:59 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or() and
	flatMap()
In-Reply-To: <d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
	<CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
	<d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
Message-ID: <CAPkkuSegz08HuO197JKVxf_FcijEFyr46xW2OQNOd10phrLUNA@mail.gmail.com>

>
> Turns out that R?mi's example exposes the difference between the wildcard
> approach and the type-parameter approach. Returning to the example,
>
>     Optional<Integer> oi = Optional.empty();
>     Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
>     Optional<CharSequence> ocs = oi.flatMap(fm);
>
> If the flatMapper function itself has a wildcard type, for example,
>
>     Function<Number, Optional<? extends CharSequence>> fm = n ->
> Optional.empty();
>
> then this will still work with the wildcard approach but fail with the
> type-parameter approach. As R?mi also pointed out, a wildcarded type can
> result from the capture of a type with a wildcarded type parameter.
>
> Based on this, I believe the nested wildcard approach to be the correct one.
>
> s'marks
>

Yes, this argument is unchallengeable.
I'm convinced now - though not happy.

Kind regards,
Stefan

From clement at unportant.info  Sun Oct  9 15:24:25 2016
From: clement at unportant.info (=?ISO-8859-1?Q?Cl=E9ment?= MATHIEU)
Date: Sun, 09 Oct 2016 17:24:25 +0200
Subject: DateTimeFormatter.format() uses exceptions for flow control
Message-ID: <1476026665.8959.9.camel@unportant.info>

Hi !?

I noticed that DateTimePrintContext.getValue() relies on exceptions to
handle optionality. Using exceptions for flow control seems both
unexpected and very costly, ie. I discovered the issue
when??LocaleDate.format(BASIC_ISO_DATE) showed up as a heavy hitter in
my application.?

Formatting a LocateDate as a "yyyyMMdd" string should take ~0.1?s but
currently takes from ~2.5?s, stack depth = 0, to ~10?s, stack depth =
128 when an optional parser is defined but the optional field is not
supported. This seems unfortunate when exceptions can easily be avoided
by testing if the field is supported before trying to get its value.?

Webrev:

? http://cdn.unportant.info/openjdk/dtf_exceptions/webrev.00/

JMH benchmarks:

? https://gist.github.com/cykl/020e1527546a1ba44b4bb3af6dc0484c


What do you think ?


Note: Many tests within java.time are currently broken because of
TestNG upgrade, see https://bugs.openjdk.java.net/browse/JDK-8152944.
Would it help if I spend some time adding the missing L suffixes ???

Regards,

Cl?ment MATHIEU

From david.holmes at oracle.com  Sun Oct  9 20:47:39 2016
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 10 Oct 2016 06:47:39 +1000
Subject: DateTimeFormatter.format() uses exceptions for flow control
In-Reply-To: <1476026665.8959.9.camel@unportant.info>
References: <1476026665.8959.9.camel@unportant.info>
Message-ID: <49e6b2e4-940a-3f76-b88d-51e845e5ec6d@oracle.com>

Hi Clement,

Please note that patches can only be accepted if they are sent through, 
or hosted upon OpenJDK infrastructure. If the patch is small enough can 
you send it inline in the email (attachments are often stripped) 
otherwise you will need to find an OpenJDK Author who can host the patch 
for you on cr.openjdk.java.net

Thanks,
David

On 10/10/2016 1:24 AM, Cl?ment MATHIEU wrote:
> Hi !
>
> I noticed that DateTimePrintContext.getValue() relies on exceptions to
> handle optionality. Using exceptions for flow control seems both
> unexpected and very costly, ie. I discovered the issue
> when  LocaleDate.format(BASIC_ISO_DATE) showed up as a heavy hitter in
> my application.
>
> Formatting a LocateDate as a "yyyyMMdd" string should take ~0.1?s but
> currently takes from ~2.5?s, stack depth = 0, to ~10?s, stack depth =
> 128 when an optional parser is defined but the optional field is not
> supported. This seems unfortunate when exceptions can easily be avoided
> by testing if the field is supported before trying to get its value.
>
> Webrev:
>
>   http://cdn.unportant.info/openjdk/dtf_exceptions/webrev.00/
>
> JMH benchmarks:
>
>   https://gist.github.com/cykl/020e1527546a1ba44b4bb3af6dc0484c
>
>
> What do you think ?
>
>
> Note: Many tests within java.time are currently broken because of
> TestNG upgrade, see https://bugs.openjdk.java.net/browse/JDK-8152944.
> Would it help if I spend some time adding the missing L suffixes ?
>
> Regards,
>
> Cl?ment MATHIEU
>

From scolebourne at joda.org  Sun Oct  9 20:52:38 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Sun, 9 Oct 2016 21:52:38 +0100
Subject: DateTimeFormatter.format() uses exceptions for flow control
In-Reply-To: <1476026665.8959.9.camel@unportant.info>
References: <1476026665.8959.9.camel@unportant.info>
Message-ID: <CACzrW9BWF+qkrTtAjYhzDx683JLP7-s0n=-p=uzEYWNrgObo+Q@mail.gmail.com>

This looks like it should be a worthwhile improvement.
Stephen

On 9 October 2016 at 16:24, Cl?ment MATHIEU <clement at unportant.info> wrote:
> Hi !
>
> I noticed that DateTimePrintContext.getValue() relies on exceptions to
> handle optionality. Using exceptions for flow control seems both
> unexpected and very costly, ie. I discovered the issue
> when  LocaleDate.format(BASIC_ISO_DATE) showed up as a heavy hitter in
> my application.
>
> Formatting a LocateDate as a "yyyyMMdd" string should take ~0.1?s but
> currently takes from ~2.5?s, stack depth = 0, to ~10?s, stack depth =
> 128 when an optional parser is defined but the optional field is not
> supported. This seems unfortunate when exceptions can easily be avoided
> by testing if the field is supported before trying to get its value.
>
> Webrev:
>
>   http://cdn.unportant.info/openjdk/dtf_exceptions/webrev.00/
>
> JMH benchmarks:
>
>   https://gist.github.com/cykl/020e1527546a1ba44b4bb3af6dc0484c
>
>
> What do you think ?
>
>
> Note: Many tests within java.time are currently broken because of
> TestNG upgrade, see https://bugs.openjdk.java.net/browse/JDK-8152944.
> Would it help if I spend some time adding the missing L suffixes ?
>
> Regards,
>
> Cl?ment MATHIEU

From clement at unportant.info  Mon Oct 10 06:53:50 2016
From: clement at unportant.info (=?ISO-8859-1?Q?Cl=E9ment?= MATHIEU)
Date: Mon, 10 Oct 2016 08:53:50 +0200
Subject: DateTimeFormatter.format() uses exceptions for flow control
In-Reply-To: <49e6b2e4-940a-3f76-b88d-51e845e5ec6d@oracle.com>
References: <1476026665.8959.9.camel@unportant.info>
	<49e6b2e4-940a-3f76-b88d-51e845e5ec6d@oracle.com>
Message-ID: <1476082430.5712.6.camel@unportant.info>

On Mon, 2016-10-10 at 06:47 +1000, David Holmes wrote:

Hi David,

> Please note that patches can only be accepted if they are sent
> through, or hosted upon OpenJDK infrastructure. If the patch is small
> enough can you send it inline in the email (attachments are often
> stripped)?

Here it is:?

--- old/src/java.base/share/classes/java/time/format/DateTimePrintContext.java	2016-10-09 17:01:30.326739656 +0200
+++ new/src/java.base/share/classes/java/time/format/DateTimePrintContext.java	2016-10-09 17:01:30.228738595 +0200
@@ -302,13 +302,10 @@
??????* @throws DateTimeException if the field is not available and the section is not optional
??????*/
?????Long getValue(TemporalField field) {
-????????try {
+????????if (optional == 0) {
?????????????return temporal.getLong(field);
-????????} catch (DateTimeException ex) {
-????????????if (optional > 0) {
-????????????????return null;
-????????????}
-????????????throw ex;
+????????} else {
+????????????return temporal.isSupported(field) ? temporal.getLong(field) : null;
?????????}
?????}

Cl?ment MATHIEU

From peter.levart at gmail.com  Mon Oct 10 08:04:26 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Mon, 10 Oct 2016 10:04:26 +0200
Subject: RFR: 8062389,8029459,8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
Message-ID: <458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>

Just a note that this is still ready to be reviewed.

I was also told that JCK tests pass with the patch applied.

Regards, Peter

On 10/04/2016 03:40 PM, Peter Levart wrote:
> Hi,
>
> I have a fix for conformance (P2) bug (8062389 
> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a 
> conformance (P3) bug (8029459 
> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance 
> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/
>
> As Daniel Smith writes in 8029459 
> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following 
> situation is as expected:
>
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends J {}
> K.class.getMethods() = [ J.m ]
>
> But the following has a different result although it should probably 
> have the same:
>
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends I, J {}
> K.class.getMethods() = [ I.m, J.m ]
>
> He then suggests the following algorithm:
>
> An implementation of getMethods consistent with JLS 8 would include 
> the following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
> 1) The class's/interface's declared (public) methods
> 2) The getMethods() of the superclass (if this is a class), minus any 
> that have a match in (1)
> 3) The getMethods() of each superinterface, minus any that have a 
> match in (1) or a _concrete_ match in (2) or a match from a 
> more-specific class/interface in (2) or (3)
>
> But even that is not sufficient for the following situation:
>
> interface E { void m(); }
> interface F extends E { void m(); }
> abstract class G implements E {}
> abstract class H extends G implements F {}
> H.class.getMethods() = [ E.m, F.m ]
>
> The desired result of H.class.getMethods() = [ F.m ]
>
> So a more precise definition is required which is implemented in the fix.
>
> The getMethods() implementation partitions the union of the following 
> methods:
>
> 1) The class's/interface's declared public methods
> 2) The getMethods() of the superclass (if this is a class)
> 3) The non-static getMethods() of each direct superinterface
>
> ...into equivalence classes (sets) of methods with same signature 
> (return type, name, parameter types). Within each such set, only the 
> "most specific" methods are kept and others are discarded. The union 
> of the kept methods is the result.
>
> The "more specific" is defined as a partial order within a set of 
> methods of same signature:
>
> Method A is more specific than method B if:
> - A is declared by a class and B is declared by an interface; or
> - A is declared by the same type as or a subtype of B's declaring type 
> and both are either declared by classes or both by interfaces (clearly 
> if A and B are declared by the same type and have the same signature, 
> they are the same method)
>
> If A and B are distinct elements from the set of methods with same 
> signature, then either:
> - A is more specific than B; or
> - B is more specific than A; or
> - A and B are incomparable
>
> A sub-set of "most specific" methods are the methods from the set 
> where for each such method M, there is no method N != M such that N is 
> "more specific" than M.
>
> There can be more than one "most specific" method for a particular 
> signature as they can be inherited from multiple unrelated interfaces, 
> but:
> - javac prevents compilation when among multiply inherited methods 
> with same signature there is at least one default method, so in 
> practice, getMethods() will only return multiple methods with same 
> signature if they are abstract interface methods. With one exception: 
> bridge methods that are created by javac for co-variant overrides are 
> default methods in interfaces. For example:
>
>     interface I { Object m(); }
>     interface J1 extends I { Map m(); }
>     interface J2 extends I { HashMap m(); }
>     interface K extends J1, J2 {}
>
> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, 
> default Object j2.m, abstract HashMap j2.m ]
>
> But this is an extreme case where one can still expect multiple 
> non-abstract methods with same signature, but different declaring 
> type, returned from getMethods().
>
> In order to also fix 8062389 
> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and 
> getMethods() share the same consolidation logic that results in a set 
> of "most specific" methods. The partitioning in getMethods() is 
> partially performed by collecting Methods into a HashMap where the 
> keys are (name, parameter types) tuples and values are linked lists of 
> Method objects with possibly varying return and declaring types. The 
> consolidation, i.e. keeping only the set of most specific methods as 
> new methods are encountered, is performed only among methods in the 
> list that share same return type and also removes duplicates. 
> getMethod() uses only one such list, consolidates methods that match 
> given name and parameter types and returns the 1st method from the 
> resulting list that has the most specific return type.
>
> That's it for algorithms used. As partitioning partially happens using 
> HashMap with (name, parameter types) keys, lists of methods that form 
> values are typically kept short with most of them containing only a 
> single method, so this fix also fixes performance issue 8061950 
> <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>
> The patch also contains some optimizations around redundant copying of 
> arrays and reflective objects.
>
> The FilterNotMostSpecific jtreg test has been updated to accommodate 
> for changed behavior. Both of the above extreme cases have been added 
> to the test.
>
> So, comments, please.
>
> Regards, Peter
>


From jbluettduncan at gmail.com  Mon Oct 10 08:34:32 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Mon, 10 Oct 2016 09:34:32 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
Message-ID: <CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>

Hi all,

Would you kindly review the latest webrev now?

http://cr.openjdk.java.net/~reinhapa/reviews/8134373/webrev.02

Thanks in advance.

Kind regards,
Jonathan

On 7 October 2016 at 21:59, Patrick Reinhart <patrick at reini.net> wrote:

> Here is the latest webrev:
>
> http://cr.openjdk.java.net/~reinhapa/reviews/8134373/webrev.02
>
> -Patrick
>
>
>
> > Am 07.10.2016 um 12:00 schrieb Jonathan Bluett-Duncan <
> jbluettduncan at gmail.com>:
> >
> > Hi all,
> >
> > Sorry for the delayed response, I've been busy lately with university
> and other things.
> >
> > Thank you all for your comments. I'll leave the DateTimeFormatter
> comment in, as you requested Stephen and Roger, and I'll work again with
> Patrick as soon as I'm ready.
> >
> > Kind regards,
> > Jonathan
> >
> > On 6 October 2016 at 09:38, Stephen Colebourne <scolebourne at joda.org>
> wrote:
> > On 6 October 2016 at 00:00, Stuart Marks <stuart.marks at oracle.com>
> wrote:
> > >> I think you should perform no change to DateTimeFormatter (other than
> > >> a comment) as part of this changeset. The behaviour of that
> > >> DateTimeFormatter method is subtle, and I now suspect that what we
> > >> have there might be the best option.
> > >
> > > I had recommended removing the comment from DateTimeFormatter, but if
> > > Stephen wants the comment in, that's fine with me. In fact I'll defer
> to
> > > what Stephen (and Roger Riggs) want with this code, since they're the
> > > maintainers.
> >
> > I think it makes sense to leave the new comment in. All further change
> > should be under 8167222.
> >
> > Stephen
> >
>
>

From peter.levart at gmail.com  Mon Oct 10 11:03:09 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Mon, 10 Oct 2016 13:03:09 +0200
Subject: using jshell in executable UNIX scripts
Message-ID: <a8f765ed-6968-473a-a6d7-fa4ae0305139@gmail.com>

Hi,

I don't know if this is the right list to discuss this, so please direct 
me to a more suitable place if there is one.

"jshell" command is a very nice interactive Java shell, but it could 
also be used for scripting. An executable script in any major UNIX OS is 
a textual file with executable permissions that starts with the 
following two characters: #!
The rest of the 1st line is the path to the interpreter executable and 
any arguments passed to it. The last argument passed to the interpreter 
is the path to the executable script. In case of jshell, one would want 
such script to be written like:

#!/home/peter/Apps64/jdk9/bin/jshell

System.out.println("Hello World!");

/exit


The problem is that jshell tries to parse the 1st line using jshell 
syntax and the result of running above executable script is:

| Error:
|  illegal character: '#'
| #!/home/peter/Apps64/jdk9/bin/jshell
|  ^
|  Error:
|  illegal start of expression
|  #!/home/peter/Apps64/jdk9/bin/jshell
|    ^
Hello World!


The script is actually executed, but the syntax error encountered in the 
1st line is printed too.

Would it be possible for jshell to skip 1st line if it starts with 
characters #! like other shells do?


Regards, Peter


From chris.hegarty at oracle.com  Mon Oct 10 13:43:09 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 10 Oct 2016 14:43:09 +0100
Subject: [9] RFR of JDK-8085192: java/rmi/activation/Activatable tests
	fail intermittently due to "Port already in use"
In-Reply-To: <2e13f9fa-cfeb-94bb-0ed5-6319c32b3e1f@Oracle.com>
References: <3c0bdf63-f55c-5b93-6d74-58e26916cb3b@oracle.com>
	<f881b76f-2a76-5abd-3118-7f6fe12d9205@Oracle.com>
	<e3b084d3-de5e-700f-2f69-22f91a0f80bf@oracle.com>
	<02147682-541f-9746-b6da-2cc2800165b5@Oracle.com>
	<3E39A1E9-0764-404D-9824-85CFB75113FF@oracle.com>
	<E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
	<02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>
	<2e13f9fa-cfeb-94bb-0ed5-6319c32b3e1f@Oracle.com>
Message-ID: <745be6ee-810a-4d2e-76b9-c0df79e6d6db@oracle.com>

Roger,

I addressed all, or most, of your comments in the following
webrev.

  1) Refactored out the use of sun.nio.ch in the test library,
    so that a reduced number of tests need their @modules tag
    updated. ( @modules support with test library usage it a
    pain )

  2) Use Boolean.getBoolean to retrieve the new property

  3) fix typos and remove stray debugging statements

   http://cr.openjdk.java.net/~chegar/8085192_webrev.03/

> ...
> I'm vaguely not very comfortable with scraping the port number off stdout
> and the inherited channel pieces seem like a lot of moving parts.

Right, I was a little uneasy with this too, to being with, but
it has grown on me ( since it appears stable and reliable in
all my builds and tests ). Also the surface area of the code
change is very small, and the inherit channel mechanism is well
specified and stable.

> Roger
>
> p.s.  Anyother idea
> I assume not all platforms can allow separate processes to open server
> sockets to the same port.
> If so, we would just have the client allocate a port (0), mark it
> non-exclusive and keep it open
> while passing the port number to RMID. Only after RMID is started close
> the allocating socket.

I believe Stuart did look at this in some detail a while back [1], and
it was somewhat dismissed because of the lack of cross platform support
for SO_REUSEPORT. Maybe things have move on, but I don't think so?

The use of inherit channel is somewhat akin to loading an agent into
the target, but more straightforward.

What do others think, that will have to maintain these tests? I don't
want to make them every harder to maintain. Hamlin's approach is still
on the table too.

-Chris.

[1] 
http://mail.openjdk.java.net/pipermail/serviceability-dev/2014-December/016251.html

From sergei.kovalev at oracle.com  Mon Oct 10 13:55:41 2016
From: sergei.kovalev at oracle.com (Sergei Kovalev)
Date: Mon, 10 Oct 2016 16:55:41 +0300
Subject: RFR(s): 8167437: Fix module dependencies for tests that use internal
	API (java/lang)
Message-ID: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>

Hello team,

Could you please review fix for:

BugID: https://bugs.openjdk.java.net/browse/JDK-8167437
WebRev: http://cr.openjdk.java.net/~skovalev/8167437/webrev.00/

Issue: two tests from java/lang uses internal API but have no 
declaration in jtreg header.
Solution: add missed declaration.

-- 
With best regards,
Sergei


From Alan.Bateman at oracle.com  Mon Oct 10 14:51:40 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 10 Oct 2016 15:51:40 +0100
Subject: RFR(s): 8167437: Fix module dependencies for tests that use
	internal API (java/lang)
In-Reply-To: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
References: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
Message-ID: <feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>

On 10/10/2016 14:55, Sergei Kovalev wrote:

> Hello team,
>
> Could you please review fix for:
>
> BugID: https://bugs.openjdk.java.net/browse/JDK-8167437
> WebRev: http://cr.openjdk.java.net/~skovalev/8167437/webrev.00/
>
> Issue: two tests from java/lang uses internal API but have no 
> declaration in jtreg header.
> Solution: add missed declaration.
>
For TestBadSignatures then what is being used in jdk.unsupported?

-Alan

From sergei.kovalev at oracle.com  Mon Oct 10 14:55:52 2016
From: sergei.kovalev at oracle.com (Sergei Kovalev)
Date: Mon, 10 Oct 2016 17:55:52 +0300
Subject: RFR(s): 8167437: Fix module dependencies for tests that use
	internal API (java/lang)
In-Reply-To: <feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>
References: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
	<feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>
Message-ID: <d78c2467-1904-a357-ee0c-2178dc7f7b49@oracle.com>

Hi Alan,

sun.reflect.*

./jdk9_latest/bin/jmod describe ./jdk9_latest/jmods/jdk.unsupported.jmod

jdk.unsupported at 9-ea
   requires mandated java.base
   exports com.sun.nio.file
   exports sun.misc
*  exports sun.reflect*
   operating-system-name Linux
   operating-system-architecture amd64
   operating-system-version 2.6


10.10.16 17:51, Alan Bateman wrote:
> On 10/10/2016 14:55, Sergei Kovalev wrote:
>
>> Hello team,
>>
>> Could you please review fix for:
>>
>> BugID: https://bugs.openjdk.java.net/browse/JDK-8167437
>> WebRev: http://cr.openjdk.java.net/~skovalev/8167437/webrev.00/
>>
>> Issue: two tests from java/lang uses internal API but have no 
>> declaration in jtreg header.
>> Solution: add missed declaration.
>>
> For TestBadSignatures then what is being used in jdk.unsupported?
>
> -Alan

-- 
With best regards,
Sergei


From Alan.Bateman at oracle.com  Mon Oct 10 15:05:16 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 10 Oct 2016 16:05:16 +0100
Subject: RFR(s): 8167437: Fix module dependencies for tests that use
	internal API (java/lang)
In-Reply-To: <d78c2467-1904-a357-ee0c-2178dc7f7b49@oracle.com>
References: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
	<feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>
	<d78c2467-1904-a357-ee0c-2178dc7f7b49@oracle.com>
Message-ID: <617c2b37-1e37-ba74-f1d9-71eb9ba760e6@oracle.com>

On 10/10/2016 15:55, Sergei Kovalev wrote:

> Hi Alan,
>
> sun.reflect.*
>
> ./jdk9_latest/bin/jmod describe ./jdk9_latest/jmods/jdk.unsupported.jmod
>
> jdk.unsupported at 9-ea
>   requires mandated java.base
>   exports com.sun.nio.file
>   exports sun.misc
> *  exports sun.reflect*
>   operating-system-name Linux
>   operating-system-architecture amd64
>   operating-system-version 2.6
sun.reflect.generics.parser is in java.base, I don't see anything using 
jdk.unsupported/sun.reflect. You can check this quickly by running the 
test without your change.

-Alan

From sergei.kovalev at oracle.com  Mon Oct 10 15:15:48 2016
From: sergei.kovalev at oracle.com (Sergei Kovalev)
Date: Mon, 10 Oct 2016 18:15:48 +0300
Subject: RFR(s): 8167437: Fix module dependencies for tests that use
	internal API (java/lang)
In-Reply-To: <617c2b37-1e37-ba74-f1d9-71eb9ba760e6@oracle.com>
References: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
	<feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>
	<d78c2467-1904-a357-ee0c-2178dc7f7b49@oracle.com>
	<617c2b37-1e37-ba74-f1d9-71eb9ba760e6@oracle.com>
Message-ID: <9cd9c074-773f-6a34-6bc1-4f215476edd9@oracle.com>

You are right. I'll rollback the change for the file.


10.10.16 18:05, Alan Bateman wrote:
> sun.reflect.generics.parser is in java.base, I don't see anything 
> using jdk.unsupported/sun.reflect. You can check this quickly by 
> running the test without your change.
>
> -Alan

-- 
With best regards,
Sergei


From martinrb at google.com  Mon Oct 10 16:49:45 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 10 Oct 2016 09:49:45 -0700
Subject: RFR: 8062389,8029459,8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
Message-ID: <CA+kOe08r=FqmqovXOnN-uKKx_Lh2M-6qAfTduOGO4o+oprCTGA@mail.gmail.com>

Does core reflection have an active maintainer?  Or is Peter that
maintainer?

On Mon, Oct 10, 2016 at 1:04 AM, Peter Levart <peter.levart at gmail.com>
wrote:

> Just a note that this is still ready to be reviewed.
>
> I was also told that JCK tests pass with the patch applied.
>
> Regards, Peter
>
>
> On 10/04/2016 03:40 PM, Peter Levart wrote:
>
>> Hi,
>>
>> I have a fix for conformance (P2) bug (8062389 <
>> https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a
>> conformance (P3) bug (8029459 <https://bugs.openjdk.java.net
>> /browse/JDK-8029459>) and a performance issue (8061950 <
>> https://bugs.openjdk.java.net/browse/JDK-8061950>):
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethod
>> s.new/webrev.04/
>>
>> As Daniel Smith writes in 8029459 <https://bugs.openjdk.java.net
>> /browse/JDK-8029459>, the following situation is as expected:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends J {}
>> K.class.getMethods() = [ J.m ]
>>
>> But the following has a different result although it should probably have
>> the same:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends I, J {}
>> K.class.getMethods() = [ I.m, J.m ]
>>
>> He then suggests the following algorithm:
>>
>> An implementation of getMethods consistent with JLS 8 would include the
>> following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
>> 1) The class's/interface's declared (public) methods
>> 2) The getMethods() of the superclass (if this is a class), minus any
>> that have a match in (1)
>> 3) The getMethods() of each superinterface, minus any that have a match
>> in (1) or a _concrete_ match in (2) or a match from a more-specific
>> class/interface in (2) or (3)
>>
>> But even that is not sufficient for the following situation:
>>
>> interface E { void m(); }
>> interface F extends E { void m(); }
>> abstract class G implements E {}
>> abstract class H extends G implements F {}
>> H.class.getMethods() = [ E.m, F.m ]
>>
>> The desired result of H.class.getMethods() = [ F.m ]
>>
>> So a more precise definition is required which is implemented in the fix.
>>
>> The getMethods() implementation partitions the union of the following
>> methods:
>>
>> 1) The class's/interface's declared public methods
>> 2) The getMethods() of the superclass (if this is a class)
>> 3) The non-static getMethods() of each direct superinterface
>>
>> ...into equivalence classes (sets) of methods with same signature (return
>> type, name, parameter types). Within each such set, only the "most
>> specific" methods are kept and others are discarded. The union of the kept
>> methods is the result.
>>
>> The "more specific" is defined as a partial order within a set of methods
>> of same signature:
>>
>> Method A is more specific than method B if:
>> - A is declared by a class and B is declared by an interface; or
>> - A is declared by the same type as or a subtype of B's declaring type
>> and both are either declared by classes or both by interfaces (clearly if A
>> and B are declared by the same type and have the same signature, they are
>> the same method)
>>
>> If A and B are distinct elements from the set of methods with same
>> signature, then either:
>> - A is more specific than B; or
>> - B is more specific than A; or
>> - A and B are incomparable
>>
>> A sub-set of "most specific" methods are the methods from the set where
>> for each such method M, there is no method N != M such that N is "more
>> specific" than M.
>>
>> There can be more than one "most specific" method for a particular
>> signature as they can be inherited from multiple unrelated interfaces, but:
>> - javac prevents compilation when among multiply inherited methods with
>> same signature there is at least one default method, so in practice,
>> getMethods() will only return multiple methods with same signature if they
>> are abstract interface methods. With one exception: bridge methods that are
>> created by javac for co-variant overrides are default methods in
>> interfaces. For example:
>>
>>     interface I { Object m(); }
>>     interface J1 extends I { Map m(); }
>>     interface J2 extends I { HashMap m(); }
>>     interface K extends J1, J2 {}
>>
>> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, default
>> Object j2.m, abstract HashMap j2.m ]
>>
>> But this is an extreme case where one can still expect multiple
>> non-abstract methods with same signature, but different declaring type,
>> returned from getMethods().
>>
>> In order to also fix 8062389 <https://bugs.openjdk.java.net
>> /browse/JDK-8062389>, getMethod() and getMethods() share the same
>> consolidation logic that results in a set of "most specific" methods. The
>> partitioning in getMethods() is partially performed by collecting Methods
>> into a HashMap where the keys are (name, parameter types) tuples and values
>> are linked lists of Method objects with possibly varying return and
>> declaring types. The consolidation, i.e. keeping only the set of most
>> specific methods as new methods are encountered, is performed only among
>> methods in the list that share same return type and also removes
>> duplicates. getMethod() uses only one such list, consolidates methods that
>> match given name and parameter types and returns the 1st method from the
>> resulting list that has the most specific return type.
>>
>> That's it for algorithms used. As partitioning partially happens using
>> HashMap with (name, parameter types) keys, lists of methods that form
>> values are typically kept short with most of them containing only a single
>> method, so this fix also fixes performance issue 8061950 <
>> https://bugs.openjdk.java.net/browse/JDK-8061950>.
>>
>> The patch also contains some optimizations around redundant copying of
>> arrays and reflective objects.
>>
>> The FilterNotMostSpecific jtreg test has been updated to accommodate for
>> changed behavior. Both of the above extreme cases have been added to the
>> test.
>>
>> So, comments, please.
>>
>> Regards, Peter
>>
>>
>

From stuart.marks at oracle.com  Mon Oct 10 18:57:02 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 10 Oct 2016 11:57:02 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
Message-ID: <0bcc77a0-af1f-7e59-3b7f-e6ac59961729@oracle.com>

OK, I'll sponsor this. I need to run this through our internal testing system 
before pushing it. I'll follow up here with results.

s'marks


On 10/10/16 1:34 AM, Jonathan Bluett-Duncan wrote:
> Hi all,
>
> Would you kindly review the latest webrev now?
>
> http://cr.openjdk.java.net/~reinhapa/reviews/8134373/webrev.02 
> <http://cr.openjdk.java.net/%7Ereinhapa/reviews/8134373/webrev.02>
>
> Thanks in advance.
>
> Kind regards,
> Jonathan
>
> On 7 October 2016 at 21:59, Patrick Reinhart <patrick at reini.net 
> <mailto:patrick at reini.net>> wrote:
>
>     Here is the latest webrev:
>
>     http://cr.openjdk.java.net/~reinhapa/reviews/8134373/webrev.02
>     <http://cr.openjdk.java.net/%7Ereinhapa/reviews/8134373/webrev.02>
>
>     -Patrick
>
>
>
>     > Am 07.10.2016 um 12:00 schrieb Jonathan Bluett-Duncan
>     <jbluettduncan at gmail.com <mailto:jbluettduncan at gmail.com>>:
>     >
>     > Hi all,
>     >
>     > Sorry for the delayed response, I've been busy lately with university
>     and other things.
>     >
>     > Thank you all for your comments. I'll leave the DateTimeFormatter
>     comment in, as you requested Stephen and Roger, and I'll work again with
>     Patrick as soon as I'm ready.
>     >
>     > Kind regards,
>     > Jonathan
>     >
>     > On 6 October 2016 at 09:38, Stephen Colebourne <scolebourne at joda.org
>     <mailto:scolebourne at joda.org>> wrote:
>     > On 6 October 2016 at 00:00, Stuart Marks <stuart.marks at oracle.com
>     <mailto:stuart.marks at oracle.com>> wrote:
>     > >> I think you should perform no change to DateTimeFormatter (other than
>     > >> a comment) as part of this changeset. The behaviour of that
>     > >> DateTimeFormatter method is subtle, and I now suspect that what we
>     > >> have there might be the best option.
>     > >
>     > > I had recommended removing the comment from DateTimeFormatter, but if
>     > > Stephen wants the comment in, that's fine with me. In fact I'll defer to
>     > > what Stephen (and Roger Riggs) want with this code, since they're the
>     > > maintainers.
>     >
>     > I think it makes sense to leave the new comment in. All further change
>     > should be under 8167222.
>     >
>     > Stephen
>     >
>
>


From brent.christian at oracle.com  Mon Oct 10 20:36:04 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Mon, 10 Oct 2016 13:36:04 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is parallel
	capable
Message-ID: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>

Hi,

Please review my fix for 8165793.  This follows the discussion here:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-September/009328.html

The proposal is to add a new public method on ClassLoader:

/**
  * Returns {@code true} if this class loader is
  * {@linkplain #registerAsParallelCapable parallel capable}, otherwise
  * {@code false}.
  *
  * @return  {@code true} if this class loader is parallel capable,
  *          otherwise {@code false}.
  * @since   9
  */
public final boolean isParallelCapable();

Bug:
https://bugs.openjdk.java.net/browse/JDK-8165793
Webrev:
http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/

Thanks!

-Brent


From mandy.chung at oracle.com  Mon Oct 10 21:30:34 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 10 Oct 2016 14:30:34 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
Message-ID: <48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>


> On Oct 10, 2016, at 1:36 PM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> Hi,
> 
> Please review my fix for 8165793.  This follows the discussion here:
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-September/009328.html
> 
> The proposal is to add a new public method on ClassLoader:
> 
> /**
> * Returns {@code true} if this class loader is
> * {@linkplain #registerAsParallelCapable parallel capable}, otherwise
> * {@code false}.
> *
> * @return  {@code true} if this class loader is parallel capable,
> *          otherwise {@code false}.
> * @since   9
> */
> public final boolean isParallelCapable();

The patch looks fine.  It would be good to add @see #registerAsParallelCapable in this new method.  Also the first ?parallel capable? occurrance in the class spec and the registerAsParallelCapable method spec to @linkplain #isParallelCapable.

Mandy

From steve.drach at oracle.com  Mon Oct 10 21:50:25 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Mon, 10 Oct 2016 14:50:25 -0700
Subject: RFR: 8166460: jdk/internal/util/jar/TestVersionedStream gets
	Assertion error
In-Reply-To: <1EC05ECD-D562-459E-BA86-E7D755B650B0@oracle.com>
References: <1EC05ECD-D562-459E-BA86-E7D755B650B0@oracle.com>
Message-ID: <27F98EEC-54A1-4B44-B5CD-AF29D8A11810@oracle.com>

As a consequence of some conversations about this test, I?ve removed the part that specifically depended on the order of the entries and now just verify that the order of the versioned entries is the same relative order as all the entries in the jar file.   The updated changeset is http://cr.openjdk.java.net/~sdrach/8166460/webrev.03/ <http://cr.openjdk.java.net/~sdrach/8166460/webrev.03/>

> On Oct 5, 2016, at 12:27 PM, Steve Drach <steve.drach at oracle.com> wrote:
> 
> Hi,
> 
> Please review the following changeset that creates a replacement for the TestVersionedStream test.  The previous test occasionally failed on Linux hotspot nightly testing due to jar tool sometimes changing the order of the entries.  The new test specifically sets the order of the entries and tests both possible orders.
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8166460 <https://bugs.openjdk.java.net/browse/JDK-8166460>
> webrev: http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/>
> 
> Thanks,
> Steve
> 


From paul.sandoz at oracle.com  Mon Oct 10 22:03:12 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 10 Oct 2016 15:03:12 -0700
Subject: RFR: 8166460: jdk/internal/util/jar/TestVersionedStream gets
	Assertion error
In-Reply-To: <27F98EEC-54A1-4B44-B5CD-AF29D8A11810@oracle.com>
References: <1EC05ECD-D562-459E-BA86-E7D755B650B0@oracle.com>
	<27F98EEC-54A1-4B44-B5CD-AF29D8A11810@oracle.com>
Message-ID: <45B83A51-31AD-4C8C-989F-A1A5E102BB00@oracle.com>


> On 10 Oct 2016, at 14:50, Steve Drach <steve.drach at oracle.com> wrote:
> 
> As a consequence of some conversations about this test, I?ve removed the part that specifically depended on the order of the entries and now just verify that the order of the versioned entries is the same relative order as all the entries in the jar file.   The updated changeset is http://cr.openjdk.java.net/~sdrach/8166460/webrev.03/ <http://cr.openjdk.java.net/~sdrach/8166460/webrev.03/>
> 

This looks better (crossing fingers JPRT passes).

Thanks,
Paul.

>> On Oct 5, 2016, at 12:27 PM, Steve Drach <steve.drach at oracle.com> wrote:
>> 
>> Hi,
>> 
>> Please review the following changeset that creates a replacement for the TestVersionedStream test.  The previous test occasionally failed on Linux hotspot nightly testing due to jar tool sometimes changing the order of the entries.  The new test specifically sets the order of the entries and tests both possible orders.
>> 
>> issue: https://bugs.openjdk.java.net/browse/JDK-8166460 <https://bugs.openjdk.java.net/browse/JDK-8166460>
>> webrev: http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8166460/webrev.00/>
>> 
>> Thanks,
>> Steve
>> 
> 


From brent.christian at oracle.com  Mon Oct 10 22:44:41 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Mon, 10 Oct 2016 15:44:41 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
Message-ID: <8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>

On 10/10/16 2:30 PM, Mandy Chung wrote:
>
> The patch looks fine.  It would be good to add @see
> #registerAsParallelCapable in this new method.  Also the first
> ?parallel capable? occurrance in the class spec and the
> registerAsParallelCapable method spec to @linkplain
> #isParallelCapable.

Thanks for having a look, and for the suggestions.  I also added an @see 
#isParallelCapable in registerAsParallelCapable().

Webrev updated in place:
http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/

-Brent

From mandy.chung at oracle.com  Mon Oct 10 22:51:39 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 10 Oct 2016 15:51:39 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
Message-ID: <5A6FC114-DA55-4AC8-B265-048092BD3530@oracle.com>


> On Oct 10, 2016, at 3:44 PM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> Thanks for having a look, and for the suggestions.  I also added an @see #isParallelCapable in registerAsParallelCapable().
> 
> Webrev updated in place:
> http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/
> 

+1

Do you mind fixing @return in the registerAsParallelCapable method with {@code true} amd {@code false} since you are in this method.  No need for a new webrev.

thanks
Mandy


From brent.christian at oracle.com  Mon Oct 10 23:12:06 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Mon, 10 Oct 2016 16:12:06 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <5A6FC114-DA55-4AC8-B265-048092BD3530@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
	<5A6FC114-DA55-4AC8-B265-048092BD3530@oracle.com>
Message-ID: <54a4059b-17c5-133c-2d67-590547991349@oracle.com>

On 10/10/16 3:51 PM, Mandy Chung wrote:
>
> Do you mind fixing @return in the registerAsParallelCapable method
> with {@code true} amd {@code false} since you are in this method.  No
> need for a new webrev.
>

Sure, no problem.

Thanks,
-Brent

From weijun.wang at oracle.com  Tue Oct 11 02:40:45 2016
From: weijun.wang at oracle.com (Weijun Wang)
Date: Tue, 11 Oct 2016 10:40:45 +0800
Subject: RFR: Release note for 8164705: Remove pathname canonicalization from
	FilePermission
In-Reply-To: <D37F53D2-6139-4A66-A7CB-4660F67E6345@oracle.com>
References: <c7f467cf-eb51-1fc6-1a03-939f43d0b861@oracle.com>
	<6484addc-45a2-a164-46b9-38de8e85a8ff@oracle.com>
	<e0a45d9e-6c54-f956-098e-d03f2b1e1c4a@oracle.com>
	<e4391932-d96f-a543-43ca-c20597b1db4e@oracle.com>
	<D37F53D2-6139-4A66-A7CB-4660F67E6345@oracle.com>
Message-ID: <402a1cac-6fbc-3152-66ef-c4ec42971012@oracle.com>

https://bugs.openjdk.java.net/browse/JDK-8165836

I added more words, please take another look.

Thanks
Max

On 9/13/2016 2:13, Brian Burkhalter wrote:
> Only picky comments: not sure but maybe change:
>
> 1) vice versa -> and vice versa
> 2) When it?s set to true -> When true
> 3) just like before -> as before
>
> Brian
>
> On Sep 12, 2016, at 12:23 AM, Weijun Wang <weijun.wang at oracle.com
> <mailto:weijun.wang at oracle.com>> wrote:
>
>> BTW, please also review the release note at
>>
>>   https://bugs.openjdk.java.net/browse/JDK-8165836
>>
>> Thanks
>> Max
>

From peter.levart at gmail.com  Tue Oct 11 06:03:45 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 11 Oct 2016 08:03:45 +0200
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
Message-ID: <81d7e7d5-527f-b4e4-4f2c-bcd0832185b1@gmail.com>

Hi Brent,


On 10/11/2016 12:44 AM, Brent Christian wrote:
> On 10/10/16 2:30 PM, Mandy Chung wrote:
>>
>> The patch looks fine.  It would be good to add @see
>> #registerAsParallelCapable in this new method.  Also the first
>> ?parallel capable? occurrance in the class spec and the
>> registerAsParallelCapable method spec to @linkplain
>> #isParallelCapable.
>
> Thanks for having a look, and for the suggestions.  I also added an 
> @see #isParallelCapable in registerAsParallelCapable().
>
> Webrev updated in place:
> http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/
>
> -Brent

It would be marginally more efficient to check for parallelLockMap != 
null instead of looking up the CL implementation class in the 
WeakHashMap within a synchronized block, but I guess the performance of 
this method is not very important, is it?

Regards, Peter


From andrej.golovnin at gmail.com  Tue Oct 11 06:26:20 2016
From: andrej.golovnin at gmail.com (Andrej Golovnin)
Date: Tue, 11 Oct 2016 08:26:20 +0200
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys85-DqhvwYaOmN+izBqPkZDGoa4Ju4wPpduPiM-3UB1msQ@mail.gmail.com>
	<FD16B50E-CC8A-4D49-94F2-80E16531B330@oracle.com>
	<26095be3-04df-3576-b41b-b7a199eaa854@oracle.com>
	<009bf30d-37ea-2362-fd9b-53a26d21fe03@oracle.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
Message-ID: <CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>

Hi Jonathan,

src/java.base/share/classes/java/util/ResourceBundle.java

2490     public static class Control {
2491         /**
2492          * The default format <code>List</code>, which contains the strings
2493          * <code>"java.class"</code> and <code>"java.properties"</code>, in
2494          * this order. This <code>List</code> is {@linkplain
2495          * Collections#unmodifiableList(List) unmodifiable}.
2496          *
2497          * @see #getFormats(String)
2498          */
2499         public static final List<String> FORMAT_DEFAULT =
List.of("java.class", "java.properties");

I think you should also change the JavaDocs in the
ResourceBundle.Control class for the constants FORMAT_CLASS,
FORMAT_DEFAULT and FORMAT_PROPERTIES, because the JavaDocs for this
constants explicitly mentions, that the lists are created by using
Collections#unmodifiableList(List). Or you cannot change this
constants at all because they are part of the Public API and existed
in this form for a long time. Please ask someone from Oracle for help.
They can explain it better when it is OK to change and when not. Maybe
Stuart can do that.

Best regards,
Andrej Golovnin

From Alan.Bateman at oracle.com  Tue Oct 11 13:13:27 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 11 Oct 2016 14:13:27 +0100
Subject: RFR: 8062389,8029459,8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
Message-ID: <233b9824-7c5b-a22f-cdf6-954f3b0eed9d@oracle.com>



On 10/10/2016 09:04, Peter Levart wrote:
> Just a note that this is still ready to be reviewed.
>
> I was also told that JCK tests pass with the patch applied.
I've taken a first pass over this and it looks quite good. I do intend 
to do a second pass and I hope others will help review this. One things 
we'll probably need to do is write a release note so that the behavioral 
change is documented, I'm trying to think if there are scenarios where 
the historical inconsistency could cause something to work that won't 
work now.

-Alan

From daniel.fuchs at oracle.com  Tue Oct 11 15:19:43 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Tue, 11 Oct 2016 16:19:43 +0100
Subject: RFR: 8062389,8029459,8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
Message-ID: <acd76378-0873-81ca-c8ce-cd490a845959@oracle.com>

Hi Peter,

I was looking at the test - I didn't see a case where the
same method would be declared by two unrelated interfaces
later implemented by the same class.
Do we have test cases to verify that:

public interface I1 {
     public Object test(String blah);
}
public interface I2 {
     public Object test(String blah);
}
public abstract class A implements I2 {}
public abstract class A1 extends A implements I1, I2 {}
public abstract class A2 implements I1, I2 {}
public abstract class A3 implements I2, I1 {}

A1.getMethods(), A2.getMethods(), A3.getMethods() return
the expected result?
In particular what should A2.class.getMethods()
and A3.class.getMethods() return?

best regards,

-- daniel

On 10/10/16 09:04, Peter Levart wrote:
> Just a note that this is still ready to be reviewed.
>
> I was also told that JCK tests pass with the patch applied.
>
> Regards, Peter
>
> On 10/04/2016 03:40 PM, Peter Levart wrote:
>> Hi,
>>
>> I have a fix for conformance (P2) bug (8062389
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a
>> conformance (P3) bug (8029459
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance
>> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/
>>
>>
>> As Daniel Smith writes in 8029459
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following
>> situation is as expected:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends J {}
>> K.class.getMethods() = [ J.m ]
>>
>> But the following has a different result although it should probably
>> have the same:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends I, J {}
>> K.class.getMethods() = [ I.m, J.m ]
>>
>> He then suggests the following algorithm:
>>
>> An implementation of getMethods consistent with JLS 8 would include
>> the following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
>> 1) The class's/interface's declared (public) methods
>> 2) The getMethods() of the superclass (if this is a class), minus any
>> that have a match in (1)
>> 3) The getMethods() of each superinterface, minus any that have a
>> match in (1) or a _concrete_ match in (2) or a match from a
>> more-specific class/interface in (2) or (3)
>>
>> But even that is not sufficient for the following situation:
>>
>> interface E { void m(); }
>> interface F extends E { void m(); }
>> abstract class G implements E {}
>> abstract class H extends G implements F {}
>> H.class.getMethods() = [ E.m, F.m ]
>>
>> The desired result of H.class.getMethods() = [ F.m ]
>>
>> So a more precise definition is required which is implemented in the fix.
>>
>> The getMethods() implementation partitions the union of the following
>> methods:
>>
>> 1) The class's/interface's declared public methods
>> 2) The getMethods() of the superclass (if this is a class)
>> 3) The non-static getMethods() of each direct superinterface
>>
>> ...into equivalence classes (sets) of methods with same signature
>> (return type, name, parameter types). Within each such set, only the
>> "most specific" methods are kept and others are discarded. The union
>> of the kept methods is the result.
>>
>> The "more specific" is defined as a partial order within a set of
>> methods of same signature:
>>
>> Method A is more specific than method B if:
>> - A is declared by a class and B is declared by an interface; or
>> - A is declared by the same type as or a subtype of B's declaring type
>> and both are either declared by classes or both by interfaces (clearly
>> if A and B are declared by the same type and have the same signature,
>> they are the same method)
>>
>> If A and B are distinct elements from the set of methods with same
>> signature, then either:
>> - A is more specific than B; or
>> - B is more specific than A; or
>> - A and B are incomparable
>>
>> A sub-set of "most specific" methods are the methods from the set
>> where for each such method M, there is no method N != M such that N is
>> "more specific" than M.
>>
>> There can be more than one "most specific" method for a particular
>> signature as they can be inherited from multiple unrelated interfaces,
>> but:
>> - javac prevents compilation when among multiply inherited methods
>> with same signature there is at least one default method, so in
>> practice, getMethods() will only return multiple methods with same
>> signature if they are abstract interface methods. With one exception:
>> bridge methods that are created by javac for co-variant overrides are
>> default methods in interfaces. For example:
>>
>>     interface I { Object m(); }
>>     interface J1 extends I { Map m(); }
>>     interface J2 extends I { HashMap m(); }
>>     interface K extends J1, J2 {}
>>
>> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m,
>> default Object j2.m, abstract HashMap j2.m ]
>>
>> But this is an extreme case where one can still expect multiple
>> non-abstract methods with same signature, but different declaring
>> type, returned from getMethods().
>>
>> In order to also fix 8062389
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and
>> getMethods() share the same consolidation logic that results in a set
>> of "most specific" methods. The partitioning in getMethods() is
>> partially performed by collecting Methods into a HashMap where the
>> keys are (name, parameter types) tuples and values are linked lists of
>> Method objects with possibly varying return and declaring types. The
>> consolidation, i.e. keeping only the set of most specific methods as
>> new methods are encountered, is performed only among methods in the
>> list that share same return type and also removes duplicates.
>> getMethod() uses only one such list, consolidates methods that match
>> given name and parameter types and returns the 1st method from the
>> resulting list that has the most specific return type.
>>
>> That's it for algorithms used. As partitioning partially happens using
>> HashMap with (name, parameter types) keys, lists of methods that form
>> values are typically kept short with most of them containing only a
>> single method, so this fix also fixes performance issue 8061950
>> <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>>
>> The patch also contains some optimizations around redundant copying of
>> arrays and reflective objects.
>>
>> The FilterNotMostSpecific jtreg test has been updated to accommodate
>> for changed behavior. Both of the above extreme cases have been added
>> to the test.
>>
>> So, comments, please.
>>
>> Regards, Peter
>>
>


From Alan.Bateman at oracle.com  Tue Oct 11 15:25:46 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 11 Oct 2016 16:25:46 +0100
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
Message-ID: <0075c1cd-9eba-8ea4-4cd4-1de546d7723b@oracle.com>

On 10/10/2016 23:44, Brent Christian wrote:

> On 10/10/16 2:30 PM, Mandy Chung wrote:
>>
>> The patch looks fine.  It would be good to add @see
>> #registerAsParallelCapable in this new method.  Also the first
>> ?parallel capable? occurrance in the class spec and the
>> registerAsParallelCapable method spec to @linkplain
>> #isParallelCapable.
>
> Thanks for having a look, and for the suggestions.  I also added an 
> @see #isParallelCapable in registerAsParallelCapable().
>
> Webrev updated in place:
> http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/
The javadoc looks okay to me. One thing that would be good is to beef up 
the test to cover more scenarios, esp. loader L1 extends loader L2 where 
you've got 4 combination of capable/non-capable to test.

-Alan

From mandy.chung at oracle.com  Tue Oct 11 15:47:01 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 11 Oct 2016 08:47:01 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <81d7e7d5-527f-b4e4-4f2c-bcd0832185b1@gmail.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
	<81d7e7d5-527f-b4e4-4f2c-bcd0832185b1@gmail.com>
Message-ID: <FC90F26F-04C9-4425-BB6E-939D3E24C7E7@oracle.com>


> On Oct 10, 2016, at 11:03 PM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi Brent,
> 
> 
> On 10/11/2016 12:44 AM, Brent Christian wrote:
>> On 10/10/16 2:30 PM, Mandy Chung wrote:
>>> 
>>> The patch looks fine.  It would be good to add @see
>>> #registerAsParallelCapable in this new method.  Also the first
>>> ?parallel capable? occurrance in the class spec and the
>>> registerAsParallelCapable method spec to @linkplain
>>> #isParallelCapable.
>> 
>> Thanks for having a look, and for the suggestions.  I also added an @see #isParallelCapable in registerAsParallelCapable().
>> 
>> Webrev updated in place:
>> http://cr.openjdk.java.net/~bchristi/8165793/webrev.00/
>> 
>> -Brent
> 
> It would be marginally more efficient to check for parallelLockMap != null instead of looking up the CL implementation class in the WeakHashMap within a synchronized block, but I guess the performance of this method is not very important, is it?

parallelLockMap != null is another way to check that.  Sinc this method isn?t performance critical, calling ParallelLoaders::isRegistered is easier to understand.

Mandy

From peter.levart at gmail.com  Tue Oct 11 17:13:26 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 11 Oct 2016 19:13:26 +0200
Subject: RFR: 8062389,8029459,8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <acd76378-0873-81ca-c8ce-cd490a845959@oracle.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<458ef080-d542-28e2-b8a5-5bc1e33f416b@gmail.com>
	<acd76378-0873-81ca-c8ce-cd490a845959@oracle.com>
Message-ID: <03ca1a1c-6a8b-72a4-707c-580014116bda@gmail.com>

Hi Daniel.


On 10/11/2016 05:19 PM, Daniel Fuchs wrote:
> Hi Peter,
>
> I was looking at the test - I didn't see a case where the
> same method would be declared by two unrelated interfaces
> later implemented by the same class.
> Do we have test cases to verify that:
>
> public interface I1 {
>     public Object test(String blah);
> }
> public interface I2 {
>     public Object test(String blah);
> }
> public abstract class A implements I2 {}
> public abstract class A1 extends A implements I1, I2 {}
> public abstract class A2 implements I1, I2 {}
> public abstract class A3 implements I2, I1 {}
>
> A1.getMethods(), A2.getMethods(), A3.getMethods() return
> the expected result?
> In particular what should A2.class.getMethods()
> and A3.class.getMethods() return?

import java.lang.reflect.Method;

public class Test {

     static String toMethodsString(Class<?> clazz) {
         StringBuilder sb = new StringBuilder();
         sb.append(clazz.getName()).append(": [");
         boolean first = true;
         for (Method m : clazz.getMethods()) {
             if (m.getDeclaringClass() != Object.class) {
                 if (first)
                     first = false;
                 else
                     sb.append(", ");
                 sb.append(m);
             }
         }
         sb.append("]");
         return sb.toString();
     }

     public static void main(String[] args) throws Exception {
         for (Class<?> clazz : new Class[]{A1.class, A2.class, A3.class}) {
             System.out.println(toMethodsString(clazz));
         }
     }
}

interface I1 {
     Object test(String blah);
}

interface I2 {
     Object test(String blah);
}

abstract class A implements I2 {}

abstract class A1 extends A implements I1, I2 {}

abstract class A2 implements I1, I2 {}

abstract class A3 implements I2, I1 {}



Above program, ignoring Object methods, executed with JDK 7, 8 and 
9+patch, returns the same results:

[peter at cube jdk9-dev-test]$ ~/Apps64/jdk1.7.0/bin/java -cp out Test

A1: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]
A2: [public abstract java.lang.Object I1.test(java.lang.String), public 
abstract java.lang.Object I2.test(java.lang.String)]
A3: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]

[peter at cube jdk9-dev-test]$ ~/Apps64/jdk1.8.0/bin/java -cp out Test

A1: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]
A2: [public abstract java.lang.Object I1.test(java.lang.String), public 
abstract java.lang.Object I2.test(java.lang.String)]
A3: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]

[peter at cube jdk9-dev-test]$ 
~/work/hg/jdk9-dev/build/linux-x86_64-normal-server-release/images/jdk/bin/java 
-cp out Test

A1: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]
A2: [public abstract java.lang.Object I1.test(java.lang.String), public 
abstract java.lang.Object I2.test(java.lang.String)]
A3: [public abstract java.lang.Object I2.test(java.lang.String), public 
abstract java.lang.Object I1.test(java.lang.String)]

...which is expected.

In above example, two abstract methods with same signature are inherited 
from two unrelated interfaces, but they are not later implemented by a 
class. If it was implemented (or overridden by an abstract class 
method), only the class method would show.

I agree that the test is not very systematic in covering all possible 
combinations. Maybe it is time to write new test. Let me see...

Regards, Peter

>
> best regards,
>
> -- daniel
>
> On 10/10/16 09:04, Peter Levart wrote:
>> Just a note that this is still ready to be reviewed.
>>
>> I was also told that JCK tests pass with the patch applied.
>>
>> Regards, Peter
>>
>> On 10/04/2016 03:40 PM, Peter Levart wrote:
>>> Hi,
>>>
>>> I have a fix for conformance (P2) bug (8062389
>>> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a
>>> conformance (P3) bug (8029459
>>> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance
>>> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>>>
>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/ 
>>>
>>>
>>>
>>> As Daniel Smith writes in 8029459
>>> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following
>>> situation is as expected:
>>>
>>> interface I { void m(); }
>>> interface J extends I { void m(); }
>>> interface K extends J {}
>>> K.class.getMethods() = [ J.m ]
>>>
>>> But the following has a different result although it should probably
>>> have the same:
>>>
>>> interface I { void m(); }
>>> interface J extends I { void m(); }
>>> interface K extends I, J {}
>>> K.class.getMethods() = [ I.m, J.m ]
>>>
>>> He then suggests the following algorithm:
>>>
>>> An implementation of getMethods consistent with JLS 8 would include
>>> the following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
>>> 1) The class's/interface's declared (public) methods
>>> 2) The getMethods() of the superclass (if this is a class), minus any
>>> that have a match in (1)
>>> 3) The getMethods() of each superinterface, minus any that have a
>>> match in (1) or a _concrete_ match in (2) or a match from a
>>> more-specific class/interface in (2) or (3)
>>>
>>> But even that is not sufficient for the following situation:
>>>
>>> interface E { void m(); }
>>> interface F extends E { void m(); }
>>> abstract class G implements E {}
>>> abstract class H extends G implements F {}
>>> H.class.getMethods() = [ E.m, F.m ]
>>>
>>> The desired result of H.class.getMethods() = [ F.m ]
>>>
>>> So a more precise definition is required which is implemented in the 
>>> fix.
>>>
>>> The getMethods() implementation partitions the union of the following
>>> methods:
>>>
>>> 1) The class's/interface's declared public methods
>>> 2) The getMethods() of the superclass (if this is a class)
>>> 3) The non-static getMethods() of each direct superinterface
>>>
>>> ...into equivalence classes (sets) of methods with same signature
>>> (return type, name, parameter types). Within each such set, only the
>>> "most specific" methods are kept and others are discarded. The union
>>> of the kept methods is the result.
>>>
>>> The "more specific" is defined as a partial order within a set of
>>> methods of same signature:
>>>
>>> Method A is more specific than method B if:
>>> - A is declared by a class and B is declared by an interface; or
>>> - A is declared by the same type as or a subtype of B's declaring type
>>> and both are either declared by classes or both by interfaces (clearly
>>> if A and B are declared by the same type and have the same signature,
>>> they are the same method)
>>>
>>> If A and B are distinct elements from the set of methods with same
>>> signature, then either:
>>> - A is more specific than B; or
>>> - B is more specific than A; or
>>> - A and B are incomparable
>>>
>>> A sub-set of "most specific" methods are the methods from the set
>>> where for each such method M, there is no method N != M such that N is
>>> "more specific" than M.
>>>
>>> There can be more than one "most specific" method for a particular
>>> signature as they can be inherited from multiple unrelated interfaces,
>>> but:
>>> - javac prevents compilation when among multiply inherited methods
>>> with same signature there is at least one default method, so in
>>> practice, getMethods() will only return multiple methods with same
>>> signature if they are abstract interface methods. With one exception:
>>> bridge methods that are created by javac for co-variant overrides are
>>> default methods in interfaces. For example:
>>>
>>>     interface I { Object m(); }
>>>     interface J1 extends I { Map m(); }
>>>     interface J2 extends I { HashMap m(); }
>>>     interface K extends J1, J2 {}
>>>
>>> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m,
>>> default Object j2.m, abstract HashMap j2.m ]
>>>
>>> But this is an extreme case where one can still expect multiple
>>> non-abstract methods with same signature, but different declaring
>>> type, returned from getMethods().
>>>
>>> In order to also fix 8062389
>>> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and
>>> getMethods() share the same consolidation logic that results in a set
>>> of "most specific" methods. The partitioning in getMethods() is
>>> partially performed by collecting Methods into a HashMap where the
>>> keys are (name, parameter types) tuples and values are linked lists of
>>> Method objects with possibly varying return and declaring types. The
>>> consolidation, i.e. keeping only the set of most specific methods as
>>> new methods are encountered, is performed only among methods in the
>>> list that share same return type and also removes duplicates.
>>> getMethod() uses only one such list, consolidates methods that match
>>> given name and parameter types and returns the 1st method from the
>>> resulting list that has the most specific return type.
>>>
>>> That's it for algorithms used. As partitioning partially happens using
>>> HashMap with (name, parameter types) keys, lists of methods that form
>>> values are typically kept short with most of them containing only a
>>> single method, so this fix also fixes performance issue 8061950
>>> <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>>>
>>> The patch also contains some optimizations around redundant copying of
>>> arrays and reflective objects.
>>>
>>> The FilterNotMostSpecific jtreg test has been updated to accommodate
>>> for changed behavior. Both of the above extreme cases have been added
>>> to the test.
>>>
>>> So, comments, please.
>>>
>>> Regards, Peter
>>>
>>
>


From mandy.chung at oracle.com  Tue Oct 11 18:03:14 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 11 Oct 2016 11:03:14 -0700
Subject: Review Request: JDK-8164689: Retrofit jar, jlink,
	jmod as a ToolProvider
Message-ID: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>

This patch updates jar, jlink, jmod tool to be a provider of the new tool SPI.  Some tests are also updated to replace the use of internal APIs with ToolProvider::findFirst to look up a tool provider.  There are more tests that can be updated and something to be cleaned up in the future.

http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.00/index.html

thanks
Mandy

From lance.andersen at oracle.com  Tue Oct 11 19:13:05 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Tue, 11 Oct 2016 15:13:05 -0400
Subject: Review Request: JDK-8164689: Retrofit jar, jlink,
	jmod as a ToolProvider
In-Reply-To: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
References: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
Message-ID: <64F78510-90B1-4EFF-AE60-BB40730DDA77@oracle.com>

The changes look good Mandy

> On Oct 11, 2016, at 2:03 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> This patch updates jar, jlink, jmod tool to be a provider of the new tool SPI.  Some tests are also updated to replace the use of internal APIs with ToolProvider::findFirst to look up a tool provider.  There are more tests that can be updated and something to be cleaned up in the future.
> 
> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.00/index.html
> 
> thanks
> Mandy

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From paul.sandoz at oracle.com  Tue Oct 11 19:26:55 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 11 Oct 2016 12:26:55 -0700
Subject: using jshell in executable UNIX scripts
In-Reply-To: <a8f765ed-6968-473a-a6d7-fa4ae0305139@gmail.com>
References: <a8f765ed-6968-473a-a6d7-fa4ae0305139@gmail.com>
Message-ID: <C1386533-A254-4698-BB14-E3A1593454AE@oracle.com>

Hi Peter,

I was wondering when this would be discussed :-)

This potentially starts moving down the slippery slope of what it means to be Java syntax and what it means to execute. Larger topics to be thought about.

Here is some interesting discussion about Go:

  http://golangcookbook.com/chapters/running/shebang/

Surprisingly you can make it work in bash (from the above link):

  //$JDK_HOME/bin/jshell $0 $@; exit $?
  System.out.println("HELLO");

  /exit

Paul.

> On 10 Oct 2016, at 04:03, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi,
> 
> I don't know if this is the right list to discuss this, so please direct me to a more suitable place if there is one.
> 
> "jshell" command is a very nice interactive Java shell, but it could also be used for scripting. An executable script in any major UNIX OS is a textual file with executable permissions that starts with the following two characters: #!
> The rest of the 1st line is the path to the interpreter executable and any arguments passed to it. The last argument passed to the interpreter is the path to the executable script. In case of jshell, one would want such script to be written like:
> 
> #!/home/peter/Apps64/jdk9/bin/jshell
> 
> System.out.println("Hello World!");
> 
> /exit
> 
> 
> The problem is that jshell tries to parse the 1st line using jshell syntax and the result of running above executable script is:
> 
> | Error:
> |  illegal character: '#'
> | #!/home/peter/Apps64/jdk9/bin/jshell
> |  ^
> |  Error:
> |  illegal start of expression
> |  #!/home/peter/Apps64/jdk9/bin/jshell
> |    ^
> Hello World!
> 
> 
> The script is actually executed, but the syntax error encountered in the 1st line is printed too.
> 
> Would it be possible for jshell to skip 1st line if it starts with characters #! like other shells do?
> 
> 
> Regards, Peter
> 


From huizhe.wang at oracle.com  Tue Oct 11 20:29:44 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Tue, 11 Oct 2016 13:29:44 -0700
Subject: RFR (JAXP) 8152530: NullPointerException when xmlns=""
Message-ID: <57FD4BB8.2010707@oracle.com>

Hi,

Please review a fix for a NPE when the source contains empty namespace.

JBS: https://bugs.openjdk.java.net/browse/JDK-8152530
webrev: http://cr.openjdk.java.net/~joehw/jdk9/8152530/webrev/

Thanks,
Joe

From daniel.fuchs at oracle.com  Tue Oct 11 20:39:32 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Tue, 11 Oct 2016 21:39:32 +0100
Subject: RFR (JAXP) 8152530: NullPointerException when xmlns=""
In-Reply-To: <57FD4BB8.2010707@oracle.com>
References: <57FD4BB8.2010707@oracle.com>
Message-ID: <c73a93bd-2e2b-9b2a-bb33-6e27f77bbf41@oracle.com>

Looks good to me Joe!

best regards,

-- daniel

On 11/10/16 21:29, Joe Wang wrote:
> Hi,
>
> Please review a fix for a NPE when the source contains empty namespace.
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8152530
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8152530/webrev/
>
> Thanks,
> Joe


From naoto.sato at oracle.com  Tue Oct 11 20:41:03 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 11 Oct 2016 13:41:03 -0700
Subject: RFR (JAXP) 8152530: NullPointerException when xmlns=""
In-Reply-To: <57FD4BB8.2010707@oracle.com>
References: <57FD4BB8.2010707@oracle.com>
Message-ID: <68a5b029-1adc-655f-57bd-e2107a123f44@oracle.com>

Hi Joe,

You could use prefix.isEmpty() to check if it's an empty string. 
Otherwise looks fine to me.

Naoto

On 10/11/16 1:29 PM, Joe Wang wrote:
> Hi,
>
> Please review a fix for a NPE when the source contains empty namespace.
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8152530
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8152530/webrev/
>
> Thanks,
> Joe

From paul.sandoz at oracle.com  Tue Oct 11 21:27:46 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 11 Oct 2016 14:27:46 -0700
Subject: RFR 8167524 Rogue character in Stream javadoc
Message-ID: <E54C52C0-3689-4DD0-9391-580185028E4A@oracle.com>

Hi,

I accidentally fat fingered the JavaDoc of Stream which got consumed into an unrelated patch that i pushed [1]. Thanks go to Martin for noticing this.

Paul.

[1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/e93b7ea55975

diff -r b909daf8fdbc src/java.base/share/classes/java/util/stream/Stream.java
--- a/src/java.base/share/classes/java/util/stream/Stream.java	Tue Oct 11 12:33:15 2016 +0200
+++ b/src/java.base/share/classes/java/util/stream/Stream.java	Tue Oct 11 14:23:46 2016 -0700
@@ -282,7 +282,7 @@
      */
     <R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper);

-    /**:
+    /**
      * Returns an {@code IntStream} consisting of the results of replacing each
      * element of this stream with the contents of a mapped stream produced by
      * applying the provided mapping function to each element.  Each mapped


From joe.darcy at oracle.com  Tue Oct 11 21:30:03 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Tue, 11 Oct 2016 14:30:03 -0700
Subject: RFR 8167524 Rogue character in Stream javadoc
In-Reply-To: <E54C52C0-3689-4DD0-9391-580185028E4A@oracle.com>
References: <E54C52C0-3689-4DD0-9391-580185028E4A@oracle.com>
Message-ID: <57FD59DB.7030604@oracle.com>

+1

-Joe

On 10/11/2016 2:27 PM, Paul Sandoz wrote:
> Hi,
>
> I accidentally fat fingered the JavaDoc of Stream which got consumed into an unrelated patch that i pushed [1]. Thanks go to Martin for noticing this.
>
> Paul.
>
> [1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/e93b7ea55975
>
> diff -r b909daf8fdbc src/java.base/share/classes/java/util/stream/Stream.java
> --- a/src/java.base/share/classes/java/util/stream/Stream.java	Tue Oct 11 12:33:15 2016 +0200
> +++ b/src/java.base/share/classes/java/util/stream/Stream.java	Tue Oct 11 14:23:46 2016 -0700
> @@ -282,7 +282,7 @@
>        */
>       <R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper);
>
> -    /**:
> +    /**
>        * Returns an {@code IntStream} consisting of the results of replacing each
>        * element of this stream with the contents of a mapped stream produced by
>        * applying the provided mapping function to each element.  Each mapped
>


From huizhe.wang at oracle.com  Tue Oct 11 21:32:04 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Tue, 11 Oct 2016 14:32:04 -0700
Subject: RFR (JAXP) 8152530: NullPointerException when xmlns=""
In-Reply-To: <68a5b029-1adc-655f-57bd-e2107a123f44@oracle.com>
References: <57FD4BB8.2010707@oracle.com>
	<68a5b029-1adc-655f-57bd-e2107a123f44@oracle.com>
Message-ID: <57FD5A54.2020802@oracle.com>

Thanks Daniel, Naoto!

Changed to use prefix.isEmpty().

-Joe

On 10/11/16, 1:41 PM, Naoto Sato wrote:
> Hi Joe,
>
> You could use prefix.isEmpty() to check if it's an empty string. 
> Otherwise looks fine to me.
>
> Naoto
>
> On 10/11/16 1:29 PM, Joe Wang wrote:
>> Hi,
>>
>> Please review a fix for a NPE when the source contains empty namespace.
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8152530
>> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8152530/webrev/
>>
>> Thanks,
>> Joe

From stuart.marks at oracle.com  Tue Oct 11 21:32:42 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 11 Oct 2016 14:32:42 -0700
Subject: RFR 8167524 Rogue character in Stream javadoc
In-Reply-To: <57FD59DB.7030604@oracle.com>
References: <E54C52C0-3689-4DD0-9391-580185028E4A@oracle.com>
	<57FD59DB.7030604@oracle.com>
Message-ID: <32055bca-134a-cb2b-610e-72044d87b04a@oracle.com>

-1 (character)

j/k, LGTM

On 10/11/16 2:30 PM, Joseph D. Darcy wrote:
> +1
>
> -Joe
>
> On 10/11/2016 2:27 PM, Paul Sandoz wrote:
>> Hi,
>>
>> I accidentally fat fingered the JavaDoc of Stream which got consumed into an
>> unrelated patch that i pushed [1]. Thanks go to Martin for noticing this.
>>
>> Paul.
>>
>> [1] http://hg.openjdk.java.net/jdk9/dev/jdk/rev/e93b7ea55975
>>
>> diff -r b909daf8fdbc src/java.base/share/classes/java/util/stream/Stream.java
>> --- a/src/java.base/share/classes/java/util/stream/Stream.java    Tue Oct 11
>> 12:33:15 2016 +0200
>> +++ b/src/java.base/share/classes/java/util/stream/Stream.java    Tue Oct 11
>> 14:23:46 2016 -0700
>> @@ -282,7 +282,7 @@
>>        */
>>       <R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>>
>> mapper);
>>
>> -    /**:
>> +    /**
>>        * Returns an {@code IntStream} consisting of the results of replacing each
>>        * element of this stream with the contents of a mapped stream produced by
>>        * applying the provided mapping function to each element.  Each mapped
>>
>

From paul.sandoz at oracle.com  Tue Oct 11 21:35:19 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 11 Oct 2016 14:35:19 -0700
Subject: RFR: 8062389, 8029459,
	8061950: Class.getMethod() is inconsistent with Class.getMethods()
	results + more
In-Reply-To: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
Message-ID: <1F5607C1-9EFC-4881-A3A9-31CB948FE67C@oracle.com>

Hi Peter,

Nice work here.

PublicMethods
?

I would be inclined to change PublicMethods to encapsulate an instance of LinkedHashMap, since it?s only the consolidate/toArray methods that really matter, and no need for the key/value types to be exposed, then no need to declare serialVersionUID, and also it can be final. The JavaDoc on consolidate would need to be tweaked.

  29      * existing methods with same signature if they are less specific then added
s/then/than

  89             if (o == null || getClass() != o.getClass()) return false;

Could be simplified to

  if (!(o instanceof Key)) return false

Paul.

> On 4 Oct 2016, at 06:40, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi,
> 
> I have a fix for conformance (P2) bug (8062389 <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a conformance (P3) bug (8029459 <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
> 
> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/
> 
> As Daniel Smith writes in 8029459 <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following situation is as expected:
> 
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends J {}
> K.class.getMethods() = [ J.m ]
> 
> But the following has a different result although it should probably have the same:
> 
> interface I { void m(); }
> interface J extends I { void m(); }
> interface K extends I, J {}
> K.class.getMethods() = [ I.m, J.m ]
> 
> He then suggests the following algorithm:
> 
> An implementation of getMethods consistent with JLS 8 would include the following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
> 1) The class's/interface's declared (public) methods
> 2) The getMethods() of the superclass (if this is a class), minus any that have a match in (1)
> 3) The getMethods() of each superinterface, minus any that have a match in (1) or a _concrete_ match in (2) or a match from a more-specific class/interface in (2) or (3)
> 
> But even that is not sufficient for the following situation:
> 
> interface E { void m(); }
> interface F extends E { void m(); }
> abstract class G implements E {}
> abstract class H extends G implements F {}
> H.class.getMethods() = [ E.m, F.m ]
> 
> The desired result of H.class.getMethods() = [ F.m ]
> 
> So a more precise definition is required which is implemented in the fix.
> 
> The getMethods() implementation partitions the union of the following methods:
> 
> 1) The class's/interface's declared public methods
> 2) The getMethods() of the superclass (if this is a class)
> 3) The non-static getMethods() of each direct superinterface
> 
> ...into equivalence classes (sets) of methods with same signature (return type, name, parameter types). Within each such set, only the "most specific" methods are kept and others are discarded. The union of the kept methods is the result.
> 
> The "more specific" is defined as a partial order within a set of methods of same signature:
> 
> Method A is more specific than method B if:
> - A is declared by a class and B is declared by an interface; or
> - A is declared by the same type as or a subtype of B's declaring type and both are either declared by classes or both by interfaces (clearly if A and B are declared by the same type and have the same signature, they are the same method)
> 
> If A and B are distinct elements from the set of methods with same signature, then either:
> - A is more specific than B; or
> - B is more specific than A; or
> - A and B are incomparable
> 
> A sub-set of "most specific" methods are the methods from the set where for each such method M, there is no method N != M such that N is "more specific" than M.
> 
> There can be more than one "most specific" method for a particular signature as they can be inherited from multiple unrelated interfaces, but:
> - javac prevents compilation when among multiply inherited methods with same signature there is at least one default method, so in practice, getMethods() will only return multiple methods with same signature if they are abstract interface methods. With one exception: bridge methods that are created by javac for co-variant overrides are default methods in interfaces. For example:
> 
>    interface I { Object m(); }
>    interface J1 extends I { Map m(); }
>    interface J2 extends I { HashMap m(); }
>    interface K extends J1, J2 {}
> 
> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, default Object j2.m, abstract HashMap j2.m ]
> 
> But this is an extreme case where one can still expect multiple non-abstract methods with same signature, but different declaring type, returned from getMethods().
> 
> In order to also fix 8062389 <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and getMethods() share the same consolidation logic that results in a set of "most specific" methods. The partitioning in getMethods() is partially performed by collecting Methods into a HashMap where the keys are (name, parameter types) tuples and values are linked lists of Method objects with possibly varying return and declaring types. The consolidation, i.e. keeping only the set of most specific methods as new methods are encountered, is performed only among methods in the list that share same return type and also removes duplicates. getMethod() uses only one such list, consolidates methods that match given name and parameter types and returns the 1st method from the resulting list that has the most specific return type.
> 
> That's it for algorithms used. As partitioning partially happens using HashMap with (name, parameter types) keys, lists of methods that form values are typically kept short with most of them containing only a single method, so this fix also fixes performance issue 8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>.
> 
> The patch also contains some optimizations around redundant copying of arrays and reflective objects.
> 
> The FilterNotMostSpecific jtreg test has been updated to accommodate for changed behavior. Both of the above extreme cases have been added to the test.
> 
> So, comments, please.
> 
> Regards, Peter
> 


From naoto.sato at oracle.com  Tue Oct 11 21:37:32 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 11 Oct 2016 14:37:32 -0700
Subject: RFR (JAXP) 8152530: NullPointerException when xmlns=""
In-Reply-To: <57FD5A54.2020802@oracle.com>
References: <57FD4BB8.2010707@oracle.com>
	<68a5b029-1adc-655f-57bd-e2107a123f44@oracle.com>
	<57FD5A54.2020802@oracle.com>
Message-ID: <7ba72d75-446e-db43-0d74-c4728562e6f4@oracle.com>

Thanks, Joe. +1

Naoto

On 10/11/16 2:32 PM, Joe Wang wrote:
> Thanks Daniel, Naoto!
>
> Changed to use prefix.isEmpty().
>
> -Joe
>
> On 10/11/16, 1:41 PM, Naoto Sato wrote:
>> Hi Joe,
>>
>> You could use prefix.isEmpty() to check if it's an empty string.
>> Otherwise looks fine to me.
>>
>> Naoto
>>
>> On 10/11/16 1:29 PM, Joe Wang wrote:
>>> Hi,
>>>
>>> Please review a fix for a NPE when the source contains empty namespace.
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8152530
>>> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8152530/webrev/
>>>
>>> Thanks,
>>> Joe

From stuart.marks at oracle.com  Tue Oct 11 23:03:08 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 11 Oct 2016 16:03:08 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
Message-ID: <15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>



On 10/10/16 11:26 PM, Andrej Golovnin wrote:
> src/java.base/share/classes/java/util/ResourceBundle.java
>
> 2490     public static class Control {
> 2491         /**
> 2492          * The default format <code>List</code>, which contains the strings
> 2493          * <code>"java.class"</code> and <code>"java.properties"</code>, in
> 2494          * this order. This <code>List</code> is {@linkplain
> 2495          * Collections#unmodifiableList(List) unmodifiable}.
> 2496          *
> 2497          * @see #getFormats(String)
> 2498          */
> 2499         public static final List<String> FORMAT_DEFAULT =
> List.of("java.class", "java.properties");
>
> I think you should also change the JavaDocs in the
> ResourceBundle.Control class for the constants FORMAT_CLASS,
> FORMAT_DEFAULT and FORMAT_PROPERTIES, because the JavaDocs for this
> constants explicitly mentions, that the lists are created by using
> Collections#unmodifiableList(List). Or you cannot change this
> constants at all because they are part of the Public API and existed
> in this form for a long time. Please ask someone from Oracle for help.
> They can explain it better when it is OK to change and when not. Maybe
> Stuart can do that.

Hi Andrej,

Thanks for pointing this out.

It appears that the changes to remove the links to 
Collections.unmodifiableList() was dropped from webrev.01 to webrev.02. I've 
restored them, along with a couple other changes that were also dropped. I also 
updated the ModuleFinder.java comment per a request from Alan Bateman. The 
revised webrev is here:

http://cr.openjdk.java.net/~smarks/reviews/8134373/webrev.03/

In any case, yes, the specifications of the ResourceBundle.Control fields should 
be changed to remove the links to Collections.unmodifiableList(). It's unclear 
whether having a link this way implies that it's part of the specification that 
these fields must be instances returned from that method. Removing the link 
makes it clear that saying the List is unmodifiable is merely a description of 
the required behavior.

I spoke with Joe Darcy (our compatibility guru) and we agreed that out of an 
abundance of caution it would be wise to file a request to make this change. 
(This is the "CCC" - basically an internal change control process for Java SE 
specifications.) Doing this is mainly for tracking purposes, as we believe this 
to be a compatible change.

I've also included in this request a mention of the change to 
CookieManager.get() which we had discussed previously. Even though we believe 
this is also a compatible change, it's also a change that should be tracked.

I'll follow up when this bit of process is finished.

s'marks

From naoto.sato at oracle.com  Tue Oct 11 23:13:42 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 11 Oct 2016 16:13:42 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
Message-ID: <6f3b07f2-569c-fca8-3730-e769d5e4a445@oracle.com>

+1 for removing the link to Collections#unmodifiableList in the spec. I 
don't see any particular reason to specify in the javadoc and don't 
believe it would break any existing apps.

Naoto

On 10/11/16 4:03 PM, Stuart Marks wrote:
>
>
> On 10/10/16 11:26 PM, Andrej Golovnin wrote:
>> src/java.base/share/classes/java/util/ResourceBundle.java
>>
>> 2490     public static class Control {
>> 2491         /**
>> 2492          * The default format <code>List</code>, which contains
>> the strings
>> 2493          * <code>"java.class"</code> and
>> <code>"java.properties"</code>, in
>> 2494          * this order. This <code>List</code> is {@linkplain
>> 2495          * Collections#unmodifiableList(List) unmodifiable}.
>> 2496          *
>> 2497          * @see #getFormats(String)
>> 2498          */
>> 2499         public static final List<String> FORMAT_DEFAULT =
>> List.of("java.class", "java.properties");
>>
>> I think you should also change the JavaDocs in the
>> ResourceBundle.Control class for the constants FORMAT_CLASS,
>> FORMAT_DEFAULT and FORMAT_PROPERTIES, because the JavaDocs for this
>> constants explicitly mentions, that the lists are created by using
>> Collections#unmodifiableList(List). Or you cannot change this
>> constants at all because they are part of the Public API and existed
>> in this form for a long time. Please ask someone from Oracle for help.
>> They can explain it better when it is OK to change and when not. Maybe
>> Stuart can do that.
>
> Hi Andrej,
>
> Thanks for pointing this out.
>
> It appears that the changes to remove the links to
> Collections.unmodifiableList() was dropped from webrev.01 to webrev.02.
> I've restored them, along with a couple other changes that were also
> dropped. I also updated the ModuleFinder.java comment per a request from
> Alan Bateman. The revised webrev is here:
>
> http://cr.openjdk.java.net/~smarks/reviews/8134373/webrev.03/
>
> In any case, yes, the specifications of the ResourceBundle.Control
> fields should be changed to remove the links to
> Collections.unmodifiableList(). It's unclear whether having a link this
> way implies that it's part of the specification that these fields must
> be instances returned from that method. Removing the link makes it clear
> that saying the List is unmodifiable is merely a description of the
> required behavior.
>
> I spoke with Joe Darcy (our compatibility guru) and we agreed that out
> of an abundance of caution it would be wise to file a request to make
> this change. (This is the "CCC" - basically an internal change control
> process for Java SE specifications.) Doing this is mainly for tracking
> purposes, as we believe this to be a compatible change.
>
> I've also included in this request a mention of the change to
> CookieManager.get() which we had discussed previously. Even though we
> believe this is also a compatible change, it's also a change that should
> be tracked.
>
> I'll follow up when this bit of process is finished.
>
> s'marks

From Alan.Bateman at oracle.com  Wed Oct 12 06:07:59 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 12 Oct 2016 07:07:59 +0100
Subject: Review Request: JDK-8164689: Retrofit jar, jlink, jmod as a
	ToolProvider
In-Reply-To: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
References: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
Message-ID: <9f125788-9866-78d9-8ccf-e06d0755426e@oracle.com>

On 11/10/2016 19:03, Mandy Chung wrote:

> This patch updates jar, jlink, jmod tool to be a provider of the new tool SPI.  Some tests are also updated to replace the use of internal APIs with ToolProvider::findFirst to look up a tool provider.  There are more tests that can be updated and something to be cleaned up in the future.
>
> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.00/index.html
>
This update to the tools looks good. For the tests then 
ToolProvider.findFirst("jar").get() will draw the attention of the 
optional police and might be better to use orElseThrow to supply an 
exception that clearly indicates that the tool cannot be found.

-Alan

From timo.kinnunen at gmail.com  Wed Oct 12 07:33:54 2016
From: timo.kinnunen at gmail.com (timo.kinnunen at gmail.com)
Date: Wed, 12 Oct 2016 09:33:54 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or()
	andflatMap()
In-Reply-To: <d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
	<CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
	<d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
Message-ID: <57fde766.c89a1c0a.97b0d.6b9c@mx.google.com>

Hi, 


I?m going to challenge the consensus a little bit. First, R?mi's example can be simplified to this one method which fails to compile with the same error message[0]:

private static Optional<?> simple1(
 Optional<String> o, Function<String, Optional<?>> f
) {return o.flatMap(f);} 

Second, although not in the webrev, Optional::map and Optional::flatMap can be implemented without needing any unchecked casts:

public <U, R extends U, O extends Optional<R>> Optional<U> flatMap(
 Function<T, O> mapper
) {
 Objects.requireNonNull(mapper);
 return ofNullable(isPresent() ? mapper.apply(get()).orElse(null) : null);
}
public <P, Q extends P> Optional<P> map(
 Function<T, Q> mapper
) {
 Objects.requireNonNull(mapper);
 return ofNullable(isPresent() ? mapper.apply(get()) : null);
}

These are fully, soundly compile-time typed methods with all types exposed and completely under the control of the programmer.

Now, to see if the signature of flatMap is truly the problem, it?s possible to write the simple1 method as a map followed by a flatMap with the identity function, like this:

private static Optional<?> simple2(
 Optional<String> o, Function<String, Optional<?>> f
) {return o.map(f).flatMap(Function.identity());}

In this version, the call to flatMap and its argument don?t use any wildcard types, both in the version above and in java.util.Optional[1]. Despite that, the compiler from jdk1.8.0_102 still gives an error[2] from the flatMap method call. This is quite a curious result as here flatMap and identity are reusing types that are already used by type inference. If this isn?t sound but using wildcards is then I would really like to see that counterexample!

Some questions that have arisen (and my answers):

Should APIs account for types that are not denotable in source code? I?d say no, such types are bugs in the language.

Can non-denotable types be eliminated at the library level by adding more wildcards? Unlikely, as such types come from using wildcards to begin with.

Is there a bug in flatMap or is the bug in the language specification? Compared to one simple method, type inference rules are much harder to understand and thus more likely to contain undiscovered problems.

What is gained if these wildcards are added? Although simple1 and simple2 would compile, you still can?t do anything interesting like calling orElse or orElseGet:

public static void main(String[] args) {
 Optional<String> foo = Optional.ofNullable(args[0]);
 Object o1 = simple1(foo, s -> foo).orElse(o1 = null);
 Object o2 = simple2(foo, s -> foo).orElseGet(() -> null);
}

Won?t compile. As far as I can tell, there is no real upside to this change.


Thanks for your consideration!




[0] Error:(18, 20) java: method flatMap in class java.util.Optional<T> cannot be applied to given types;
  required: java.util.function.Function<? super java.lang.String,java.util.Optional<U>>
  found: java.util.function.Function<java.lang.String,java.util.Optional<?>>
  reason: cannot infer type-variable(s) U
    (argument mismatch; java.util.function.Function<java.lang.String,java.util.Optional<?>> cannot be converted to java.util.function.Function<? super java.lang.String,java.util.Optional<U>>)

[1] The <? super T> type can be discounted as its presence doesn?t make a difference in this case.

[2] Error:(21, 27) java: method flatMap in class wildcards.another.Optional<T> cannot be applied to given types;
  required: java.util.function.Function<wildcards.another.Optional<?>,O>
  found: java.util.function.Function<java.lang.Object,java.lang.Object>
  reason: inference variable O has incompatible bounds
    equality constraints: wildcards.another.Optional<?>,T
    upper bounds: wildcards.another.Optional<capture#11 of ?>,java.lang.Object


-- 
Have a nice day, 
Timo

Sent from Mail for Windows 10

From: Stuart Marks
Sent: Saturday, October 8, 2016 02:28
To: Stefan Zobel
Cc: core-libs-dev
Subject: Re: RFR(s): 8152617 add missing wildcards to Optional or() andflatMap()



On 10/7/16 3:12 PM, Stefan Zobel wrote:
>> ... After having looked at lots of generic APIs, it seems to me that a
>> style has emerged where wildcards are used whenever possible, and type
>> parameters are used only when necessary, ...
>
> Yes, I'm familiar with that kind of reasoning (I think Josh Bloch stated
> that principle in "Effective Java"). But, to be honest, I never thought
> that it should apply as a strict rule in all circumstances.

Yep, it's in Effective Java, near the end of Item 28:

     ?As a rule, if a type parameter appears only once in a method
     declaration, replace it with a wildcard.?

> Rhetorical question: do you really think that a signature employing 3
> wildcards is easier to understand for the proverbial "average Joe" than
> a bounded type parameter that expresses the sub-type relationship clearly?
> I do not.
>
> But anyway, you probably have to follow the established "style".
>
> It's just too bad that most Java programmers I know won't understand the
> proposed signature of 'flatMap'.

Turns out that R?mi's example exposes the difference between the wildcard 
approach and the type-parameter approach. Returning to the example,

     Optional<Integer> oi = Optional.empty();
     Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
     Optional<CharSequence> ocs = oi.flatMap(fm);

If the flatMapper function itself has a wildcard type, for example,

     Function<Number, Optional<? extends CharSequence>> fm = n -> Optional.empty();

then this will still work with the wildcard approach but fail with the 
type-parameter approach. As R?mi also pointed out, a wildcarded type can result 
from the capture of a type with a wildcarded type parameter.

Based on this, I believe the nested wildcard approach to be the correct one.

s'marks



From andrej.golovnin at gmail.com  Wed Oct 12 09:07:48 2016
From: andrej.golovnin at gmail.com (Andrej Golovnin)
Date: Wed, 12 Oct 2016 11:07:48 +0200
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
Message-ID: <CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>

Hi Stuart,

> It appears that the changes to remove the links to
> Collections.unmodifiableList() was dropped from webrev.01 to webrev.02. I've
> restored them, along with a couple other changes that were also dropped. I
> also updated the ModuleFinder.java comment per a request from Alan Bateman.
> The revised webrev is here:
>
> http://cr.openjdk.java.net/~smarks/reviews/8134373/webrev.03/

looks good. Thanks!

Best regards,
Andrej Golovnin

>
> In any case, yes, the specifications of the ResourceBundle.Control fields
> should be changed to remove the links to Collections.unmodifiableList().
> It's unclear whether having a link this way implies that it's part of the
> specification that these fields must be instances returned from that method.
> Removing the link makes it clear that saying the List is unmodifiable is
> merely a description of the required behavior.
>
> I spoke with Joe Darcy (our compatibility guru) and we agreed that out of an
> abundance of caution it would be wise to file a request to make this change.
> (This is the "CCC" - basically an internal change control process for Java
> SE specifications.) Doing this is mainly for tracking purposes, as we
> believe this to be a compatible change.
>
> I've also included in this request a mention of the change to
> CookieManager.get() which we had discussed previously. Even though we
> believe this is also a compatible change, it's also a change that should be
> tracked.
>
> I'll follow up when this bit of process is finished.
>
> s'marks

From anubhav.meena at oracle.com  Wed Oct 12 09:42:58 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Wed, 12 Oct 2016 15:12:58 +0530
Subject: Reminder
In-Reply-To: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
Message-ID: <0BC3DCE0-CD18-44D8-B830-034DE3DF42A4@oracle.com>

Hi All,

This is a reminder for the pending review.

> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
> 
> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
> 
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>

Thanks,
Anubhav


> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com> wrote:
> 
> Hi all,
> 
> Please review.
> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
> 
> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
> 
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>
> 
> -- 
> Thanks and Regards,
> Anubhav


From anubhav.meena at oracle.com  Wed Oct 12 09:51:52 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Wed, 12 Oct 2016 15:21:52 +0530
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
Message-ID: <542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>

Gentle reminder.

> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com> wrote:
> 
> Hi all,
> 
> Please review.
> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
> 
> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
> 
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>
> 
> -- 
> Thanks and Regards,
> Anubhav


From sergei.kovalev at oracle.com  Wed Oct 12 09:54:04 2016
From: sergei.kovalev at oracle.com (Sergei Kovalev)
Date: Wed, 12 Oct 2016 12:54:04 +0300
Subject: RFR(s): 8167437: Fix module dependencies for tests that use
	internal API (java/lang)
In-Reply-To: <9cd9c074-773f-6a34-6bc1-4f215476edd9@oracle.com>
References: <023d8d95-a4ab-fbbd-729c-b78a16b0c57c@oracle.com>
	<feaad26a-f27f-1138-e5e3-5c08a2642e5a@oracle.com>
	<d78c2467-1904-a357-ee0c-2178dc7f7b49@oracle.com>
	<617c2b37-1e37-ba74-f1d9-71eb9ba760e6@oracle.com>
	<9cd9c074-773f-6a34-6bc1-4f215476edd9@oracle.com>
Message-ID: <cc0706d4-71a1-b6ab-bb53-8e7cb9a91385@oracle.com>

The fixed version is here: 
http://cr.openjdk.java.net/~skovalev/8167437/webrev.01/

I also modified CR description.


10.10.16 18:15, Sergei Kovalev wrote:
> You are right. I'll rollback the change for the file.
>
>
> 10.10.16 18:05, Alan Bateman wrote:
>> sun.reflect.generics.parser is in java.base, I don't see anything 
>> using jdk.unsupported/sun.reflect. You can check this quickly by 
>> running the test without your change.
>>
>> -Alan
>

-- 
With best regards,
Sergei


From scolebourne at joda.org  Wed Oct 12 11:00:05 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 12 Oct 2016 12:00:05 +0100
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
Message-ID: <CACzrW9Daze6c0m2BrZxqK1Uxx8fU1_mRWSnrWLZeHSTUBb_pKQ@mail.gmail.com>

The indentation in TCKHijrahChronology is not correct.

In TCKHijrahChronology, it would be better to add the expected values
to the provider
ie two additional value for aligned week and aligned day:
{1437, 9, 1, 1, 1}
{1437, 9, 2, 1, 2}
...

That way, the test relies on facts, not on a formula.

Stephen



On 7 October 2016 at 09:42, Anubhav Meena <anubhav.meena at oracle.com> wrote:
> Hi all,
>
> Please review.
>
> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330
>
> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week
> field. It based the calculation on the day-of-week, when it should be based
> on the day-of-month.
>
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/
>
> --
> Thanks and Regards,
>
> Anubhav

From forax at univ-mlv.fr  Wed Oct 12 12:39:29 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 12 Oct 2016 14:39:29 +0200 (CEST)
Subject: RFR(s): 8152617 add missing wildcards to Optional or()
	andflatMap()
In-Reply-To: <57fde766.c89a1c0a.97b0d.6b9c@mx.google.com>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
	<CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
	<d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
	<57fde766.c89a1c0a.97b0d.6b9c@mx.google.com>
Message-ID: <1450399505.1503099.1476275969715.JavaMail.zimbra@u-pem.fr>

Hi Timo,

----- Mail original -----
> De: "timo kinnunen" <timo.kinnunen at gmail.com>
> ?: "Stuart Marks" <stuart.marks at oracle.com>
> Cc: "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoy?: Mercredi 12 Octobre 2016 09:33:54
> Objet: RE: RFR(s): 8152617 add missing wildcards to Optional or()	andflatMap()

> Hi,
> 
> 
> I?m going to challenge the consensus a little bit. First, R?mi's example can be
> simplified to this one method which fails to compile with the same error
> message[0]:
> 
> private static Optional<?> simple1(
> Optional<String> o, Function<String, Optional<?>> f
> ) {return o.flatMap(f);}
> 
> Second, although not in the webrev, Optional::map and Optional::flatMap can be
> implemented without needing any unchecked casts:
> 
> public <U, R extends U, O extends Optional<R>> Optional<U> flatMap(
> Function<T, O> mapper
> ) {
> Objects.requireNonNull(mapper);
> return ofNullable(isPresent() ? mapper.apply(get()).orElse(null) : null);
> }
> public <P, Q extends P> Optional<P> map(
> Function<T, Q> mapper
> ) {
> Objects.requireNonNull(mapper);
> return ofNullable(isPresent() ? mapper.apply(get()) : null);
> }
> 
> These are fully, soundly compile-time typed methods with all types exposed and
> completely under the control of the programmer.

You miss the whole point of wildcards, wildcards are use site variance declarations.
When you have a method that takes a Function<T, R> (with T the type of the parameter type and R the return type) as parameter, you can always send a function that takes a super type and returns a subtype.
A function is contravariant on its parameter types and covariant on the return type.
In a sane language, you should be able to declare something like:

  interface Function<T or a super type of T, R or a subtype of R> {
    public R apply(T t);
  }  

but because of the backward compatibility of the collections, in Java, you do not use variance annotations when you declare Function but at each time when you use it.
So you have to write:
  <U, V> V foo(Optional<U or a subtype of U> optional, Function<U or a superType of U, V or a subtype of V> function) {
    ...
  }

and to make things less readable, instead of using U+ for U or a subtype of U and U- for U and a supertype of U, we use (respectively) ? extends U and ? super U, which can be great when your a beginner because you can read it in english, by example, ? extends U can be read like that: it's a type i don't want to know (?) which is a subtype of U but at the same time because it uses the 'extends' keyword like when you specify the bound of a type variable, you can easily found that using only type variables is more readable, but it's not. A type variable should be used to 'connect' several types in the signature together, like the type of the second parameter is the same as the type of the return type. So fundamentally, a type variable and a wildcard while denote subtyping relationship are two different notations for two different kind of usages.  

> 
> Now, to see if the signature of flatMap is truly the problem, it?s possible to
> write the simple1 method as a map followed by a flatMap with the identity
> function, like this:
> 
> private static Optional<?> simple2(
> Optional<String> o, Function<String, Optional<?>> f
> ) {return o.map(f).flatMap(Function.identity());}
> 
> In this version, the call to flatMap and its argument don?t use any wildcard
> types, both in the version above and in java.util.Optional[1]. Despite that,
> the compiler from jdk1.8.0_102 still gives an error[2] from the flatMap method
> call. This is quite a curious result as here flatMap and identity are reusing
> types that are already used by type inference. If this isn?t sound but using
> wildcards is then I would really like to see that counterexample!
> 
> Some questions that have arisen (and my answers):
> 
> Should APIs account for types that are not denotable in source code? I?d say no,
> such types are bugs in the language.
> 
> Can non-denotable types be eliminated at the library level by adding more
> wildcards? Unlikely, as such types come from using wildcards to begin with.
> 
> Is there a bug in flatMap or is the bug in the language specification? Compared
> to one simple method, type inference rules are much harder to understand and
> thus more likely to contain undiscovered problems.
> 
> What is gained if these wildcards are added? Although simple1 and simple2 would
> compile, you still can?t do anything interesting like calling orElse or
> orElseGet:
> 
> public static void main(String[] args) {
> Optional<String> foo = Optional.ofNullable(args[0]);
> Object o1 = simple1(foo, s -> foo).orElse(o1 = null);
> Object o2 = simple2(foo, s -> foo).orElseGet(() -> null);
> }
> 
> Won?t compile. As far as I can tell, there is no real upside to this change.
> 
> 
> Thanks for your consideration!
> 

R?mi

> 
> 
> 
> [0] Error:(18, 20) java: method flatMap in class java.util.Optional<T> cannot be
> applied to given types;
>  required: java.util.function.Function<? super
>  java.lang.String,java.util.Optional<U>>
>  found: java.util.function.Function<java.lang.String,java.util.Optional<?>>
>  reason: cannot infer type-variable(s) U
>    (argument mismatch;
>    java.util.function.Function<java.lang.String,java.util.Optional<?>> cannot be
>    converted to java.util.function.Function<? super
>    java.lang.String,java.util.Optional<U>>)
> 
> [1] The <? super T> type can be discounted as its presence doesn?t make a
> difference in this case.
> 
> [2] Error:(21, 27) java: method flatMap in class wildcards.another.Optional<T>
> cannot be applied to given types;
>  required: java.util.function.Function<wildcards.another.Optional<?>,O>
>  found: java.util.function.Function<java.lang.Object,java.lang.Object>
>  reason: inference variable O has incompatible bounds
>    equality constraints: wildcards.another.Optional<?>,T
>    upper bounds: wildcards.another.Optional<capture#11 of ?>,java.lang.Object
> 
> 
> --
> Have a nice day,
> Timo
> 
> Sent from Mail for Windows 10
> 
> From: Stuart Marks
> Sent: Saturday, October 8, 2016 02:28
> To: Stefan Zobel
> Cc: core-libs-dev
> Subject: Re: RFR(s): 8152617 add missing wildcards to Optional or() andflatMap()
> 
> 
> 
> On 10/7/16 3:12 PM, Stefan Zobel wrote:
>>> ... After having looked at lots of generic APIs, it seems to me that a
>>> style has emerged where wildcards are used whenever possible, and type
>>> parameters are used only when necessary, ...
>>
>> Yes, I'm familiar with that kind of reasoning (I think Josh Bloch stated
>> that principle in "Effective Java"). But, to be honest, I never thought
>> that it should apply as a strict rule in all circumstances.
> 
> Yep, it's in Effective Java, near the end of Item 28:
> 
>     ?As a rule, if a type parameter appears only once in a method
>     declaration, replace it with a wildcard.?
> 
>> Rhetorical question: do you really think that a signature employing 3
>> wildcards is easier to understand for the proverbial "average Joe" than
>> a bounded type parameter that expresses the sub-type relationship clearly?
>> I do not.
>>
>> But anyway, you probably have to follow the established "style".
>>
>> It's just too bad that most Java programmers I know won't understand the
>> proposed signature of 'flatMap'.
> 
> Turns out that R?mi's example exposes the difference between the wildcard
> approach and the type-parameter approach. Returning to the example,
> 
>     Optional<Integer> oi = Optional.empty();
>     Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
>     Optional<CharSequence> ocs = oi.flatMap(fm);
> 
> If the flatMapper function itself has a wildcard type, for example,
> 
>     Function<Number, Optional<? extends CharSequence>> fm = n -> Optional.empty();
> 
> then this will still work with the wildcard approach but fail with the
> type-parameter approach. As R?mi also pointed out, a wildcarded type can result
> from the capture of a type with a wildcarded type parameter.
> 
> Based on this, I believe the nested wildcard approach to be the correct one.
> 
> s'marks

From jbluettduncan at gmail.com  Wed Oct 12 13:53:16 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Wed, 12 Oct 2016 14:53:16 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<312895EC-5916-491C-AB16-26307CDC362F@oracle.com>
	<95429a179c85cae087303afb969b35bd@reini.net>
	<CAPMys86FH=Nf2T2kBMxAFX38MDyHqYsA0ZoDTCy21XZR3ccv=w@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
	<CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>
Message-ID: <CAPMys87m5ZM+DPeVvkAvnzntotO+088rZFMj55EtzUqvE=kJfw@mail.gmail.com>

Hi all,

Thank you very much for all reviewing my changeset over the last few days
and for finding the bits I forgot to transfer from webrev01 to webrev02.
I've been quiet as I'm still busy with university and will be for the
foreseeable future.

Stuart, many thanks again for sponsoring the change and going through the
whole procedure for me. I look forward to the rest of your results.

Kind regards,
Jonathan

From anubhav.meena at oracle.com  Wed Oct 12 15:03:35 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Wed, 12 Oct 2016 20:33:35 +0530
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
Message-ID: <99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>

Hi Stephen,

Have incorporated the changes you suggested. Updated webrev is available here 
http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/

Please review and suggest if anymore changes are required.

Thanks,
Anubhav

> On Oct 12, 2016, at 3:21 PM, Anubhav Meena <anubhav.meena at oracle.com> wrote:
> 
> Gentle reminder.
> 
>> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>> 
>> Hi all,
>> 
>> Please review.
>> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
>> 
>> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
>> 
>> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>
>> -- 
>> Thanks and Regards,
>> Anubhav
> 


From Roger.Riggs at Oracle.com  Wed Oct 12 15:15:37 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 12 Oct 2016 11:15:37 -0400
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
	<99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
Message-ID: <a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>

Hi Anubhav,

In general, I agree with Stephen that the tests should be testing an 
algorithm against facts.
Embedding an algorithm in the test increases the risk that the test will 
just replicate the
implementation code and therefor not be much of a test.

Though in this case, the specification of aligned day of week is of a 
computation.
If the test were to independently compute the correct answer, it would 
be valid as a 'tck' test.

Since the Hijrah calendar is data driven, the tests should be in 
/java/time/*test*/java/time/chrono/TestUmmAlQuraChronology.java.
Tests in java/time/tck/... should correspond directly to specified 
behaviors.
In this case, the algorithm is specified but the test is data dependent. 
(Perhaps a gray area).

Thanks, Roger


On 10/12/2016 11:03 AM, Anubhav Meena wrote:
> Hi Stephen,
>
> Have incorporated the changes you suggested. Updated webrev is 
> available here
> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/ 
> <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.03/>
>
> Please review and suggest if anymore changes are required.
>
> Thanks,
> Anubhav
>
>> On Oct 12, 2016, at 3:21 PM, Anubhav Meena <anubhav.meena at oracle.com 
>> <mailto:anubhav.meena at oracle.com>> wrote:
>>
>> Gentle reminder.
>>
>>> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com 
>>> <mailto:anubhav.meena at oracle.com>> wrote:
>>>
>>> Hi all,
>>>
>>> Please review.
>>> Bug Id :https://bugs.openjdk.java.net/browse/JDK-8163330
>>>
>>> Issue:The HijrahDate class incorrectly calculates the aligned-day-of-week 
>>> field. It based the calculation on the day-of-week, when it should 
>>> be based on the day-of-month.
>>>
>>> Webrev:http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ 
>>> <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.02/>
>>> -- 
>>> Thanks and Regards,
>>> Anubhav
>>
>


From Roger.Riggs at Oracle.com  Wed Oct 12 15:50:12 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 12 Oct 2016 11:50:12 -0400
Subject: [9] RFR of JDK-8085192: java/rmi/activation/Activatable tests
	fail intermittently due to "Port already in use"
In-Reply-To: <745be6ee-810a-4d2e-76b9-c0df79e6d6db@oracle.com>
References: <3c0bdf63-f55c-5b93-6d74-58e26916cb3b@oracle.com>
	<f881b76f-2a76-5abd-3118-7f6fe12d9205@Oracle.com>
	<e3b084d3-de5e-700f-2f69-22f91a0f80bf@oracle.com>
	<02147682-541f-9746-b6da-2cc2800165b5@Oracle.com>
	<3E39A1E9-0764-404D-9824-85CFB75113FF@oracle.com>
	<E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
	<02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>
	<2e13f9fa-cfeb-94bb-0ed5-6319c32b3e1f@Oracle.com>
	<745be6ee-810a-4d2e-76b9-c0df79e6d6db@oracle.com>
Message-ID: <75410cc2-45c4-f5e7-46b0-4f254e1c3702@Oracle.com>

Hi Chris,


On 10/10/2016 9:43 AM, Chris Hegarty wrote:
> Roger,
>
> I addressed all, or most, of your comments in the following
> webrev.
>
>  1) Refactored out the use of sun.nio.ch in the test library,
>    so that a reduced number of tests need their @modules tag
>    updated. ( @modules support with test library usage it a
>    pain )
It would be ok to put the modules java.base/sun.nio.ch dependency in the 
test/java/rmi/TEST.properties.
Or perhaps in a new test/java/rmi/activation/TEST.properties

(I would probably move many of the @modules in individual tests there.
Though it is cleaner to have only the ones that are really needed by 
each test.)

Can the @build dependencies also go in TEST.properties using lib.build =

>
>  2) Use Boolean.getBoolean to retrieve the new property
tnx
>
>  3) fix typos and remove stray debugging statements
>
>   http://cr.openjdk.java.net/~chegar/8085192_webrev.03/

test/java/rmi/testlibrary/RMID.java:
  line 60:  ephemeralPort could be removed; (unused)

  line 106: the string 
"java.nio.channels.spi.SelectorProvider=RMIDSelectorProvider" should be
     a static and used the JavaVM.  (similar to RMID.EPHEMERAL_MSG)

>
>> ...
>> I'm vaguely not very comfortable with scraping the port number off 
>> stdout
>> and the inherited channel pieces seem like a lot of moving parts.
>
> Right, I was a little uneasy with this too, to being with, but
> it has grown on me ( since it appears stable and reliable in
> all my builds and tests ). Also the surface area of the code
> change is very small, and the inherit channel mechanism is well
> specified and stable.

ok, lets give it a try.

Thanks, Roger

>
>> Roger
>>
>> p.s.  Anyother idea
>> I assume not all platforms can allow separate processes to open server
>> sockets to the same port.
>> If so, we would just have the client allocate a port (0), mark it
>> non-exclusive and keep it open
>> while passing the port number to RMID. Only after RMID is started close
>> the allocating socket.
>
> I believe Stuart did look at this in some detail a while back [1], and
> it was somewhat dismissed because of the lack of cross platform support
> for SO_REUSEPORT. Maybe things have move on, but I don't think so?
>
> The use of inherit channel is somewhat akin to loading an agent into
> the target, but more straightforward.
>
> What do others think, that will have to maintain these tests? I don't
> want to make them every harder to maintain. Hamlin's approach is still
> on the table too.
>
> -Chris.
>
> [1] 
> http://mail.openjdk.java.net/pipermail/serviceability-dev/2014-December/016251.html


From Roger.Riggs at Oracle.com  Wed Oct 12 15:57:19 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 12 Oct 2016 11:57:19 -0400
Subject: DateTimeFormatter.format() uses exceptions for flow control
In-Reply-To: <1476082430.5712.6.camel@unportant.info>
References: <1476026665.8959.9.camel@unportant.info>
	<49e6b2e4-940a-3f76-b88d-51e845e5ec6d@oracle.com>
	<1476082430.5712.6.camel@unportant.info>
Message-ID: <444e7e56-8d6b-dd3b-9c80-c016916b2203@Oracle.com>

Created an issue [1] and included the patch.

Thanks, Roger

[1] https://bugs.openjdk.java.net/browse/JDK-8167618


On 10/10/2016 2:53 AM, Cl?ment MATHIEU wrote:
> On Mon, 2016-10-10 at 06:47 +1000, David Holmes wrote:
>
> Hi David,
>
>> Please note that patches can only be accepted if they are sent
>> through, or hosted upon OpenJDK infrastructure. If the patch is small
>> enough can you send it inline in the email (attachments are often
>> stripped)
> Here it is:
>
> --- old/src/java.base/share/classes/java/time/format/DateTimePrintContext.java	2016-10-09 17:01:30.326739656 +0200
> +++ new/src/java.base/share/classes/java/time/format/DateTimePrintContext.java	2016-10-09 17:01:30.228738595 +0200
> @@ -302,13 +302,10 @@
>        * @throws DateTimeException if the field is not available and the section is not optional
>        */
>       Long getValue(TemporalField field) {
> -        try {
> +        if (optional == 0) {
>               return temporal.getLong(field);
> -        } catch (DateTimeException ex) {
> -            if (optional > 0) {
> -                return null;
> -            }
> -            throw ex;
> +        } else {
> +            return temporal.isSupported(field) ? temporal.getLong(field) : null;
>           }
>       }
>
> Cl?ment MATHIEU


From mandy.chung at oracle.com  Wed Oct 12 16:10:24 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 12 Oct 2016 09:10:24 -0700
Subject: Review Request: JDK-8164689: Retrofit jar, jlink,
	jmod as a ToolProvider
In-Reply-To: <9f125788-9866-78d9-8ccf-e06d0755426e@oracle.com>
References: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
	<9f125788-9866-78d9-8ccf-e06d0755426e@oracle.com>
Message-ID: <3FA9F82F-A937-4B76-AC23-E1E6D105ACD0@oracle.com>


> On Oct 11, 2016, at 11:07 PM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 11/10/2016 19:03, Mandy Chung wrote:
> 
>> This patch updates jar, jlink, jmod tool to be a provider of the new tool SPI.  Some tests are also updated to replace the use of internal APIs with ToolProvider::findFirst to look up a tool provider.  There are more tests that can be updated and something to be cleaned up in the future.
>> 
>> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.00/index.html
>> 
> This update to the tools looks good. For the tests then ToolProvider.findFirst("jar").get() will draw the attention of the optional police and might be better to use orElseThrow to supply an exception that clearly indicates that the tool cannot be found.


Good suggestion.  Will change that before pushing.

Thanks
Mandy

From anubhav.meena at oracle.com  Wed Oct 12 18:16:10 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Wed, 12 Oct 2016 23:46:10 +0530
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
	<99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
	<a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>
Message-ID: <F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>

Hi Roger,

Thanks for the suggestion, have made the suggested changes and shifted the tests to /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java. 

Updated webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.04/

Thanks,
Anubhav

> On Oct 12, 2016, at 8:45 PM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Hi Anubhav,
> 
> In general, I agree with Stephen that the tests should be testing an algorithm against facts.
> Embedding an algorithm in the test increases the risk that the test will just replicate the 
> implementation code and therefor not be much of a test.
> 
> Though in this case, the specification of aligned day of week is of a computation.
> If the test were to independently compute the correct answer, it would be valid as a 'tck' test.
> 
> Since the Hijrah calendar is data driven, the tests should be in /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java.
> Tests in java/time/tck/... should correspond directly to specified behaviors.
> In this case, the algorithm is specified but the test is data dependent. (Perhaps a gray area).
> 
> Thanks, Roger
> 
> 
> On 10/12/2016 11:03 AM, Anubhav Meena wrote:
>> Hi Stephen,
>> 
>> Have incorporated the changes you suggested. Updated webrev is available here 
>> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/ <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.03/>
>> 
>> Please review and suggest if anymore changes are required.
>> 
>> Thanks,
>> Anubhav
>> 
>>> On Oct 12, 2016, at 3:21 PM, Anubhav Meena <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>> 
>>> Gentle reminder.
>>> 
>>>> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> Please review.
>>>> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
>>>> 
>>>> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
>>>> 
>>>> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.02/>
>>>> -- 
>>>> Thanks and Regards,
>>>> Anubhav
>>> 
>> 
> 


From brent.christian at oracle.com  Wed Oct 12 18:31:52 2016
From: brent.christian at oracle.com (Brent Christian)
Date: Wed, 12 Oct 2016 11:31:52 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <0075c1cd-9eba-8ea4-4cd4-1de546d7723b@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
	<0075c1cd-9eba-8ea4-4cd4-1de546d7723b@oracle.com>
Message-ID: <8193e69f-cb17-aa8c-1201-7057df564975@oracle.com>

On 10/11/16 8:25 AM, Alan Bateman wrote:
 >
> One thing that would be good is to beef up
> the test to cover more scenarios, esp. loader L1 extends loader L2 where
> you've got 4 combination of capable/non-capable to test.

I updated the test case to provide better coverage:

http://cr.openjdk.java.net/~bchristi/8165793/webrev.01/

Thanks,
-Brent

From mandy.chung at oracle.com  Wed Oct 12 19:20:29 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 12 Oct 2016 12:20:29 -0700
Subject: Review Request: JDK-8164689: Retrofit jar, jlink,
	jmod as a ToolProvider
In-Reply-To: <3FA9F82F-A937-4B76-AC23-E1E6D105ACD0@oracle.com>
References: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
	<9f125788-9866-78d9-8ccf-e06d0755426e@oracle.com>
	<3FA9F82F-A937-4B76-AC23-E1E6D105ACD0@oracle.com>
Message-ID: <227D1AC3-C613-40B0-BCC3-3ED6414CD613@oracle.com>

Updated webrev:
   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.01

Mandy

> On Oct 12, 2016, at 9:10 AM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> 
>> On Oct 11, 2016, at 11:07 PM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
>> 
>> On 11/10/2016 19:03, Mandy Chung wrote:
>> 
>>> This patch updates jar, jlink, jmod tool to be a provider of the new tool SPI.  Some tests are also updated to replace the use of internal APIs with ToolProvider::findFirst to look up a tool provider.  There are more tests that can be updated and something to be cleaned up in the future.
>>> 
>>> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.00/index.html
>>> 
>> This update to the tools looks good. For the tests then ToolProvider.findFirst("jar").get() will draw the attention of the optional police and might be better to use orElseThrow to supply an exception that clearly indicates that the tool cannot be found.
> 
> 
> Good suggestion.  Will change that before pushing.
> 
> Thanks
> Mandy


From mandy.chung at oracle.com  Wed Oct 12 19:24:22 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 12 Oct 2016 12:24:22 -0700
Subject: RFR 8165793 : Provide an API to query if a ClassLoader is
	parallel capable
In-Reply-To: <8193e69f-cb17-aa8c-1201-7057df564975@oracle.com>
References: <ba123024-016d-bb13-d5ab-8c37ca356509@oracle.com>
	<48AD8602-EF67-4B21-AF9A-E23C34D5550F@oracle.com>
	<8cb07fae-19cb-d73e-1d61-42aa46e0c9c2@oracle.com>
	<0075c1cd-9eba-8ea4-4cd4-1de546d7723b@oracle.com>
	<8193e69f-cb17-aa8c-1201-7057df564975@oracle.com>
Message-ID: <3C4A8F09-6FE7-40BB-A5A0-C72040A1D446@oracle.com>


> On Oct 12, 2016, at 11:31 AM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> On 10/11/16 8:25 AM, Alan Bateman wrote:
> >
>> One thing that would be good is to beef up
>> the test to cover more scenarios, esp. loader L1 extends loader L2 where
>> you've got 4 combination of capable/non-capable to test.
> 
> I updated the test case to provide better coverage:
> 
> http://cr.openjdk.java.net/~bchristi/8165793/webrev.01/
> 

+1.

Formatting nit: you could break .forEach to the next line

  96 ParaSubCL.class).forEach(IsParallelCapable::testClassLoaderClass);

Mandy

From Alan.Bateman at oracle.com  Wed Oct 12 19:53:14 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 12 Oct 2016 20:53:14 +0100
Subject: Review Request: JDK-8164689: Retrofit jar, jlink, jmod as a
	ToolProvider
In-Reply-To: <227D1AC3-C613-40B0-BCC3-3ED6414CD613@oracle.com>
References: <7FA9409E-BFE8-4813-8757-1F67FA3D578B@oracle.com>
	<9f125788-9866-78d9-8ccf-e06d0755426e@oracle.com>
	<3FA9F82F-A937-4B76-AC23-E1E6D105ACD0@oracle.com>
	<227D1AC3-C613-40B0-BCC3-3ED6414CD613@oracle.com>
Message-ID: <b849fbeb-6f41-d47f-6dd5-c94900cb9ba7@oracle.com>



On 12/10/2016 20:20, Mandy Chung wrote:
> Updated webrev:
>     http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8164689/webrev.01
>
> Mandy
>
This looks good.

-Alan

From huizhe.wang at oracle.com  Wed Oct 12 20:37:22 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 12 Oct 2016 13:37:22 -0700
Subject: RFR (JAXP) 8058152: JDK accepts XSLT stylesheet having import element
	erroneously placed
Message-ID: <57FE9F02.206@oracle.com>

Hi,

Please review a change to validate that xsl:import element children 
/must precede/ all other element children of an <xsl:stylesheet> 
element, including any </xsl/:/include/> element children as required by 
the specification [1].

[1] https://www.w3.org/TR/xslt#import

JBS: https://bugs.openjdk.java.net/browse/JDK-8058152
webrev: http://cr.openjdk.java.net/~joehw/jdk9/8058152/webrev/

Thanks,
Joe

From stuart.marks at oracle.com  Wed Oct 12 20:39:14 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Wed, 12 Oct 2016 13:39:14 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <6f3b07f2-569c-fca8-3730-e769d5e4a445@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<8c206fe5-6caa-bc5b-c073-dcc38fb3d0a0@oracle.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
	<6f3b07f2-569c-fca8-3730-e769d5e4a445@oracle.com>
Message-ID: <2335b3e2-d4d7-cd17-eca9-b8851127beb2@oracle.com>

Hi Naoto,

Great, thanks for confirming this.

s'marks

On 10/11/16 4:13 PM, Naoto Sato wrote:
> +1 for removing the link to Collections#unmodifiableList in the spec. I don't
> see any particular reason to specify in the javadoc and don't believe it would
> break any existing apps.
>
> Naoto
>
> On 10/11/16 4:03 PM, Stuart Marks wrote:
>>
>>
>> On 10/10/16 11:26 PM, Andrej Golovnin wrote:
>>> src/java.base/share/classes/java/util/ResourceBundle.java
>>>
>>> 2490     public static class Control {
>>> 2491         /**
>>> 2492          * The default format <code>List</code>, which contains
>>> the strings
>>> 2493          * <code>"java.class"</code> and
>>> <code>"java.properties"</code>, in
>>> 2494          * this order. This <code>List</code> is {@linkplain
>>> 2495          * Collections#unmodifiableList(List) unmodifiable}.
>>> 2496          *
>>> 2497          * @see #getFormats(String)
>>> 2498          */
>>> 2499         public static final List<String> FORMAT_DEFAULT =
>>> List.of("java.class", "java.properties");
>>>
>>> I think you should also change the JavaDocs in the
>>> ResourceBundle.Control class for the constants FORMAT_CLASS,
>>> FORMAT_DEFAULT and FORMAT_PROPERTIES, because the JavaDocs for this
>>> constants explicitly mentions, that the lists are created by using
>>> Collections#unmodifiableList(List). Or you cannot change this
>>> constants at all because they are part of the Public API and existed
>>> in this form for a long time. Please ask someone from Oracle for help.
>>> They can explain it better when it is OK to change and when not. Maybe
>>> Stuart can do that.
>>
>> Hi Andrej,
>>
>> Thanks for pointing this out.
>>
>> It appears that the changes to remove the links to
>> Collections.unmodifiableList() was dropped from webrev.01 to webrev.02.
>> I've restored them, along with a couple other changes that were also
>> dropped. I also updated the ModuleFinder.java comment per a request from
>> Alan Bateman. The revised webrev is here:
>>
>> http://cr.openjdk.java.net/~smarks/reviews/8134373/webrev.03/
>>
>> In any case, yes, the specifications of the ResourceBundle.Control
>> fields should be changed to remove the links to
>> Collections.unmodifiableList(). It's unclear whether having a link this
>> way implies that it's part of the specification that these fields must
>> be instances returned from that method. Removing the link makes it clear
>> that saying the List is unmodifiable is merely a description of the
>> required behavior.
>>
>> I spoke with Joe Darcy (our compatibility guru) and we agreed that out
>> of an abundance of caution it would be wise to file a request to make
>> this change. (This is the "CCC" - basically an internal change control
>> process for Java SE specifications.) Doing this is mainly for tracking
>> purposes, as we believe this to be a compatible change.
>>
>> I've also included in this request a mention of the change to
>> CookieManager.get() which we had discussed previously. Even though we
>> believe this is also a compatible change, it's also a change that should
>> be tracked.
>>
>> I'll follow up when this bit of process is finished.
>>
>> s'marks

From stuart.marks at oracle.com  Wed Oct 12 20:41:12 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Wed, 12 Oct 2016 13:41:12 -0700
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys87m5ZM+DPeVvkAvnzntotO+088rZFMj55EtzUqvE=kJfw@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
	<CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>
	<CAPMys87m5ZM+DPeVvkAvnzntotO+088rZFMj55EtzUqvE=kJfw@mail.gmail.com>
Message-ID: <8db6b9fb-d690-ce20-e011-861274666d70@oracle.com>

Tests passed, spec change approved, changeset pushed:

     http://hg.openjdk.java.net/jdk9/dev/jdk/rev/af71f6a36731

Jonathan, thanks for your contribution. And Patrick, thanks again for hosting 
the webrev.

s'marks


On 10/12/16 6:53 AM, Jonathan Bluett-Duncan wrote:
> Hi all,
>
> Thank you very much for all reviewing my changeset over the last few days and 
> for finding the bits I forgot to transfer from webrev01 to webrev02. I've been 
> quiet as I'm still busy with university and will be for the foreseeable future.
>
> Stuart, many thanks again for sponsoring the change and going through the 
> whole procedure for me. I look forward to the rest of your results.
>
> Kind regards,
> Jonathan
>


From patrick at reini.net  Wed Oct 12 20:49:26 2016
From: patrick at reini.net (Patrick Reinhart)
Date: Wed, 12 Oct 2016 22:49:26 +0200
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <8db6b9fb-d690-ce20-e011-861274666d70@oracle.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
	<CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>
	<CAPMys87m5ZM+DPeVvkAvnzntotO+088rZFMj55EtzUqvE=kJfw@mail.gmail.com>
	<8db6b9fb-d690-ce20-e011-...
Message-ID: <C96ECAC6-7F84-458A-80A5-EB8E1C9FEEC7@reini.net>

You?re welcome :-)

-Patrick

> Am 12.10.2016 um 22:41 schrieb Stuart Marks <stuart.marks at oracle.com>:
> 
> Tests passed, spec change approved, changeset pushed:
> 
>     http://hg.openjdk.java.net/jdk9/dev/jdk/rev/af71f6a36731 <http://hg.openjdk.java.net/jdk9/dev/jdk/rev/af71f6a36731>
> Jonathan, thanks for your contribution. And Patrick, thanks again for hosting the webrev.
> 
> s'marks
> 
> On 10/12/16 6:53 AM, Jonathan Bluett-Duncan wrote:
>> Hi all,
>> 
>> Thank you very much for all reviewing my changeset over the last few days and for finding the bits I forgot to transfer from webrev01 to webrev02. I've been quiet as I'm still busy with university and will be for the foreseeable future.
>> 
>> Stuart, many thanks again for sponsoring the change and going through the whole procedure for me. I look forward to the rest of your results.
>> 
>> Kind regards,
>> Jonathan
>> 
> 


From lance.andersen at oracle.com  Wed Oct 12 21:26:38 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Wed, 12 Oct 2016 17:26:38 -0400
Subject: RFR (JAXP) 8058152: JDK accepts XSLT stylesheet having import
	element erroneously placed
In-Reply-To: <57FE9F02.206@oracle.com>
References: <57FE9F02.206@oracle.com>
Message-ID: <38EA3AFC-09D8-47CC-BDDB-B346AFB65A53@oracle.com>

Looks OK Joe

> On Oct 12, 2016, at 4:37 PM, Joe Wang <huizhe.wang at oracle.com> wrote:
> 
> Hi,
> 
> Please review a change to validate that xsl:import element children /must precede/ all other element children of an <xsl:stylesheet> element, including any </xsl/:/include/> element children as required by the specification [1].
> 
> [1] https://www.w3.org/TR/xslt#import
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8058152
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8058152/webrev/
> 
> Thanks,
> Joe

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From Roger.Riggs at Oracle.com  Wed Oct 12 21:39:55 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 12 Oct 2016 17:39:55 -0400
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
	<99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
	<a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>
	<F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>
Message-ID: <bc42fe3c-1b04-84bc-34d4-66fb9f28990b@Oracle.com>

Hi,

Looks ok.

Typically, there is a space after the '//' comment characters, it makes 
it easier to read.
Also, in this case, I don't think the comments help.  "Monday" isn't 
important to the test
and the 'Any Other day' is still 1, not something else.

I would remove those comments.
Otherwise fine, no need for another webrev.

Thanks, Roger


On 10/12/2016 2:16 PM, Anubhav Meena wrote:
> Hi Roger,
>
> Thanks for the suggestion, have made the suggested changes and shifted 
> the tests to 
> /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java.
>
> Updated webrev: 
> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.04/ 
> <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.04/>
>
> Thanks,
> Anubhav
>
>> On Oct 12, 2016, at 8:45 PM, Roger Riggs <Roger.Riggs at Oracle.com 
>> <mailto:Roger.Riggs at oracle.com>> wrote:
>>
>> Hi Anubhav,
>>
>> In general, I agree with Stephen that the tests should be testing an 
>> algorithm against facts.
>> Embedding an algorithm in the test increases the risk that the test 
>> will just replicate the
>> implementation code and therefor not be much of a test.
>>
>> Though in this case, the specification of aligned day of week is of a 
>> computation.
>> If the test were to independently compute the correct answer, it 
>> would be valid as a 'tck' test.
>>
>> Since the Hijrah calendar is data driven, the tests should be in 
>> /java/time/*test*/java/time/chrono/TestUmmAlQuraChronology.java.
>> Tests in java/time/tck/... should correspond directly to specified 
>> behaviors.
>> In this case, the algorithm is specified but the test is data 
>> dependent. (Perhaps a gray area).
>>
>> Thanks, Roger
>>
>>
>> On 10/12/2016 11:03 AM, Anubhav Meena wrote:
>>> Hi Stephen,
>>>
>>> Have incorporated the changes you suggested. Updated webrev is 
>>> available here
>>> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/ 
>>> <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.03/>
>>>
>>> Please review and suggest if anymore changes are required.
>>>
>>> Thanks,
>>> Anubhav
>>>
>>>> On Oct 12, 2016, at 3:21 PM, Anubhav Meena 
>>>> <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>>>
>>>> Gentle reminder.
>>>>
>>>>> On Oct 7, 2016, at 2:12 PM, Anubhav Meena 
>>>>> <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Please review.
>>>>> Bug Id :https://bugs.openjdk.java.net/browse/JDK-8163330
>>>>>
>>>>> Issue:The HijrahDate class incorrectly calculates the 
>>>>> aligned-day-of-week field. It based the calculation on the 
>>>>> day-of-week, when it should be based on the day-of-month.
>>>>>
>>>>> Webrev:http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ 
>>>>> <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.02/>
>>>>> -- 
>>>>> Thanks and Regards,
>>>>> Anubhav
>>>>
>>>
>>
>


From naoto.sato at oracle.com  Wed Oct 12 21:50:58 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Wed, 12 Oct 2016 14:50:58 -0700
Subject: RFR (JAXP) 8058152: JDK accepts XSLT stylesheet having import
	element erroneously placed
In-Reply-To: <57FE9F02.206@oracle.com>
References: <57FE9F02.206@oracle.com>
Message-ID: <c7fea3c5-3144-5b04-03cc-e1f3df291c4f@oracle.com>

+1. You might want to append "_ERR" to the field "IMPORT_PRECEDE_OTHERS" 
to align with other errors (no need for further review).

Naoto

On 10/12/16 1:37 PM, Joe Wang wrote:
> Hi,
>
> Please review a change to validate that xsl:import element children
> /must precede/ all other element children of an <xsl:stylesheet>
> element, including any </xsl/:/include/> element children as required by
> the specification [1].
>
> [1] https://www.w3.org/TR/xslt#import
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8058152
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8058152/webrev/
>
> Thanks,
> Joe

From huizhe.wang at oracle.com  Wed Oct 12 22:11:05 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 12 Oct 2016 15:11:05 -0700
Subject: RFR (JAXP) 8058152: JDK accepts XSLT stylesheet having import
	element erroneously placed
In-Reply-To: <c7fea3c5-3144-5b04-03cc-e1f3df291c4f@oracle.com>
References: <57FE9F02.206@oracle.com>
	<c7fea3c5-3144-5b04-03cc-e1f3df291c4f@oracle.com>
Message-ID: <57FEB4F9.3060402@oracle.com>

Thanks Naoto, Lance!

I appended _ERR.

-Joe

On 10/12/16, 2:50 PM, Naoto Sato wrote:
> +1. You might want to append "_ERR" to the field 
> "IMPORT_PRECEDE_OTHERS" to align with other errors (no need for 
> further review).
>
> Naoto
>
> On 10/12/16 1:37 PM, Joe Wang wrote:
>> Hi,
>>
>> Please review a change to validate that xsl:import element children
>> /must precede/ all other element children of an <xsl:stylesheet>
>> element, including any </xsl/:/include/> element children as required by
>> the specification [1].
>>
>> [1] https://www.w3.org/TR/xslt#import
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8058152
>> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8058152/webrev/
>>
>> Thanks,
>> Joe

From xueming.shen at oracle.com  Wed Oct 12 22:50:23 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Wed, 12 Oct 2016 15:50:23 -0700
Subject: RFR JDK-8166258: Unexpected code conversion by HKSCS converters
Message-ID: <57FEBE2F.6050503@oracle.com>

Hi

Please help review the change for #8166258

issue: https://bugs.openjdk.java.net/browse/JDK-8166258
webrev: http://cr.openjdk.java.net/~sherman/8166258/webrev/

This is an overlook in the code that generates the c2b (encoding) mapping
table from the b2c data. \ufffd is the special code point that used to 
indicate
"no mapping" from a native code point to unicode code point. It's wrong to
try to generate a c2b mapping entry from \ufffd.

Thanks,
Sherman

From mandy.chung at oracle.com  Wed Oct 12 22:52:56 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 12 Oct 2016 15:52:56 -0700
Subject: Review request: JDK-8167630 jdeps --generate-module-info forgets to
	close the resource after checking any unnamed package
Message-ID: <FE9D0163-90EF-41E5-82E0-BED006604959@oracle.com>

Simple patch close the ClassFileReader with try-with-resource. 


diff --git a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
--- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
+++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
@@ -680,9 +680,9 @@
     private boolean genModuleInfo(JdepsConfiguration config) throws IOException {
         // check if any JAR file contains unnamed package
         for (String arg : inputArgs) {
+            try (ClassFileReader reader = ClassFileReader.newInstance(Paths.get(arg))) {
             Optional<String> classInUnnamedPackage =
-                ClassFileReader.newInstance(Paths.get(arg))
-                    .entries().stream()
+                    reader.entries().stream()
                     .filter(n -> n.endsWith(".class"))
                     .filter(cn -> toPackageName(cn).isEmpty())
                     .findFirst();
@@ -696,6 +696,7 @@
                 return false;
             }
         }
+        }
 
         ModuleInfoBuilder builder
             = new ModuleInfoBuilder(config, inputArgs, options.genModuleInfo);

Thanks
Mandy

From lance.andersen at oracle.com  Wed Oct 12 22:54:11 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Wed, 12 Oct 2016 18:54:11 -0400
Subject: Review request: JDK-8167630 jdeps --generate-module-info forgets
	to close the resource after checking any unnamed package
In-Reply-To: <FE9D0163-90EF-41E5-82E0-BED006604959@oracle.com>
References: <FE9D0163-90EF-41E5-82E0-BED006604959@oracle.com>
Message-ID: <6B57D497-E01A-4C9F-9635-22B94EDD63C8@oracle.com>

+1
> On Oct 12, 2016, at 6:52 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Simple patch close the ClassFileReader with try-with-resource. 
> 
> 
> diff --git a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> @@ -680,9 +680,9 @@
>     private boolean genModuleInfo(JdepsConfiguration config) throws IOException {
>         // check if any JAR file contains unnamed package
>         for (String arg : inputArgs) {
> +            try (ClassFileReader reader = ClassFileReader.newInstance(Paths.get(arg))) {
>             Optional<String> classInUnnamedPackage =
> -                ClassFileReader.newInstance(Paths.get(arg))
> -                    .entries().stream()
> +                    reader.entries().stream()
>                     .filter(n -> n.endsWith(".class"))
>                     .filter(cn -> toPackageName(cn).isEmpty())
>                     .findFirst();
> @@ -696,6 +696,7 @@
>                 return false;
>             }
>         }
> +        }
> 
>         ModuleInfoBuilder builder
>             = new ModuleInfoBuilder(config, inputArgs, options.genModuleInfo);
> 
> Thanks
> Mandy

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From jbluettduncan at gmail.com  Wed Oct 12 22:59:35 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Wed, 12 Oct 2016 23:59:35 +0100
Subject: RFR: JDK-8134373: explore potential uses of convenience factories
	within the JDK
In-Reply-To: <CAPMys85RoCmGkwe1bc3Wtkv6HN5KNP2S_AsrL3RDhEqJT_waKw@mail.gmail.com>
References: <CAPMys85yi4kTFBftKctHeh0EqNTq1XmkYi8pCewKctk3u03C-Q@mail.gmail.com>
	<CAPMys87hMkqYNmtweK1h3p1h3tEzsgBBdZt1YWap3jTYaJq=_g@mail.gmail.com>
	<d496a2ca-33a0-f0f6-27d3-3dec6ce22839@oracle.com>
	<CAPMys87T2-JfYz8w+VUu86ciQDU-h2es0H+R5PeARB4j9bbu3g@mail.gmail.com>
	<a4100cf6-a192-12e5-8d8f-935411cf45c4@oracle.com>
	<CACzrW9Db60ti9rx+Tyyir24XAsX+rGzrRR6RSW_cZAHskBFP+g@mail.gmail.com>
	<CAPMys86KhWkrZQu4410FVwPouPcdhJ3jKVnheo_VMy_7=9OSfg@mail.gmail.com>
	<CACzrW9CbL+rYTSj_YpxxGY_YDrecaj7CAGJ1zbSxrgycKoxgwA@mail.gmail.com>
	<CAPMys871O6iGikp8cGzpEQK5vnYzfLMcxdnUjQOXb-ADJfbmYA@mail.gmail.com>
	<AD0DE602-CFC8-4507-8797-49D549095073@reini.net>
	<CAPMys869h=oTLz1RL8AeyncK6gE6Gz9Mh5jgfrLd0xXDtTsMPQ@mail.gmail.com>
	<CAM6ZMz_5bURHJzcvQ5pu04G8KswxTfGH=zcJX39VN6qF21+iyA@mail.gmail.com>
	<15deca88-8784-2c2e-bcf3-930b23071e47@oracle.com>
	<CAM6ZMz-HSSHTPyhpqUp5T7_puhpDNJD-rMgOFUW1GWr3iehKgw@mail.gmail.com>
	<CAPMys87m5ZM+DPeVvkAvnzntotO+088rZFMj55EtzUqvE=kJfw@mail.gmail.com>
	<8db6b9fb-d690-ce20-e011-861274666d70@oracle.com>
	<C96ECAC6-7F84-458A-80A5-EB8E1C9FEEC7@reini.net>
	<CAPMys86dq72PEQm+7oWguJp2up6GFFerm_5gNQSAYgt8Riz9-g@mail.gmail.com>
	<CAPMys85RoCmGkwe1bc3Wtkv6HN5KNP2S_AsrL3RDhEqJT_waKw@mail.gmail.com>
Message-ID: <CAPMys86RL3Za30KCZu16J5gBtNSMbHa63jkPdN6MmW2Q_sf5FQ@mail.gmail.com>

Yes, you're very welcome Stuart. :-)

Kind regards,
Jonathan

On 12 Oct 2016 21:49, "Patrick Reinhart" <patrick at reini.net> wrote:

You?re welcome :-)

-Patrick

Am 12.10.2016 um 22:41 schrieb Stuart Marks <stuart.marks at oracle.com>:

Tests passed, spec change approved, changeset pushed:

    http://hg.openjdk.java.net/jdk9/dev/jdk/rev/af71f6a36731

Jonathan, thanks for your contribution. And Patrick, thanks again for
hosting the webrev.

s'marks

On 10/12/16 6:53 AM, Jonathan Bluett-Duncan wrote:

Hi all,

Thank you very much for all reviewing my changeset over the last few days
and for finding the bits I forgot to transfer from webrev01 to webrev02.
I've been quiet as I'm still busy with university and will be for the
foreseeable future.

Stuart, many thanks again for sponsoring the change and going through the
whole procedure for me. I look forward to the rest of your results.

Kind regards,
Jonathan

From jbluettduncan at gmail.com  Wed Oct 12 23:03:52 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Thu, 13 Oct 2016 00:03:52 +0100
Subject: Review request: JDK-8167630 jdeps --generate-module-info forgets
	to close the resource after checking any unnamed package
In-Reply-To: <6B57D497-E01A-4C9F-9635-22B94EDD63C8@oracle.com>
References: <FE9D0163-90EF-41E5-82E0-BED006604959@oracle.com>
	<6B57D497-E01A-4C9F-9635-22B94EDD63C8@oracle.com>
Message-ID: <CAPMys86u2ga2J0Br9WA_psa69F8v+vogytb+KMA9RERJPLHkxA@mail.gmail.com>

Not a reviewer, but looks good to me. :-)

Kind regards,
Jonathan

On 12 Oct 2016 23:54, "Lance Andersen" <lance.andersen at oracle.com> wrote:

> +1
> > On Oct 12, 2016, at 6:52 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> >
> > Simple patch close the ClassFileReader with try-with-resource.
> >
> >
> > diff --git a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> > --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> > +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java
> > @@ -680,9 +680,9 @@
> >     private boolean genModuleInfo(JdepsConfiguration config) throws
> IOException {
> >         // check if any JAR file contains unnamed package
> >         for (String arg : inputArgs) {
> > +            try (ClassFileReader reader = ClassFileReader.newInstance(Paths.get(arg)))
> {
> >             Optional<String> classInUnnamedPackage =
> > -                ClassFileReader.newInstance(Paths.get(arg))
> > -                    .entries().stream()
> > +                    reader.entries().stream()
> >                     .filter(n -> n.endsWith(".class"))
> >                     .filter(cn -> toPackageName(cn).isEmpty())
> >                     .findFirst();
> > @@ -696,6 +696,7 @@
> >                 return false;
> >             }
> >         }
> > +        }
> >
> >         ModuleInfoBuilder builder
> >             = new ModuleInfoBuilder(config, inputArgs,
> options.genModuleInfo);
> >
> > Thanks
> > Mandy
>
>  <http://oracle.com/us/design/oracle-email-sig-198324.gif>
>  <http://oracle.com/us/design/oracle-email-sig-198324.gif> <
> http://oracle.com/us/design/oracle-email-sig-198324.gif>
>  <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen|
> Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>
>
>
>
>

From vladimir.kozlov at oracle.com  Wed Oct 12 23:09:07 2016
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 12 Oct 2016 16:09:07 -0700
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
	<b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
Message-ID: <f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>

Hi Goetz and Volker,

Positive news is that JEP status is moving, changes are reviewed and some changes were already pushed.

But because of our testing issues during past week I was not able to execute the testing.

We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge hs-comp repository first. hs/ repo will be 
opened for other pushes soon after that.

I added estimated integration date to the JEP (Oct 28). We would like to test and integrate this port before JDK 10 
forest is forked. Do you think all s390 changes and new code will be ready by that date?

Do you have all shared changes reviewed and approved for push?

Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare changeset based on hs/ repo for changes which 
are not pushed yet? I will try to submit testing over weekend.

Regards,
Vladimir

On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote:
> Hi Vladimir,
>
> This webrev contains all the changes to hotspot needed for the port:
> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/
>
> It includes
> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/
> http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/
> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/
> which are out for review. Further it includes
> the one change to relocate the pc-relative instructions where we didn't open
> a bug for yet, and the new s390-files.
>
> Altogether this passed all our tests that were running on the weekend
> on linuxs390.
>
> The s390-files though are not yet fully in shape, I'm still editing them to get
> rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded by
> #ifdef SAPJVM  will go away in the end.
>
> I hope to have the final versions by end of this week.
>
> Best regards,
>   Goetz.
>
>
>> -----Original Message-----
>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net]
>> On Behalf Of Vladimir Kozlov
>> Sent: Montag, 3. Oktober 2016 23:50
>> To: Volker Simonis <volker.simonis at gmail.com>
>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
>> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source Developers
>> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
>> dev at openjdk.java.net>
>> Subject: Re: RFR: JEP draft for Linux/s3990x port
>>
>> Hi Volker,
>>
>> Can you prepare combined patch (or set of patches) based on latest
>> reviews together with s390 code as it will be in final push?
>>
>> We want to run it through our pre-integration testing to verify that it
>> does not have problems.
>>
>> Thanks,
>> Vladimir
>>
>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
>>> other) state:
>>>
>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
>>>
>>> Vladimir
>>>
>>> On 9/29/16 9:55 AM, Volker Simonis wrote:
>>>> Hi Vladimir,
>>>>
>>>> thanks a lot for reviewing and endorsing the JEP.
>>>>
>>>> I've linked all the relevant issues to the JEP  (they all have a link
>>>> to a webrev) and change the state to "Submitted".
>>>>
>>>> There's just one more small shared change we need for the port for
>>>> which we haven't opened a bug now because we are still working on
>>>> simplifying it. The current version looks as follows:
>>>>
>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
>> constant_table_offset.patch
>>>>
>>>>
>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t
>>>> he JEP and add a corresponding "FC Extension Request" comment to it?
>>>> Or will this be done automatically once I move it to "Candidate"?
>>>>
>>>> Is there anything left to do before I can move it to "Candidate" state?
>>>>
>>>> Thanks a lot and best regards,
>>>> Volker
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
>>>> <vladimir.kozlov at oracle.com> wrote:
>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> can you please review and endorse the following draft JEP for the
>>>>>> integration of the Linux/s390x port into the jkd9 master repository:
>>>>>>
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
>>>>>
>>>>>
>>>>> Good.
>>>>> Add links to webrevs in a comment. It will help to get umbrella FC
>>>>> extension
>>>>> approval.
>>>>>
>>>>>>
>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared
>>>>>> changes and we therefore don't foresee any impact on the existing
>>>>>> platforms at all. Following you can find a short description of the
>>>>>> planned changes:
>>>>>>
>>>>>> hotspot:
>>>>>> =======
>>>>>>
>>>>>> Out for review:
>>>>>> 8166560: [s390] Basic enablement of s390 port.
>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>> basic_s390/hotspot.wr01/
>>>>>>
>>>>>> Reviewed:
>>>>>> 8166562: C2: Suppress relocations in scratch emit.
>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>> scratch_emit/webrev.01/
>>>>>>
>>>>>> Will send RFR soon (depends on 8166560):
>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>> scratch_emit/webrev.01
>>>>>
>>>>>
>>>>> Wrong link.
>>>>>
>>>>> Thanks,
>>>>> Vladimir
>>>>>
>>>>>
>>>>>>
>>>>>> We are still investigating the need of these shared changes:
>>>>>>
>>>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>> 011-pass_PC_to_retAddrOffset.patch
>>>>>>
>>>>>>
>>>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>> 016-constant_table_offset.patch
>>>>>>
>>>>>>
>>>>>> And finally the patch with the s390x-only platform files. We are still
>>>>>> editing these to get them into OpenJdk style and shape.
>>>>>> Hotspot passes most jck, jtreg and spec tests with these.
>>>>>>
>>>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>> 101-zFiles.patch
>>>>>>
>>>>>>
>>>>>> top-level repository:
>>>>>> ===============
>>>>>>
>>>>>> The following is just adding some s390x specific compiler flags to
>>>>>> flags.m4
>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
>>>>>>
>>>>>> jdk repository:
>>>>>> ============
>>>>>>
>>>>>> This one just adds a new jvm.cfg file for s390x
>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
>>>>>>
>>>>>>
>>>>>> And finally we plan to do one more change which fixes the jtreg test
>>>>>> on Linux/s390x. But this is mainly for the correct detection of the
>>>>>> platform and for excluding the tests which are not appropriate for
>>>>>> s390x.
>>>>>>
>>>>>> Thank you and best regards,
>>>>>> Volker
>>>>>>
>>>>>

From timo.kinnunen at gmail.com  Wed Oct 12 23:49:05 2016
From: timo.kinnunen at gmail.com (timo.kinnunen at gmail.com)
Date: Thu, 13 Oct 2016 01:49:05 +0200
Subject: RFR(s): 8152617 add missing wildcards to Optional or()andflatMap()
In-Reply-To: <1450399505.1503099.1476275969715.JavaMail.zimbra@u-pem.fr>
References: <9cd9ab08-a8f8-e348-c09c-86a70a715648@oracle.com>
	<85CEBF2B-7D04-424B-847C-F633F823E159@oracle.com>
	<9c0676cc-cf87-ba5e-4842-a74e848cc982@oracle.com>
	<CAPkkuSff=imzhoDvXOefTEtkYRpzGzBEvMPOL-V2a_M40ejL+g@mail.gmail.com>
	<aa611977-969c-f0d4-2644-10417c12db57@oracle.com>
	<CAPkkuSfNxc170b68dWPNBP4jrCoXL3iMBMhE8tPK6F0fZt4GYA@mail.gmail.com>
	<d761bc9a-231c-2f53-cd37-d45b0b575f49@oracle.com>
	<57fde766.c89a1c0a.97b0d.6b9c@mx.google.com>
	<1450399505.1503099.1476275969715.JavaMail.zimbra@u-pem.fr>
Message-ID: <57fecbfd.88abc20a.7b8ef.5991@mx.google.com>

Hi, 

I know of in/inout/out parameters from GLSL and, with most all conversions in GLSL being explicit, they are the easiest thing ever to reason about. In GLSL a 2D vector and a float array of 2 can both be accessed using array indexing, but nobody cares if one should be a subtype of another or vice versa. Not me, not the compiler and not the hardware.

Having the caller reason about co-/in-/contra-/use site/declaration site variance or existential types (such as subtypes of Optional that will only exist in the future, if at all) in exchange for an explicit conversion being hidden as an unchecked cast inside a library method (as in this RFR) is a lose-lose situation, in my opinion.

If there is no other use case for existential types in Java then Project Valhalla needs to start making noises about the imminent deprecation of wildcards today.


-- 
Have a nice day, 
Timo

Sent from Mail for Windows 10

From: Remi Forax
Sent: Wednesday, October 12, 2016 14:44
To: timo kinnunen
Cc: Stuart Marks; core-libs-dev
Subject: Re: RFR(s): 8152617 add missing wildcards to Optional or()andflatMap()

Hi Timo,

----- Mail original -----
> De: "timo kinnunen" <timo.kinnunen at gmail.com>
> ?: "Stuart Marks" <stuart.marks at oracle.com>
> Cc: "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoy?: Mercredi 12 Octobre 2016 09:33:54
> Objet: RE: RFR(s): 8152617 add missing wildcards to Optional or()	andflatMap()

> Hi,
> 
> 
> I?m going to challenge the consensus a little bit. First, R?mi's example can be
> simplified to this one method which fails to compile with the same error
> message[0]:
> 
> private static Optional<?> simple1(
> Optional<String> o, Function<String, Optional<?>> f
> ) {return o.flatMap(f);}
> 
> Second, although not in the webrev, Optional::map and Optional::flatMap can be
> implemented without needing any unchecked casts:
> 
> public <U, R extends U, O extends Optional<R>> Optional<U> flatMap(
> Function<T, O> mapper
> ) {
> Objects.requireNonNull(mapper);
> return ofNullable(isPresent() ? mapper.apply(get()).orElse(null) : null);
> }
> public <P, Q extends P> Optional<P> map(
> Function<T, Q> mapper
> ) {
> Objects.requireNonNull(mapper);
> return ofNullable(isPresent() ? mapper.apply(get()) : null);
> }
> 
> These are fully, soundly compile-time typed methods with all types exposed and
> completely under the control of the programmer.

You miss the whole point of wildcards, wildcards are use site variance declarations.
When you have a method that takes a Function<T, R> (with T the type of the parameter type and R the return type) as parameter, you can always send a function that takes a super type and returns a subtype.
A function is contravariant on its parameter types and covariant on the return type.
In a sane language, you should be able to declare something like:

  interface Function<T or a super type of T, R or a subtype of R> {
    public R apply(T t);
  }  

but because of the backward compatibility of the collections, in Java, you do not use variance annotations when you declare Function but at each time when you use it.
So you have to write:
  <U, V> V foo(Optional<U or a subtype of U> optional, Function<U or a superType of U, V or a subtype of V> function) {
    ...
  }

and to make things less readable, instead of using U+ for U or a subtype of U and U- for U and a supertype of U, we use (respectively) ? extends U and ? super U, which can be great when your a beginner because you can read it in english, by example, ? extends U can be read like that: it's a type i don't want to know (?) which is a subtype of U but at the same time because it uses the 'extends' keyword like when you specify the bound of a type variable, you can easily found that using only type variables is more readable, but it's not. A type variable should be used to 'connect' several types in the signature together, like the type of the second parameter is the same as the type of the return type. So fundamentally, a type variable and a wildcard while denote subtyping relationship are two different notations for two different kind of usages.  

> 
> Now, to see if the signature of flatMap is truly the problem, it?s possible to
> write the simple1 method as a map followed by a flatMap with the identity
> function, like this:
> 
> private static Optional<?> simple2(
> Optional<String> o, Function<String, Optional<?>> f
> ) {return o.map(f).flatMap(Function.identity());}
> 
> In this version, the call to flatMap and its argument don?t use any wildcard
> types, both in the version above and in java.util.Optional[1]. Despite that,
> the compiler from jdk1.8.0_102 still gives an error[2] from the flatMap method
> call. This is quite a curious result as here flatMap and identity are reusing
> types that are already used by type inference. If this isn?t sound but using
> wildcards is then I would really like to see that counterexample!
> 
> Some questions that have arisen (and my answers):
> 
> Should APIs account for types that are not denotable in source code? I?d say no,
> such types are bugs in the language.
> 
> Can non-denotable types be eliminated at the library level by adding more
> wildcards? Unlikely, as such types come from using wildcards to begin with.
> 
> Is there a bug in flatMap or is the bug in the language specification? Compared
> to one simple method, type inference rules are much harder to understand and
> thus more likely to contain undiscovered problems.
> 
> What is gained if these wildcards are added? Although simple1 and simple2 would
> compile, you still can?t do anything interesting like calling orElse or
> orElseGet:
> 
> public static void main(String[] args) {
> Optional<String> foo = Optional.ofNullable(args[0]);
> Object o1 = simple1(foo, s -> foo).orElse(o1 = null);
> Object o2 = simple2(foo, s -> foo).orElseGet(() -> null);
> }
> 
> Won?t compile. As far as I can tell, there is no real upside to this change.
> 
> 
> Thanks for your consideration!
> 

R?mi

> 
> 
> 
> [0] Error:(18, 20) java: method flatMap in class java.util.Optional<T> cannot be
> applied to given types;
>  required: java.util.function.Function<? super
>  java.lang.String,java.util.Optional<U>>
>  found: java.util.function.Function<java.lang.String,java.util.Optional<?>>
>  reason: cannot infer type-variable(s) U
>    (argument mismatch;
>    java.util.function.Function<java.lang.String,java.util.Optional<?>> cannot be
>    converted to java.util.function.Function<? super
>    java.lang.String,java.util.Optional<U>>)
> 
> [1] The <? super T> type can be discounted as its presence doesn?t make a
> difference in this case.
> 
> [2] Error:(21, 27) java: method flatMap in class wildcards.another.Optional<T>
> cannot be applied to given types;
>  required: java.util.function.Function<wildcards.another.Optional<?>,O>
>  found: java.util.function.Function<java.lang.Object,java.lang.Object>
>  reason: inference variable O has incompatible bounds
>    equality constraints: wildcards.another.Optional<?>,T
>    upper bounds: wildcards.another.Optional<capture#11 of ?>,java.lang.Object
> 
> 
> --
> Have a nice day,
> Timo
> 
> Sent from Mail for Windows 10
> 
> From: Stuart Marks
> Sent: Saturday, October 8, 2016 02:28
> To: Stefan Zobel
> Cc: core-libs-dev
> Subject: Re: RFR(s): 8152617 add missing wildcards to Optional or() andflatMap()
> 
> 
> 
> On 10/7/16 3:12 PM, Stefan Zobel wrote:
>>> ... After having looked at lots of generic APIs, it seems to me that a
>>> style has emerged where wildcards are used whenever possible, and type
>>> parameters are used only when necessary, ...
>>
>> Yes, I'm familiar with that kind of reasoning (I think Josh Bloch stated
>> that principle in "Effective Java"). But, to be honest, I never thought
>> that it should apply as a strict rule in all circumstances.
> 
> Yep, it's in Effective Java, near the end of Item 28:
> 
>     ?As a rule, if a type parameter appears only once in a method
>     declaration, replace it with a wildcard.?
> 
>> Rhetorical question: do you really think that a signature employing 3
>> wildcards is easier to understand for the proverbial "average Joe" than
>> a bounded type parameter that expresses the sub-type relationship clearly?
>> I do not.
>>
>> But anyway, you probably have to follow the established "style".
>>
>> It's just too bad that most Java programmers I know won't understand the
>> proposed signature of 'flatMap'.
> 
> Turns out that R?mi's example exposes the difference between the wildcard
> approach and the type-parameter approach. Returning to the example,
> 
>     Optional<Integer> oi = Optional.empty();
>     Function<Number, Optional<StringBuilder>> fm = n -> Optional.empty();
>     Optional<CharSequence> ocs = oi.flatMap(fm);
> 
> If the flatMapper function itself has a wildcard type, for example,
> 
>     Function<Number, Optional<? extends CharSequence>> fm = n -> Optional.empty();
> 
> then this will still work with the wildcard approach but fail with the
> type-parameter approach. As R?mi also pointed out, a wildcarded type can result
> from the capture of a type with a wildcarded type parameter.
> 
> Based on this, I believe the nested wildcard approach to be the correct one.
> 
> s'marks


From steve.drach at oracle.com  Thu Oct 13 00:10:12 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 12 Oct 2016 17:10:12 -0700
Subject: Fwd: RFR: 8167237: Jar tool can not correctly find/process the
	--release option if it occurs before the file list
References: <D299E831-210E-42DD-BBC8-3F8107B416ED@oracle.com>
Message-ID: <1F0B5EE9-B9A9-499B-9F99-D29B183E6F47@oracle.com>

Hi again,

Is there anyone willing to review this?  It?s a very simple change.

Thanks
Steve

> Begin forwarded message:
> 
> From: Steve Drach <steve.drach at oracle.com>
> Subject: RFR: 8167237: Jar tool can not correctly find/process the --release option if it occurs before the file list
> Date: October 6, 2016 at 1:50:30 PM PDT
> To: core-libs-dev <core-libs-dev at openjdk.java.net>
> 
> Hi,
> 
> Please review the changeset that fixes the problem of not ?seeing? the jar tool ?release  option if it is not preceded by anything other than gnu-style options.  It?s a simple one line change to process ?release the same way as -C.
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8167237 <https://bugs.openjdk.java.net/browse/JDK-8167237>webrev: http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/>
> 
> Thanks,
> Steve

From naoto.sato at oracle.com  Thu Oct 13 00:20:11 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Wed, 12 Oct 2016 17:20:11 -0700
Subject: RFR JDK-8166258: Unexpected code conversion by HKSCS converters
In-Reply-To: <57FEBE2F.6050503@oracle.com>
References: <57FEBE2F.6050503@oracle.com>
Message-ID: <df40ac58-bd85-36d0-6086-e2236fe392a1@oracle.com>

Looks good to me.

Naoto

On 10/12/16 3:50 PM, Xueming Shen wrote:
> Hi
>
> Please help review the change for #8166258
>
> issue: https://bugs.openjdk.java.net/browse/JDK-8166258
> webrev: http://cr.openjdk.java.net/~sherman/8166258/webrev/
>
> This is an overlook in the code that generates the c2b (encoding) mapping
> table from the b2c data. \ufffd is the special code point that used to
> indicate
> "no mapping" from a native code point to unicode code point. It's wrong to
> try to generate a c2b mapping entry from \ufffd.
>
> Thanks,
> Sherman

From lance.andersen at oracle.com  Thu Oct 13 00:38:12 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Wed, 12 Oct 2016 20:38:12 -0400
Subject: RFR: 8167237: Jar tool can not correctly find/process the
	--release option if it occurs before the file list
In-Reply-To: <1F0B5EE9-B9A9-499B-9F99-D29B183E6F47@oracle.com>
References: <D299E831-210E-42DD-BBC8-3F8107B416ED@oracle.com>
	<1F0B5EE9-B9A9-499B-9F99-D29B183E6F47@oracle.com>
Message-ID: <8D7768FE-1079-471A-A072-FDB450EE787F@oracle.com>

looks OK Steve

> On Oct 12, 2016, at 8:10 PM, Steve Drach <steve.drach at oracle.com> wrote:
> 
> Hi again,
> 
> Is there anyone willing to review this?  It?s a very simple change.
> 
> Thanks
> Steve
> 
>> Begin forwarded message:
>> 
>> From: Steve Drach <steve.drach at oracle.com>
>> Subject: RFR: 8167237: Jar tool can not correctly find/process the --release option if it occurs before the file list
>> Date: October 6, 2016 at 1:50:30 PM PDT
>> To: core-libs-dev <core-libs-dev at openjdk.java.net>
>> 
>> Hi,
>> 
>> Please review the changeset that fixes the problem of not ?seeing? the jar tool ?release  option if it is not preceded by anything other than gnu-style options.  It?s a simple one line change to process ?release the same way as -C.
>> 
>> issue: https://bugs.openjdk.java.net/browse/JDK-8167237 <https://bugs.openjdk.java.net/browse/JDK-8167237>webrev: http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8167237/webrev.00/>
>> 
>> Thanks,
>> Steve

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From anubhav.meena at oracle.com  Thu Oct 13 06:28:31 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Thu, 13 Oct 2016 11:58:31 +0530
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <bc42fe3c-1b04-84bc-34d4-66fb9f28990b@Oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
	<99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
	<a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>
	<F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>
	<bc42fe3c-1b04-84bc-34d4-66fb9f28990b@Oracle.com>
Message-ID: <D56792DB-1562-408F-8609-57065DA857FF@oracle.com>

Thanks Roger, appreciate that.

> On Oct 13, 2016, at 3:09 AM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Hi,
> 
> Looks ok.
> 
> Typically, there is a space after the '//' comment characters, it makes it easier to read.
> Also, in this case, I don't think the comments help.  "Monday" isn't important to the test 
> and the 'Any Other day' is still 1, not something else.
> 
> I would remove those comments.
> Otherwise fine, no need for another webrev.
> 
> Thanks, Roger
> 
> 
> On 10/12/2016 2:16 PM, Anubhav Meena wrote:
>> Hi Roger,
>> 
>> Thanks for the suggestion, have made the suggested changes and shifted the tests to /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java. 
>> 
>> Updated webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.04/ <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.04/>
>> 
>> Thanks,
>> Anubhav
>> 
>>> On Oct 12, 2016, at 8:45 PM, Roger Riggs <Roger.Riggs at Oracle.com <mailto:Roger.Riggs at oracle.com>> wrote:
>>> 
>>> Hi Anubhav,
>>> 
>>> In general, I agree with Stephen that the tests should be testing an algorithm against facts.
>>> Embedding an algorithm in the test increases the risk that the test will just replicate the 
>>> implementation code and therefor not be much of a test.
>>> 
>>> Though in this case, the specification of aligned day of week is of a computation.
>>> If the test were to independently compute the correct answer, it would be valid as a 'tck' test.
>>> 
>>> Since the Hijrah calendar is data driven, the tests should be in /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java.
>>> Tests in java/time/tck/... should correspond directly to specified behaviors.
>>> In this case, the algorithm is specified but the test is data dependent. (Perhaps a gray area).
>>> 
>>> Thanks, Roger
>>> 
>>> 
>>> On 10/12/2016 11:03 AM, Anubhav Meena wrote:
>>>> Hi Stephen,
>>>> 
>>>> Have incorporated the changes you suggested. Updated webrev is available here 
>>>> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/ <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.03/>
>>>> 
>>>> Please review and suggest if anymore changes are required.
>>>> 
>>>> Thanks,
>>>> Anubhav
>>>> 
>>>>> On Oct 12, 2016, at 3:21 PM, Anubhav Meena <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>>>> 
>>>>> Gentle reminder.
>>>>> 
>>>>>> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com <mailto:anubhav.meena at oracle.com>> wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> Please review.
>>>>>> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330 <https://bugs.openjdk.java.net/browse/JDK-8163330>
>>>>>> 
>>>>>> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week field. It based the calculation on the day-of-week, when it should be based on the day-of-month.
>>>>>> 
>>>>>> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/ <http://cr.openjdk.java.net/%7Erchamyal/anmeena/8163330/webrev.02/>
>>>>>> -- 
>>>>>> Thanks and Regards,
>>>>>> Anubhav
>>>>> 
>>>> 
>>> 
>> 
> 


From volker.simonis at gmail.com  Thu Oct 13 06:31:31 2016
From: volker.simonis at gmail.com (Volker Simonis)
Date: Thu, 13 Oct 2016 08:31:31 +0200
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
	<b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
	<f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
Message-ID: <CA+3eh11eoAefQYRQ7rMtN9EN7rnh50pEtZk0CdF3rz43ef29qw@mail.gmail.com>

Hi Vladimir,

thanks for keeping us updated and hanks for setting the integrations
date on the JEP. We think it is reasonable to do it before the JDK 10
forest will be forked as this will obviously save us a lot of work.

The remaining shared changes are

"8166561: [s390] Adaptions needed for s390 port in C1 and C2."
"8166560: [s390] Basic enablement of s390 port."
"8167184: [s390] Extend relocations for pc-relative instructions."
"8166837: [TESTBUG] Fix tests on Linux/s390x"


Except the TESTBUG, they are all reviewed but I think the testbug is
not so important and could be done later if we don't get it reviewd
until the integration date.

I'm currently traveling, but I'm sure Goetz can provide a new hs/based
changeset for testing, right Goetz :)

Regards,
Volker


On Thu, Oct 13, 2016 at 1:09 AM, Vladimir Kozlov
<vladimir.kozlov at oracle.com> wrote:
> Hi Goetz and Volker,
>
> Positive news is that JEP status is moving, changes are reviewed and some
> changes were already pushed.
>
> But because of our testing issues during past week I was not able to execute
> the testing.
>
> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge
> hs-comp repository first. hs/ repo will be opened for other pushes soon
> after that.
>
> I added estimated integration date to the JEP (Oct 28). We would like to
> test and integrate this port before JDK 10 forest is forked. Do you think
> all s390 changes and new code will be ready by that date?
>
> Do you have all shared changes reviewed and approved for push?
>
> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare
> changeset based on hs/ repo for changes which are not pushed yet? I will try
> to submit testing over weekend.
>
> Regards,
> Vladimir
>
>
> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote:
>>
>> Hi Vladimir,
>>
>> This webrev contains all the changes to hotspot needed for the port:
>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/
>>
>> It includes
>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/
>> http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/
>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/
>> which are out for review. Further it includes
>> the one change to relocate the pc-relative instructions where we didn't
>> open
>> a bug for yet, and the new s390-files.
>>
>> Altogether this passed all our tests that were running on the weekend
>> on linuxs390.
>>
>> The s390-files though are not yet fully in shape, I'm still editing them
>> to get
>> rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded
>> by
>> #ifdef SAPJVM  will go away in the end.
>>
>> I hope to have the final versions by end of this week.
>>
>> Best regards,
>>   Goetz.
>>
>>
>>> -----Original Message-----
>>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net]
>>> On Behalf Of Vladimir Kozlov
>>> Sent: Montag, 3. Oktober 2016 23:50
>>> To: Volker Simonis <volker.simonis at gmail.com>
>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
>>> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source Developers
>>> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
>>> dev at openjdk.java.net>
>>> Subject: Re: RFR: JEP draft for Linux/s3990x port
>>>
>>> Hi Volker,
>>>
>>> Can you prepare combined patch (or set of patches) based on latest
>>> reviews together with s390 code as it will be in final push?
>>>
>>> We want to run it through our pre-integration testing to verify that it
>>> does not have problems.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
>>>>
>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
>>>> other) state:
>>>>
>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
>>>>
>>>> Vladimir
>>>>
>>>> On 9/29/16 9:55 AM, Volker Simonis wrote:
>>>>>
>>>>> Hi Vladimir,
>>>>>
>>>>> thanks a lot for reviewing and endorsing the JEP.
>>>>>
>>>>> I've linked all the relevant issues to the JEP  (they all have a link
>>>>> to a webrev) and change the state to "Submitted".
>>>>>
>>>>> There's just one more small shared change we need for the port for
>>>>> which we haven't opened a bug now because we are still working on
>>>>> simplifying it. The current version looks as follows:
>>>>>
>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
>>>
>>> constant_table_offset.patch
>>>>>
>>>>>
>>>>>
>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t
>>>>> he JEP and add a corresponding "FC Extension Request" comment to it?
>>>>> Or will this be done automatically once I move it to "Candidate"?
>>>>>
>>>>> Is there anything left to do before I can move it to "Candidate" state?
>>>>>
>>>>> Thanks a lot and best regards,
>>>>> Volker
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
>>>>> <vladimir.kozlov at oracle.com> wrote:
>>>>>>
>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> can you please review and endorse the following draft JEP for the
>>>>>>> integration of the Linux/s390x port into the jkd9 master repository:
>>>>>>>
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
>>>>>>
>>>>>>
>>>>>>
>>>>>> Good.
>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC
>>>>>> extension
>>>>>> approval.
>>>>>>
>>>>>>>
>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared
>>>>>>> changes and we therefore don't foresee any impact on the existing
>>>>>>> platforms at all. Following you can find a short description of the
>>>>>>> planned changes:
>>>>>>>
>>>>>>> hotspot:
>>>>>>> =======
>>>>>>>
>>>>>>> Out for review:
>>>>>>> 8166560: [s390] Basic enablement of s390 port.
>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>>>
>>> basic_s390/hotspot.wr01/
>>>>>>>
>>>>>>>
>>>>>>> Reviewed:
>>>>>>> 8166562: C2: Suppress relocations in scratch emit.
>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>>>
>>> scratch_emit/webrev.01/
>>>>>>>
>>>>>>>
>>>>>>> Will send RFR soon (depends on 8166560):
>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>>>
>>> scratch_emit/webrev.01
>>>>>>
>>>>>>
>>>>>>
>>>>>> Wrong link.
>>>>>>
>>>>>> Thanks,
>>>>>> Vladimir
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> We are still investigating the need of these shared changes:
>>>>>>>
>>>>>>>
>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>> 011-pass_PC_to_retAddrOffset.patch
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>> 016-constant_table_offset.patch
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> And finally the patch with the s390x-only platform files. We are
>>>>>>> still
>>>>>>> editing these to get them into OpenJdk style and shape.
>>>>>>> Hotspot passes most jck, jtreg and spec tests with these.
>>>>>>>
>>>>>>>
>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>> 101-zFiles.patch
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> top-level repository:
>>>>>>> ===============
>>>>>>>
>>>>>>> The following is just adding some s390x specific compiler flags to
>>>>>>> flags.m4
>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
>>>>>>>
>>>>>>> jdk repository:
>>>>>>> ============
>>>>>>>
>>>>>>> This one just adds a new jvm.cfg file for s390x
>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
>>>>>>>
>>>>>>>
>>>>>>> And finally we plan to do one more change which fixes the jtreg test
>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the
>>>>>>> platform and for excluding the tests which are not appropriate for
>>>>>>> s390x.
>>>>>>>
>>>>>>> Thank you and best regards,
>>>>>>> Volker
>>>>>>>
>>>>>>
>

From frank.yuan at oracle.com  Thu Oct 13 09:05:18 2016
From: frank.yuan at oracle.com (Frank Yuan)
Date: Thu, 13 Oct 2016 17:05:18 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
Message-ID: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>

Hi all

 

Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/ ?

Bug: https://bugs.openjdk.java.net/browse/JDK-8167478 

 

This is a test bug, because Bug6341770.java is invalid when the system environment doesn't support non-ascii characters, the test
will exit immediately in this condition.

 

 

Thanks,

 

Frank

 


From goetz.lindenmaier at sap.com  Thu Oct 13 11:08:41 2016
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Thu, 13 Oct 2016 11:08:41 +0000
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <CA+3eh11eoAefQYRQ7rMtN9EN7rnh50pEtZk0CdF3rz43ef29qw@mail.gmail.com>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
	<b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
	<f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
	<CA+3eh11eoAefQYRQ7rMtN9EN7rnh50pEtZk0CdF3rz43ef29qw@mail.gmail.com>
Message-ID: <53905441dbdc41159896a1b143fdde70@DEWDFE13DE50.global.corp.sap>

Hi Vladimir, 

I'm about to post the final change with the s390 files.
All the others are reviewed.  I'll then also send a 
complete webrev for testing. 

Best regards,
  Goetz

> -----Original Message-----
> From: Volker Simonis [mailto:volker.simonis at gmail.com]
> Sent: Thursday, October 13, 2016 8:32 AM
> To: Vladimir Kozlov <vladimir.kozlov at oracle.com>
> Cc: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; s390x-port-
> dev at openjdk.java.net; porters-dev at openjdk.java.net; build-dev <build-
> dev at openjdk.java.net>; HotSpot Open Source Developers <hotspot-
> dev at openjdk.java.net>; Java Core Libs <core-libs-dev at openjdk.java.net>
> Subject: Re: RFR: JEP draft for Linux/s3990x port
> 
> Hi Vladimir,
> 
> thanks for keeping us updated and hanks for setting the integrations
> date on the JEP. We think it is reasonable to do it before the JDK 10
> forest will be forked as this will obviously save us a lot of work.
> 
> The remaining shared changes are
> 
> "8166561: [s390] Adaptions needed for s390 port in C1 and C2."
> "8166560: [s390] Basic enablement of s390 port."
> "8167184: [s390] Extend relocations for pc-relative instructions."
> "8166837: [TESTBUG] Fix tests on Linux/s390x"
> 
> 
> Except the TESTBUG, they are all reviewed but I think the testbug is
> not so important and could be done later if we don't get it reviewd
> until the integration date.
> 
> I'm currently traveling, but I'm sure Goetz can provide a new hs/based
> changeset for testing, right Goetz :)
> 
> Regards,
> Volker
> 
> 
> On Thu, Oct 13, 2016 at 1:09 AM, Vladimir Kozlov
> <vladimir.kozlov at oracle.com> wrote:
> > Hi Goetz and Volker,
> >
> > Positive news is that JEP status is moving, changes are reviewed and some
> > changes were already pushed.
> >
> > But because of our testing issues during past week I was not able to
> execute
> > the testing.
> >
> > We hope jdk9/hs will be open soon but we want to sync jdk9/dev and
> merge
> > hs-comp repository first. hs/ repo will be opened for other pushes soon
> > after that.
> >
> > I added estimated integration date to the JEP (Oct 28). We would like to
> > test and integrate this port before JDK 10 forest is forked. Do you think
> > all s390 changes and new code will be ready by that date?
> >
> > Do you have all shared changes reviewed and approved for push?
> >
> > Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare
> > changeset based on hs/ repo for changes which are not pushed yet? I will
> try
> > to submit testing over weekend.
> >
> > Regards,
> > Vladimir
> >
> >
> > On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote:
> >>
> >> Hi Vladimir,
> >>
> >> This webrev contains all the changes to hotspot needed for the port:
> >> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-
> all/hotspot.wr01/
> >>
> >> It includes
> >> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> basic_s390/hotspot.wr03/
> >> http://cr.openjdk.java.net/~goetz/wr16/8166561-
> basic_C1C2_s390/webrev.01/
> >> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> scratch_emit/webrev.01/
> >> which are out for review. Further it includes
> >> the one change to relocate the pc-relative instructions where we didn't
> >> open
> >> a bug for yet, and the new s390-files.
> >>
> >> Altogether this passed all our tests that were running on the weekend
> >> on linuxs390.
> >>
> >> The s390-files though are not yet fully in shape, I'm still editing them
> >> to get
> >> rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded
> >> by
> >> #ifdef SAPJVM  will go away in the end.
> >>
> >> I hope to have the final versions by end of this week.
> >>
> >> Best regards,
> >>   Goetz.
> >>
> >>
> >>> -----Original Message-----
> >>> From: s390x-port-dev [mailto:s390x-port-dev-
> bounces at openjdk.java.net]
> >>> On Behalf Of Vladimir Kozlov
> >>> Sent: Montag, 3. Oktober 2016 23:50
> >>> To: Volker Simonis <volker.simonis at gmail.com>
> >>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
> >>> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source
> Developers
> >>> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
> >>> dev at openjdk.java.net>
> >>> Subject: Re: RFR: JEP draft for Linux/s3990x port
> >>>
> >>> Hi Volker,
> >>>
> >>> Can you prepare combined patch (or set of patches) based on latest
> >>> reviews together with s390 code as it will be in final push?
> >>>
> >>> We want to run it through our pre-integration testing to verify that it
> >>> does not have problems.
> >>>
> >>> Thanks,
> >>> Vladimir
> >>>
> >>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
> >>>>
> >>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
> >>>> other) state:
> >>>>
> >>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
> >>>>
> >>>> Vladimir
> >>>>
> >>>> On 9/29/16 9:55 AM, Volker Simonis wrote:
> >>>>>
> >>>>> Hi Vladimir,
> >>>>>
> >>>>> thanks a lot for reviewing and endorsing the JEP.
> >>>>>
> >>>>> I've linked all the relevant issues to the JEP  (they all have a link
> >>>>> to a webrev) and change the state to "Submitted".
> >>>>>
> >>>>> There's just one more small shared change we need for the port for
> >>>>> which we haven't opened a bug now because we are still working on
> >>>>> simplifying it. The current version looks as follows:
> >>>>>
> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
> >>>
> >>> constant_table_offset.patch
> >>>>>
> >>>>>
> >>>>>
> >>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t
> >>>>> he JEP and add a corresponding "FC Extension Request" comment to it?
> >>>>> Or will this be done automatically once I move it to "Candidate"?
> >>>>>
> >>>>> Is there anything left to do before I can move it to "Candidate" state?
> >>>>>
> >>>>> Thanks a lot and best regards,
> >>>>> Volker
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
> >>>>> <vladimir.kozlov at oracle.com> wrote:
> >>>>>>
> >>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> can you please review and endorse the following draft JEP for the
> >>>>>>> integration of the Linux/s390x port into the jkd9 master repository:
> >>>>>>>
> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Good.
> >>>>>> Add links to webrevs in a comment. It will help to get umbrella FC
> >>>>>> extension
> >>>>>> approval.
> >>>>>>
> >>>>>>>
> >>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared
> >>>>>>> changes and we therefore don't foresee any impact on the existing
> >>>>>>> platforms at all. Following you can find a short description of the
> >>>>>>> planned changes:
> >>>>>>>
> >>>>>>> hotspot:
> >>>>>>> =======
> >>>>>>>
> >>>>>>> Out for review:
> >>>>>>> 8166560: [s390] Basic enablement of s390 port.
> >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> >>>
> >>> basic_s390/hotspot.wr01/
> >>>>>>>
> >>>>>>>
> >>>>>>> Reviewed:
> >>>>>>> 8166562: C2: Suppress relocations in scratch emit.
> >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> >>>
> >>> scratch_emit/webrev.01/
> >>>>>>>
> >>>>>>>
> >>>>>>> Will send RFR soon (depends on 8166560):
> >>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
> >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> >>>
> >>> scratch_emit/webrev.01
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Wrong link.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Vladimir
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> We are still investigating the need of these shared changes:
> >>>>>>>
> >>>>>>>
> >>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >>> 011-pass_PC_to_retAddrOffset.patch
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >>> 016-constant_table_offset.patch
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> And finally the patch with the s390x-only platform files. We are
> >>>>>>> still
> >>>>>>> editing these to get them into OpenJdk style and shape.
> >>>>>>> Hotspot passes most jck, jtreg and spec tests with these.
> >>>>>>>
> >>>>>>>
> >>>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >>> 101-zFiles.patch
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> top-level repository:
> >>>>>>> ===============
> >>>>>>>
> >>>>>>> The following is just adding some s390x specific compiler flags to
> >>>>>>> flags.m4
> >>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
> >>>>>>>
> >>>>>>> jdk repository:
> >>>>>>> ============
> >>>>>>>
> >>>>>>> This one just adds a new jvm.cfg file for s390x
> >>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
> >>>>>>>
> >>>>>>>
> >>>>>>> And finally we plan to do one more change which fixes the jtreg test
> >>>>>>> on Linux/s390x. But this is mainly for the correct detection of the
> >>>>>>> platform and for excluding the tests which are not appropriate for
> >>>>>>> s390x.
> >>>>>>>
> >>>>>>> Thank you and best regards,
> >>>>>>> Volker
> >>>>>>>
> >>>>>>
> >

From Alan.Burlison at oracle.com  Thu Oct 13 12:19:43 2016
From: Alan.Burlison at oracle.com (Alan Burlison)
Date: Thu, 13 Oct 2016 13:19:43 +0100
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
Message-ID: <d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>

On 06/10/2016 00:31, Brian Burkhalter wrote:

> Given that the functionality of O_DIRECT on Linux appears to be
> supported by other interfaces on OS X, Solaris, and Windows, I wonder
> whether the patch will need to be refactored in some way to
> accommodate these other operating systems? For reference it looks as
> if direct I/O on OS X uses the F_NOCACHE command of fcntl(2) [1]
> (although per some online comments this might have some problems),
> Solaris uses the advice argument of directio(3c) [2], and Windows
> uses a combination of flags passed to CreateFile() [3, 4].

The Linux open(2) manpage contains a long list of warnings about 
O_DIRECT, including:

----------
In Linux alignment restrictions vary by filesystem and kernel version 
and might be absent entirely.  However there is currently no 
filesystem-independent interface for an application to discover these 
restrictions for a given file or filesystem.
----------

----------
O_DIRECT I/Os should never be run concurrently with the fork(2) system
call, if the memory buffer is a private mapping (i.e., any
mapping created with the mmap(2) MAP_PRIVATE flag; this includes
memory allocated on the heap and statically allocated buffers). Any
such I/Os, whether submitted via an asynchronous I/O interface or
from another thread in the process, should be completed before
fork(2) is called. Failure to do so can result in data corruption
and undefined behavior in parent and child processes.
----------

----------
Applications should avoid mixing O_DIRECT and normal I/O to the same
file, and especially to overlapping byte regions in the same file.
Even when the filesystem correctly handles the coherency issues in
this situation, overall I/O throughput is likely to be slower than
using either mode alone. Likewise, applications should avoid mixing
mmap(2) of files with direct I/O to the same files.
----------

----------
  "The thing that has always disturbed me about O_DIRECT is that
the whole interface is just stupid, and was probably designed
by a deranged monkey on some serious mind-controlling
substances." - Linus
----------

Adding support for O_DIRECT has a far wider impact than adding just 
another IO handle flag. As such I'm opposed to this change as it seems 
to be prone to cause hard-to-diagnose failures on Linux and it is also 
specific to just Linux.

-- 
Alan Burlison
--

From scolebourne at joda.org  Thu Oct 13 13:03:30 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Thu, 13 Oct 2016 14:03:30 +0100
Subject: RFR:JDK-8163330:HijrahDate aligned day of week incorrect
In-Reply-To: <F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>
References: <B8BC0F92-C036-4DAF-84A8-8C86ADE1CC28@oracle.com>
	<542EF0AE-3F81-4999-A66F-B1FE4535B0EC@oracle.com>
	<99A33B3A-C0A4-4980-A17D-84A1D7085AFD@oracle.com>
	<a04491ca-ffd3-b7b2-5d23-6bed1241c733@Oracle.com>
	<F65B5430-39CF-4EED-B97B-FDFD612D3CD7@oracle.com>
Message-ID: <CACzrW9C+g-6ju28AvpcHmt89MOhJq48omXpXM80TLRHa=3vmrQ@mail.gmail.com>

+1
Stephen

On 12 October 2016 at 19:16, Anubhav Meena <anubhav.meena at oracle.com> wrote:
> Hi Roger,
>
> Thanks for the suggestion, have made the suggested changes and shifted the
> tests to /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java.
>
> Updated webrev:
> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.04/
>
> Thanks,
> Anubhav
>
>
> On Oct 12, 2016, at 8:45 PM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>
> Hi Anubhav,
>
> In general, I agree with Stephen that the tests should be testing an
> algorithm against facts.
> Embedding an algorithm in the test increases the risk that the test will
> just replicate the
> implementation code and therefor not be much of a test.
>
> Though in this case, the specification of aligned day of week is of a
> computation.
> If the test were to independently compute the correct answer, it would be
> valid as a 'tck' test.
>
> Since the Hijrah calendar is data driven, the tests should be in
> /java/time/test/java/time/chrono/TestUmmAlQuraChronology.java.
> Tests in java/time/tck/... should correspond directly to specified
> behaviors.
> In this case, the algorithm is specified but the test is data dependent.
> (Perhaps a gray area).
>
> Thanks, Roger
>
>
> On 10/12/2016 11:03 AM, Anubhav Meena wrote:
>
> Hi Stephen,
>
> Have incorporated the changes you suggested. Updated webrev is available
> here
> http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.03/
>
> Please review and suggest if anymore changes are required.
>
> Thanks,
> Anubhav
>
> On Oct 12, 2016, at 3:21 PM, Anubhav Meena <anubhav.meena at oracle.com> wrote:
>
> Gentle reminder.
>
> On Oct 7, 2016, at 2:12 PM, Anubhav Meena <anubhav.meena at oracle.com> wrote:
>
> Hi all,
>
> Please review.
>
> Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8163330
>
> Issue:  The HijrahDate class incorrectly calculates the aligned-day-of-week
> field. It based the calculation on the day-of-week, when it should be based
> on the day-of-month.
>
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/
>
> --
> Thanks and Regards,
>
> Anubhav
>
>
>
>
>

From brunoaiss at gmail.com  Thu Oct 13 13:45:09 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 13 Oct 2016 14:45:09 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
Message-ID: <1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>

Hi,

I looked at BufferedReader source code for java 9 long with the source 
code of the channels/streams used. I noticed that, like in java 7, 
BufferedReader does not use an Async API to load data from files, 
instead, the data loading is all done synchronously even when the OS 
allows requesting a file to be read and getting a warning later when the 
file is effectively read.

Why Is BufferedReader not async while providing a sync API?


From felix.yang at oracle.com  Thu Oct 13 13:56:21 2016
From: felix.yang at oracle.com (Felix Yang)
Date: Thu, 13 Oct 2016 21:56:21 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
Message-ID: <12733BE7-D427-43F2-891B-4CE3BF291791@oracle.com>

A comment on naming, alpha to ALPHA ?
-Felix

> ? 2016?10?13??17:05?Frank Yuan <frank.yuan at oracle.com> ???
> 
> Hi all
> 
> 
> 
> Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/ ?
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478 
> 
> 
> 
> This is a test bug, because Bug6341770.java is invalid when the system environment doesn't support non-ascii characters, the test
> will exit immediately in this condition.
> 
> 
> 
> 
> 
> Thanks,
> 
> 
> 
> Frank
> 
> 
> 


From brunoaiss at gmail.com  Thu Oct 13 13:41:31 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 13 Oct 2016 14:41:31 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
Message-ID: <71017846-900e-22e4-ca0c-486272efdf75@gmail.com>

Hi,

I'm new here, so if this is belongs to a different mailing list please
point me in the right direction.

I looked at BufferedReader source code for java 9 long with the source
code of the channels/streams used. I noticed that, like in java 7,
BufferedReader does not use an Async API to load data from files,
instead, the data loading is all done synchronously even when the OS
allows requesting a file to be read and getting a warning later when the
file is effectively read.

Why Is BufferedReader not async while providing a sync API?


From goetz.lindenmaier at sap.com  Thu Oct 13 16:22:16 2016
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Thu, 13 Oct 2016 16:22:16 +0000
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
	<b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
	<f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
Message-ID: <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap>

Hi Vladimir, 

I made a new webrev containing all outstanding changes merged into one patch
http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/

You probably saw my RFR with the s390 files.

Best regards,
  Goetz.

> -----Original Message-----
> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com]
> Sent: Thursday, October 13, 2016 1:09 AM
> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Volker Simonis
> <volker.simonis at gmail.com>
> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build-
> dev <build-dev at openjdk.java.net>; HotSpot Open Source Developers
> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
> dev at openjdk.java.net>
> Subject: Re: RFR: JEP draft for Linux/s3990x port
> 
> Hi Goetz and Volker,
> 
> Positive news is that JEP status is moving, changes are reviewed and some
> changes were already pushed.
> 
> But because of our testing issues during past week I was not able to execute
> the testing.
> 
> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge
> hs-comp repository first. hs/ repo will be
> opened for other pushes soon after that.
> 
> I added estimated integration date to the JEP (Oct 28). We would like to test
> and integrate this port before JDK 10
> forest is forked. Do you think all s390 changes and new code will be ready by
> that date?
> 
> Do you have all shared changes reviewed and approved for push?
> 
> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare
> changeset based on hs/ repo for changes which
> are not pushed yet? I will try to submit testing over weekend.
> 
> Regards,
> Vladimir
> 
> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote:
> > Hi Vladimir,
> >
> > This webrev contains all the changes to hotspot needed for the port:
> > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-
> all/hotspot.wr01/
> >
> > It includes
> > http://cr.openjdk.java.net/~goetz/wr16/8166560-
> basic_s390/hotspot.wr03/
> > http://cr.openjdk.java.net/~goetz/wr16/8166561-
> basic_C1C2_s390/webrev.01/
> > http://cr.openjdk.java.net/~goetz/wr16/8166562-
> scratch_emit/webrev.01/
> > which are out for review. Further it includes
> > the one change to relocate the pc-relative instructions where we didn't
> open
> > a bug for yet, and the new s390-files.
> >
> > Altogether this passed all our tests that were running on the weekend
> > on linuxs390.
> >
> > The s390-files though are not yet fully in shape, I'm still editing them to get
> > rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded by
> > #ifdef SAPJVM  will go away in the end.
> >
> > I hope to have the final versions by end of this week.
> >
> > Best regards,
> >   Goetz.
> >
> >
> >> -----Original Message-----
> >> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net]
> >> On Behalf Of Vladimir Kozlov
> >> Sent: Montag, 3. Oktober 2016 23:50
> >> To: Volker Simonis <volker.simonis at gmail.com>
> >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
> >> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source
> Developers
> >> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
> >> dev at openjdk.java.net>
> >> Subject: Re: RFR: JEP draft for Linux/s3990x port
> >>
> >> Hi Volker,
> >>
> >> Can you prepare combined patch (or set of patches) based on latest
> >> reviews together with s390 code as it will be in final push?
> >>
> >> We want to run it through our pre-integration testing to verify that it
> >> does not have problems.
> >>
> >> Thanks,
> >> Vladimir
> >>
> >> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
> >>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
> >>> other) state:
> >>>
> >>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
> >>>
> >>> Vladimir
> >>>
> >>> On 9/29/16 9:55 AM, Volker Simonis wrote:
> >>>> Hi Vladimir,
> >>>>
> >>>> thanks a lot for reviewing and endorsing the JEP.
> >>>>
> >>>> I've linked all the relevant issues to the JEP  (they all have a link
> >>>> to a webrev) and change the state to "Submitted".
> >>>>
> >>>> There's just one more small shared change we need for the port for
> >>>> which we haven't opened a bug now because we are still working on
> >>>> simplifying it. The current version looks as follows:
> >>>>
> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
> >> constant_table_offset.patch
> >>>>
> >>>>
> >>>> What are the next steps? Should I add a "jdk9-fc-request" label to t
> >>>> he JEP and add a corresponding "FC Extension Request" comment to it?
> >>>> Or will this be done automatically once I move it to "Candidate"?
> >>>>
> >>>> Is there anything left to do before I can move it to "Candidate" state?
> >>>>
> >>>> Thanks a lot and best regards,
> >>>> Volker
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
> >>>> <vladimir.kozlov at oracle.com> wrote:
> >>>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> can you please review and endorse the following draft JEP for the
> >>>>>> integration of the Linux/s390x port into the jkd9 master repository:
> >>>>>>
> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
> >>>>>
> >>>>>
> >>>>> Good.
> >>>>> Add links to webrevs in a comment. It will help to get umbrella FC
> >>>>> extension
> >>>>> approval.
> >>>>>
> >>>>>>
> >>>>>> As detailed in the JEP, the Linux/s390x requires very few shared
> >>>>>> changes and we therefore don't foresee any impact on the existing
> >>>>>> platforms at all. Following you can find a short description of the
> >>>>>> planned changes:
> >>>>>>
> >>>>>> hotspot:
> >>>>>> =======
> >>>>>>
> >>>>>> Out for review:
> >>>>>> 8166560: [s390] Basic enablement of s390 port.
> >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
> >> basic_s390/hotspot.wr01/
> >>>>>>
> >>>>>> Reviewed:
> >>>>>> 8166562: C2: Suppress relocations in scratch emit.
> >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> >> scratch_emit/webrev.01/
> >>>>>>
> >>>>>> Will send RFR soon (depends on 8166560):
> >>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
> >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
> >> scratch_emit/webrev.01
> >>>>>
> >>>>>
> >>>>> Wrong link.
> >>>>>
> >>>>> Thanks,
> >>>>> Vladimir
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> We are still investigating the need of these shared changes:
> >>>>>>
> >>>>>>
> >>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >> 011-pass_PC_to_retAddrOffset.patch
> >>>>>>
> >>>>>>
> >>>>>>
> >>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >> 016-constant_table_offset.patch
> >>>>>>
> >>>>>>
> >>>>>> And finally the patch with the s390x-only platform files. We are still
> >>>>>> editing these to get them into OpenJdk style and shape.
> >>>>>> Hotspot passes most jck, jtreg and spec tests with these.
> >>>>>>
> >>>>>>
> >>
> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
> >> 101-zFiles.patch
> >>>>>>
> >>>>>>
> >>>>>> top-level repository:
> >>>>>> ===============
> >>>>>>
> >>>>>> The following is just adding some s390x specific compiler flags to
> >>>>>> flags.m4
> >>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
> >>>>>>
> >>>>>> jdk repository:
> >>>>>> ============
> >>>>>>
> >>>>>> This one just adds a new jvm.cfg file for s390x
> >>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
> >>>>>>
> >>>>>>
> >>>>>> And finally we plan to do one more change which fixes the jtreg test
> >>>>>> on Linux/s390x. But this is mainly for the correct detection of the
> >>>>>> platform and for excluding the tests which are not appropriate for
> >>>>>> s390x.
> >>>>>>
> >>>>>> Thank you and best regards,
> >>>>>> Volker
> >>>>>>
> >>>>>

From adinn at redhat.com  Thu Oct 13 16:40:34 2016
From: adinn at redhat.com (Andrew Dinn)
Date: Thu, 13 Oct 2016 17:40:34 +0100
Subject: Problem with multi-release jars, agents and the bootstrap class path
Message-ID: <06f196fb-d9d8-f5f8-5363-0453fce96e53@redhat.com>

Disclaimer:

Although this post relates to handling of multi-release jars I'm
cross-posting to jigsaw-dev as well as core-libs-dev. That's because the
problem arises only because my JVMTI Java agent now needs to operate in
dual mode to cope with Jigsaw when deployed on JDK9+ and to avoid
reference to Jigsaw when deployed on JDK8-, hence the desire to use the
multi-release format. Apologies if the cross-post is redundant.

The Problem:

I have re-packaged my Byteman JVMTI Java agent using a multi-release jar
format so that it can operate as is on JDK8- (compiled as target 6 and
no reference to Jigsaw code) while providing revised functionality on
JDK9 that accounts for the presence of modules. This works fine if I
deploy the agent on the system classpath.

However, for some modes of operation my agent Main class hoists the
agent jar into the bootstrap classpath. This is needed so that I can
inject references to agent classes into JDK bootstrap code like Thread
etc. The problem is that bootstrap loader does not appear to recognise
that the jar is in multi-release format.

I can understand that this may seem to be an appropriate behaviour under
the assumption that code in the bootstrap ought to be the JDK runtime
code and therefore ought to be compiled at the current JDK/JVM version.
However, that assumption doesn't really hold when the code in question
is an agent which may well need to operate on old and new JDK versions.


A Few Questions:

Am I right that this is a deliberate choice of bootstrap loader
behaviour? Or is there perhaps some other bug here?

If this is a deliberate choice is there any possibility of reviewing it?

Would it be particularly difficult to modify the boot loader to
recognise jars in multi-release format?

Thanks in advance!

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander

From ljnelson at gmail.com  Thu Oct 13 17:14:48 2016
From: ljnelson at gmail.com (Laird Nelson)
Date: Thu, 13 Oct 2016 17:14:48 +0000
Subject: Process to add some default methods to javax.naming.Context?
Message-ID: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>

Hello; I am new to this mailing list and its conventions; if I misstep I am
happy to be pointed in the right direction.  I was directed here by Mark
Reinhold.

Disclaimer in case it matters: I work for Oracle.

I have long wanted there to be methods in javax.naming.Context like the
following:

// typed from memory off the cuff
default <T> T lookup(final String name, final Class<T> type) {
  return type.cast(this.lookup(name));
}
// do similar things for other lookup* methods here

What would be the steps I would next need to take to help with this?

Thanks in advance for help, time and guidance,
Best,
Laird
--
http://about.me/lairdnelson

From Alan.Bateman at oracle.com  Thu Oct 13 17:27:36 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 13 Oct 2016 18:27:36 +0100
Subject: Problem with multi-release jars, agents and the bootstrap class
	path
In-Reply-To: <06f196fb-d9d8-f5f8-5363-0453fce96e53@redhat.com>
References: <06f196fb-d9d8-f5f8-5363-0453fce96e53@redhat.com>
Message-ID: <831b14f2-49e1-3487-f7c5-13d307dea5e0@oracle.com>

On 13/10/2016 17:40, Andrew Dinn wrote:

> :
>
>
> A Few Questions:
>
> Am I right that this is a deliberate choice of bootstrap loader
> behaviour? Or is there perhaps some other bug here?
>
> If this is a deliberate choice is there any possibility of reviewing it?
>
> Would it be particularly difficult to modify the boot loader to
> recognise jars in multi-release format?
>
>
What you see is deliberate. It wouldn't be impossible for the boot 
loader to support MR JARs but it hardly seems worth the complexity. One 
reason is that it's not a common way to deploy a general purpose 
library. Even if it were then it would be problematic going forward as 
we reduce the number of modules defined to the boot loader (you can't 
instrument code in core modules to call into a SQL library for example 
because java.sql types are not visible to code in the boot loader). The 
other thing is that the boot class path as we used to know it is mostly 
gone, it only really exists when using -Xbootclasspath/a or when JVM TI 
or java.lang.instrument do the equivalent.

For the "hoist"-ing scenario then I assume you mean 
Instrumentation::appendToBootstrapClassLoaderSearch to add supporting 
classes to the boot class path so that they are visible to instrumented 
code. Can you use the same method to append a JAR file containing 
53.0/9+ classes before you append the other JAR. That should be 
equivalent to what a MR JAR would give you, albeit without the 
convenience of everything in one artifact.

-Alan

From peter.levart at gmail.com  Thu Oct 13 17:30:41 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 13 Oct 2016 19:30:41 +0200
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <1F5607C1-9EFC-4881-A3A9-31CB948FE67C@oracle.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<1F5607C1-9EFC-4881-A3A9-31CB948FE67C@oracle.com>
Message-ID: <1a075ccc-bdd9-ffe9-6a7b-2e734c6233e8@gmail.com>

Hi Paul, Alan,

I incorporated Paul's suggestions into new webrev:

http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.05/

This iteration also contains a nearly-exhaustive test of 
Class.getMethods(): PublicMethodsTest. It is actually a test generator. 
Given a Case template, it generates all variants of methods for each of 
the types in the case. Case1 contains 4 interface method variants ^ 3 
interfaces * 4 class method variants ^ 3 classes = 4^6 = 4096 different 
sub-cases of which only 1379 compile. The results of those 1379 
sub-cases are persisted in the Case1.results file. Running the test 
compares the persisted results with actual result of executing each 
sub-case. When running this test on plain JDK 9 (without patch), the 
test finds 218 sub-cases where results differ:

http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/PublicMethodsTest.jtr

Looking at those differences gives the impression of the effects of the 
patch.

Regards, Peter

On 10/11/2016 11:35 PM, Paul Sandoz wrote:
> Hi Peter,
>
> Nice work here.
>
> PublicMethods
> ?
>
> I would be inclined to change PublicMethods to encapsulate an instance 
> of LinkedHashMap, since it?s only the consolidate/toArray methods that 
> really matter, and no need for the key/value types to be exposed, then 
> no need to declare serialVersionUID, and also it can be final. The 
> JavaDoc on consolidate would need to be tweaked.
>
>    29      * existing methods with same signature if they are less specific then added
> s/then/than
>
>    89             if (o == null || getClass() != o.getClass()) return false;
>
> Could be simplified to
>
>   if (!(o instanceof Key)) return false
>
> Paul.
>
>> On 4 Oct 2016, at 06:40, Peter Levart <peter.levart at gmail.com 
>> <mailto:peter.levart at gmail.com>> wrote:
>>
>> Hi,
>>
>> I have a fix for conformance (P2) bug (8062389 
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>) that also fixes a 
>> conformance (P3) bug (8029459 
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>) and a performance 
>> issue (8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>):
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.04/ 
>> <http://cr.openjdk.java.net/%7Eplevart/jdk9-dev/Class.getMethods.new/webrev.04/>
>>
>> As Daniel Smith writes in 8029459 
>> <https://bugs.openjdk.java.net/browse/JDK-8029459>, the following 
>> situation is as expected:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends J {}
>> K.class.getMethods() = [ J.m ]
>>
>> But the following has a different result although it should probably 
>> have the same:
>>
>> interface I { void m(); }
>> interface J extends I { void m(); }
>> interface K extends I, J {}
>> K.class.getMethods() = [ I.m, J.m ]
>>
>> He then suggests the following algorithm:
>>
>> An implementation of getMethods consistent with JLS 8 would include 
>> the following (see Lambda Spec, Part H, 9.4.1 and 8.4.8):
>> 1) The class's/interface's declared (public) methods
>> 2) The getMethods() of the superclass (if this is a class), minus any 
>> that have a match in (1)
>> 3) The getMethods() of each superinterface, minus any that have a 
>> match in (1) or a _concrete_ match in (2) or a match from a 
>> more-specific class/interface in (2) or (3)
>>
>> But even that is not sufficient for the following situation:
>>
>> interface E { void m(); }
>> interface F extends E { void m(); }
>> abstract class G implements E {}
>> abstract class H extends G implements F {}
>> H.class.getMethods() = [ E.m, F.m ]
>>
>> The desired result of H.class.getMethods() = [ F.m ]
>>
>> So a more precise definition is required which is implemented in the fix.
>>
>> The getMethods() implementation partitions the union of the following 
>> methods:
>>
>> 1) The class's/interface's declared public methods
>> 2) The getMethods() of the superclass (if this is a class)
>> 3) The non-static getMethods() of each direct superinterface
>>
>> ...into equivalence classes (sets) of methods with same signature 
>> (return type, name, parameter types). Within each such set, only the 
>> "most specific" methods are kept and others are discarded. The union 
>> of the kept methods is the result.
>>
>> The "more specific" is defined as a partial order within a set of 
>> methods of same signature:
>>
>> Method A is more specific than method B if:
>> - A is declared by a class and B is declared by an interface; or
>> - A is declared by the same type as or a subtype of B's declaring 
>> type and both are either declared by classes or both by interfaces 
>> (clearly if A and B are declared by the same type and have the same 
>> signature, they are the same method)
>>
>> If A and B are distinct elements from the set of methods with same 
>> signature, then either:
>> - A is more specific than B; or
>> - B is more specific than A; or
>> - A and B are incomparable
>>
>> A sub-set of "most specific" methods are the methods from the set 
>> where for each such method M, there is no method N != M such that N 
>> is "more specific" than M.
>>
>> There can be more than one "most specific" method for a particular 
>> signature as they can be inherited from multiple unrelated 
>> interfaces, but:
>> - javac prevents compilation when among multiply inherited methods 
>> with same signature there is at least one default method, so in 
>> practice, getMethods() will only return multiple methods with same 
>> signature if they are abstract interface methods. With one exception: 
>> bridge methods that are created by javac for co-variant overrides are 
>> default methods in interfaces. For example:
>>
>>    interface I { Object m(); }
>>    interface J1 extends I { Map m(); }
>>    interface J2 extends I { HashMap m(); }
>>    interface K extends J1, J2 {}
>>
>> K.class.getMethods() = [ default Object j1.m, abstract Map j1.m, 
>> default Object j2.m, abstract HashMap j2.m ]
>>
>> But this is an extreme case where one can still expect multiple 
>> non-abstract methods with same signature, but different declaring 
>> type, returned from getMethods().
>>
>> In order to also fix 8062389 
>> <https://bugs.openjdk.java.net/browse/JDK-8062389>, getMethod() and 
>> getMethods() share the same consolidation logic that results in a set 
>> of "most specific" methods. The partitioning in getMethods() is 
>> partially performed by collecting Methods into a HashMap where the 
>> keys are (name, parameter types) tuples and values are linked lists 
>> of Method objects with possibly varying return and declaring types. 
>> The consolidation, i.e. keeping only the set of most specific methods 
>> as new methods are encountered, is performed only among methods in 
>> the list that share same return type and also removes duplicates. 
>> getMethod() uses only one such list, consolidates methods that match 
>> given name and parameter types and returns the 1st method from the 
>> resulting list that has the most specific return type.
>>
>> That's it for algorithms used. As partitioning partially happens 
>> using HashMap with (name, parameter types) keys, lists of methods 
>> that form values are typically kept short with most of them 
>> containing only a single method, so this fix also fixes performance 
>> issue 8061950 <https://bugs.openjdk.java.net/browse/JDK-8061950>.
>>
>> The patch also contains some optimizations around redundant copying 
>> of arrays and reflective objects.
>>
>> The FilterNotMostSpecific jtreg test has been updated to accommodate 
>> for changed behavior. Both of the above extreme cases have been added 
>> to the test.
>>
>> So, comments, please.
>>
>> Regards, Peter
>>
>


From brian.goetz at oracle.com  Thu Oct 13 20:39:44 2016
From: brian.goetz at oracle.com (Brian Goetz)
Date: Thu, 13 Oct 2016 16:39:44 -0400
Subject: Process to add some default methods to javax.naming.Context?
In-Reply-To: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
References: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
Message-ID: <1513bcdf-af70-c0ef-ae79-72fdc37ad608@oracle.com>

javax.naming is a JCP-controlled API.  Adding to it would require a 
maintenance release of whichever JSR covers JNDI (probably the Java EE 
umbrella JSR.)  You'll need to track down the spec lead and see if a MR 
is in the works, and if you can convince them that this is a desirable 
addition.

On 10/13/2016 1:14 PM, Laird Nelson wrote:
> Hello; I am new to this mailing list and its conventions; if I misstep I am
> happy to be pointed in the right direction.  I was directed here by Mark
> Reinhold.
>
> Disclaimer in case it matters: I work for Oracle.
>
> I have long wanted there to be methods in javax.naming.Context like the
> following:
>
> // typed from memory off the cuff
> default <T> T lookup(final String name, final Class<T> type) {
>    return type.cast(this.lookup(name));
> }
> // do similar things for other lookup* methods here
>
> What would be the steps I would next need to take to help with this?
>
> Thanks in advance for help, time and guidance,
> Best,
> Laird
> --
> http://about.me/lairdnelson


From adinn at redhat.com  Thu Oct 13 20:43:24 2016
From: adinn at redhat.com (Andrew Dinn)
Date: Thu, 13 Oct 2016 21:43:24 +0100
Subject: Problem with multi-release jars, agents and the bootstrap class
	path
In-Reply-To: <831b14f2-49e1-3487-f7c5-13d307dea5e0@oracle.com>
References: <06f196fb-d9d8-f5f8-5363-0453fce96e53@redhat.com>
	<831b14f2-49e1-3487-f7c5-13d307dea5e0@oracle.com>
Message-ID: <f9db6e82-3769-fb39-65eb-9a0066a2c205@redhat.com>

On 13/10/16 18:27, Alan Bateman wrote:
> What you see is deliberate. It wouldn't be impossible for the boot
> loader to support MR JARs but it hardly seems worth the complexity. One
> reason is that it's not a common way to deploy a general purpose
> library. Even if it were then it would be problematic going forward as
> we reduce the number of modules defined to the boot loader (you can't
> instrument code in core modules to call into a SQL library for example
> because java.sql types are not visible to code in the boot loader). The
> other thing is that the boot class path as we used to know it is mostly
> gone, it only really exists when using -Xbootclasspath/a or when JVM TI
> or java.lang.instrument do the equivalent.

Ok, that's what I was expecting to be the answer :-)

I am aware that the code I add to the bootstrap can only rely on a core
subset of the JDK runtime classes being available. That ought not to be
an issue -- I have deliberately kept the agent's footprint as small as
possible because every time Byteman uses an API it makes it trickier to
instrument the API class. I'd be interested to know what you are hoping
to exclude though, just in case. Am I ok with print stream? file
streams? localhost server sockets? Just checking :-)

> For the "hoist"-ing scenario then I assume you mean
> Instrumentation::appendToBootstrapClassLoaderSearch to add supporting
> classes to the boot class path so that they are visible to instrumented
> code. Can you use the same method to append a JAR file containing
> 53.0/9+ classes before you append the other JAR. That should be
> equivalent to what a MR JAR would give you, albeit without the
> convenience of everything in one artifact.

Yes, by hoisting I mean precisely that. The minor wrinkle that the class
which adds the agent jar to the bootstrap path actually gets loaded from
that same jar (but via the system classpath) is what led me to use  the
term 'hoist'. There is probably a joke in here somewhere about 'my own
petard' but I'll demur.

I could indeed use two jar files -- in fact my last attempt at a JDK9
compatible implementation did precisely that. However, that actually
turns out to be /much/ more inconvenient because it means the agent jar
doing the hoisting needs to be able to work out where the auxiliary jar
is. This results in a lot of configuration grief (because of the need to
cope with command line vs dynamic install and also to make it easy to do
from tools like maven). So I much prefer a one jar solution.

As it happens, I have managed to encapsulate the JDK8- vs JDK9+ variant
functionality behind an interface. This means that I ham able to employ
just one setup class which needs a JDK8- and JDK9+ variant -- it's job
is to provide a static method which populates the interface with a
suitable implementation. So, I /can/ actually resolve this issue by
bundling all the code in one jar but with this variant implemented as
two distinct classes. I will simply load the relevant variant by name +
reflectively invoke it's static setup method conditional on whether or
not the JDK includes Jigsaw.

I was asking not because this problem is not resolvable but because

  I thought the multi-release option was a very neat solution that I
really ought to try to use

  I envisaged that other agent implementors might not find it so easy to
encapsulate the required functionality

It's a shame multi-release won't work for all agents but never mind.

Thanks very much for the quick and helpful response.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander

From xueming.shen at oracle.com  Thu Oct 13 21:03:21 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Thu, 13 Oct 2016 14:03:21 -0700
Subject: RFR JDK-8167957: Remove FilePermission from default policy for
	jdk.charsets module
Message-ID: <57FFF699.7020805@oracle.com>

Hi

Please help  review the change for JDK-8167957

issue: https://bugs.openjdk.java.net/browse/JDK-8167957
webrev: http://cr.openjdk.java.net/~sherman/8167957/webrev

There have been changes in jdk.charsets implementation to move pieces
around for jigsaw. It appears the latest implementation does not require
the read access to java.home at all. So we are proposing to remove this
permision from the default.policy for jdk.charsets.module.

Thanks,
Sherman

From mandy.chung at oracle.com  Thu Oct 13 21:13:57 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 13 Oct 2016 14:13:57 -0700
Subject: RFR JDK-8167957: Remove FilePermission from default policy for
	jdk.charsets module
In-Reply-To: <57FFF699.7020805@oracle.com>
References: <57FFF699.7020805@oracle.com>
Message-ID: <3D9FEEED-35DB-4D3E-9681-80967A6815EE@oracle.com>


> On Oct 13, 2016, at 2:03 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
> 
> Hi
> 
> Please help  review the change for JDK-8167957
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8167957
> webrev: http://cr.openjdk.java.net/~sherman/8167957/webrev

The change looks fine.  Is there any existing test running with security manager to verify this?

Mandy


From xueming.shen at oracle.com  Thu Oct 13 21:19:44 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Thu, 13 Oct 2016 14:19:44 -0700
Subject: RFR JDK-8167957: Remove FilePermission from default policy for
	jdk.charsets module
In-Reply-To: <3D9FEEED-35DB-4D3E-9681-80967A6815EE@oracle.com>
References: <57FFF699.7020805@oracle.com>
	<3D9FEEED-35DB-4D3E-9681-80967A6815EE@oracle.com>
Message-ID: <57FFFA70.1030509@oracle.com>

Yes, there are couple test cases in the cs unit/regression tests running under security manager
when testing.

sherman

On 10/13/2016 02:13 PM, Mandy Chung wrote:
>> On Oct 13, 2016, at 2:03 PM, Xueming Shen<xueming.shen at oracle.com>  wrote:
>>
>> Hi
>>
>> Please help  review the change for JDK-8167957
>>
>> issue: https://bugs.openjdk.java.net/browse/JDK-8167957
>> webrev: http://cr.openjdk.java.net/~sherman/8167957/webrev
> The change looks fine.  Is there any existing test running with security manager to verify this?
>
> Mandy
>


From mandy.chung at oracle.com  Thu Oct 13 21:26:27 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 13 Oct 2016 14:26:27 -0700
Subject: RFR JDK-8167957: Remove FilePermission from default policy for
	jdk.charsets module
In-Reply-To: <57FFFA70.1030509@oracle.com>
References: <57FFF699.7020805@oracle.com>
	<3D9FEEED-35DB-4D3E-9681-80967A6815EE@oracle.com>
	<57FFFA70.1030509@oracle.com>
Message-ID: <CD09320C-0EF4-4F88-860C-49D8AADA6B13@oracle.com>

Thumbs up as long as the tests well cover the cases.

Mandy

> On Oct 13, 2016, at 2:19 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
> 
> Yes, there are couple test cases in the cs unit/regression tests running under security manager
> when testing.
> 
> sherman
> 
> On 10/13/2016 02:13 PM, Mandy Chung wrote:
>>> On Oct 13, 2016, at 2:03 PM, Xueming Shen<xueming.shen at oracle.com>  wrote:
>>> 
>>> Hi
>>> 
>>> Please help  review the change for JDK-8167957
>>> 
>>> issue: https://bugs.openjdk.java.net/browse/JDK-8167957
>>> webrev: http://cr.openjdk.java.net/~sherman/8167957/webrev
>> The change looks fine.  Is there any existing test running with security manager to verify this?
>> 
>> Mandy
>> 
> 


From ecki at zusammenkunft.net  Thu Oct 13 22:09:31 2016
From: ecki at zusammenkunft.net (ecki at zusammenkunft.net)
Date: Fri, 14 Oct 2016 00:09:31 +0200
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
	<d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
Message-ID: <5800061e.423cc20a.87625.39eb@mx.google.com>

The question is, if support for fadvice() would be more flexible (and somewhat saner).

And of course real async fio .)

Gruss
Bernd
-- 
http://bernd.eckenfels.net

Von: Alan Burlison
Gesendet: Donnerstag, 13. Oktober 2016 14:30
An: Brian Burkhalter; Lu, Yingqi
Cc: nio-dev at openjdk.java.net; Kaczmarek, Eric; core-libs-dev at openjdk.java.net; Kharbas, Kishor
Betreff: Re: Proposal for adding O_DIRECT support into JDK 9

On 06/10/2016 00:31, Brian Burkhalter wrote:

> Given that the functionality of O_DIRECT on Linux appears to be
> supported by other interfaces on OS X, Solaris, and Windows, I wonder
> whether the patch will need to be refactored in some way to
> accommodate these other operating systems? For reference it looks as
> if direct I/O on OS X uses the F_NOCACHE command of fcntl(2) [1]
> (although per some online comments this might have some problems),
> Solaris uses the advice argument of directio(3c) [2], and Windows
> uses a combination of flags passed to CreateFile() [3, 4].

The Linux open(2) manpage contains a long list of warnings about 
O_DIRECT, including:

----------
In Linux alignment restrictions vary by filesystem and kernel version 
and might be absent entirely.  However there is currently no 
filesystem-independent interface for an application to discover these 
restrictions for a given file or filesystem.
----------

----------
O_DIRECT I/Os should never be run concurrently with the fork(2) system
call, if the memory buffer is a private mapping (i.e., any
mapping created with the mmap(2) MAP_PRIVATE flag; this includes
memory allocated on the heap and statically allocated buffers). Any
such I/Os, whether submitted via an asynchronous I/O interface or
from another thread in the process, should be completed before
fork(2) is called. Failure to do so can result in data corruption
and undefined behavior in parent and child processes.
----------

----------
Applications should avoid mixing O_DIRECT and normal I/O to the same
file, and especially to overlapping byte regions in the same file.
Even when the filesystem correctly handles the coherency issues in
this situation, overall I/O throughput is likely to be slower than
using either mode alone. Likewise, applications should avoid mixing
mmap(2) of files with direct I/O to the same files.
----------

----------
  "The thing that has always disturbed me about O_DIRECT is that
the whole interface is just stupid, and was probably designed
by a deranged monkey on some serious mind-controlling
substances." - Linus
----------

Adding support for O_DIRECT has a far wider impact than adding just 
another IO handle flag. As such I'm opposed to this change as it seems 
to be prone to cause hard-to-diagnose failures on Linux and it is also 
specific to just Linux.

-- 
Alan Burlison
--


From yingqi.lu at intel.com  Thu Oct 13 22:37:07 2016
From: yingqi.lu at intel.com (Lu, Yingqi)
Date: Thu, 13 Oct 2016 22:37:07 +0000
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
	<d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA5146652@ORSMSX113.amr.corp.intel.com>

O_DIRECT is widely adopted in applications designed for high IO throughput, such as webservers and Databases. It bypasses filesystem cache and Linux readahead buffer which some time slow down the throughput and cause unpredictable IO performance. As an example, we recently measured on our Apache Cassandra database (one of the most popular distributed database systems written in Java) on default buffered IO, lowering readahead buffer from 128KB (default) to 8KB improves throughout by up to 4X. This is a typical usage for DirectIO and we expect to see even greater gains by doing that. Another example is we have enabled Hadoop Distributed File System (HDFS) with O_DIRECT through native calls and also measure significant performance gains running a cloud workload. It would be really important to have O_DIRECT supported by Java so that all Java based applications can take advantage of it.

We agree on the other hand, O_DIRECT has certain limitations. That is why it is normally recommended to be used as a performance option. Our purpose is to enable it inside Java and provide the application writers a uniform and secure way to use it. All the limitations apply to native applications as well. 
1. Regarding to alignment restriction, I cross checked with our kernel experts, and we think it is safe to align the memory buffer used by DirectIO to the kernel page size. 

2. In term of having O_DIRECT I/O running concurrently with fork(2) on privately mapped memory buffer, the issue can be solved by creating DirectIO memory buffer using shmat(2) or mmap(2) with the MAP_SHARED flag. Alternatively, MADV_DONTFORK with madvise(2) can be used on the memory buffer to avoid the issue as well. Details can be find http://man7.org/linux/man-pages/man2/open.2.html

3. Combination of DirectIO and BufferedIO on the same file is not recommended for performance reason. However, we think application writers should be familiar with all of these before using it.

Now, we are planning to do the following changes to the existing patch, hope it will be structured a little better this way :-)
1. Create a function to allocate aligned DirectByteBuffer and use it for Direct I/O (default is to use DirectByteBuffer as well, but not aligned). The buffer will be aligned to page boundary. There are some existing code in Direct-X-Buffer.java.template for VM.isDirectMemoryPageAligned. We will follow this as an example. We think this will address the "extra copy" issue in the last version of the patch.

2. Move all the changes to FileDispatcherImpl.c to Java level. 

3. Remove the changes to FileDescriptor and do the DirectIO check inside FileChannelImpl.

Thanks,
Lucy

>-----Original Message-----
>From: Alan Burlison [mailto:Alan.Burlison at oracle.com]
>Sent: Thursday, October 13, 2016 5:20 AM
>To: Brian Burkhalter <brian.burkhalter at oracle.com>; Lu, Yingqi
><yingqi.lu at intel.com>
>Cc: Kharbas, Kishor <kishor.kharbas at intel.com>; nio-dev at openjdk.java.net;
>core-libs-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>
>Subject: Re: Proposal for adding O_DIRECT support into JDK 9
>
>On 06/10/2016 00:31, Brian Burkhalter wrote:
>
>> Given that the functionality of O_DIRECT on Linux appears to be
>> supported by other interfaces on OS X, Solaris, and Windows, I wonder
>> whether the patch will need to be refactored in some way to
>> accommodate these other operating systems? For reference it looks as
>> if direct I/O on OS X uses the F_NOCACHE command of fcntl(2) [1]
>> (although per some online comments this might have some problems),
>> Solaris uses the advice argument of directio(3c) [2], and Windows uses
>> a combination of flags passed to CreateFile() [3, 4].
>
>The Linux open(2) manpage contains a long list of warnings about O_DIRECT,
>including:
>
>----------
>In Linux alignment restrictions vary by filesystem and kernel version and might be
>absent entirely.  However there is currently no filesystem-independent interface
>for an application to discover these restrictions for a given file or filesystem.
>----------
>
>----------
>O_DIRECT I/Os should never be run concurrently with the fork(2) system call, if
>the memory buffer is a private mapping (i.e., any mapping created with the
>mmap(2) MAP_PRIVATE flag; this includes memory allocated on the heap and
>statically allocated buffers). Any such I/Os, whether submitted via an
>asynchronous I/O interface or from another thread in the process, should be
>completed before
>fork(2) is called. Failure to do so can result in data corruption and undefined
>behavior in parent and child processes.
>----------
>
>----------
>Applications should avoid mixing O_DIRECT and normal I/O to the same file, and
>especially to overlapping byte regions in the same file.
>Even when the filesystem correctly handles the coherency issues in this situation,
>overall I/O throughput is likely to be slower than using either mode alone.
>Likewise, applications should avoid mixing
>mmap(2) of files with direct I/O to the same files.
>----------
>
>----------
>  "The thing that has always disturbed me about O_DIRECT is that the whole
>interface is just stupid, and was probably designed by a deranged monkey on
>some serious mind-controlling substances." - Linus
>----------
>
>Adding support for O_DIRECT has a far wider impact than adding just another IO
>handle flag. As such I'm opposed to this change as it seems to be prone to cause
>hard-to-diagnose failures on Linux and it is also specific to just Linux.
>
>--
>Alan Burlison
>--

From huizhe.wang at oracle.com  Thu Oct 13 22:49:37 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Thu, 13 Oct 2016 15:49:37 -0700
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java
	failed with "java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
Message-ID: <58000F81.9010704@oracle.com>

Hi Frank,

Does this work as expected? The method doesn't seem to validate whether 
a character is legal as a file name. For example, on Windows, the 
original char (e.g. \u0159) used in the test is legal in a file name, 
but it didn't pass that decode/encode test by the Windows' default 
encoding (Windows-1252). Does that mean this test will no longer run on 
Windows?

Thanks,
Joe

On 10/13/16, 2:05 AM, Frank Yuan wrote:
>
> Hi all
>
> Would you like to review 
> http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/ 
> <http://cr.openjdk.java.net/%7Efyuan/8167478/webrev.00/> ?
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
>
> This is a test bug, because Bug6341770.java is invalid when the system 
> environment doesn't support non-ascii characters, the test will exit 
> immediately in this condition.
>
> Thanks,
>
> Frank
>

From david.dehaven at oracle.com  Thu Oct 13 23:08:35 2016
From: david.dehaven at oracle.com (David DeHaven)
Date: Thu, 13 Oct 2016 16:08:35 -0700
Subject: [9] RfR: 8165271: Fix use of reflection to gain access to private
	fields
Message-ID: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>


JBS Issue:
https://bugs.openjdk.java.net/browse/JDK-8165271

Webrev:
http://cr.openjdk.java.net/~ddehaven/8165271/jdk.0/

Synopsis:
An upcoming change in the module system will prevent setAccessible calls from working across module boundaries. This patch allows the use of SharedSecrets instead to preserve the underlying logic.

-DrD-


From brian.burkhalter at oracle.com  Thu Oct 13 23:37:34 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Thu, 13 Oct 2016 16:37:34 -0700
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <5800061e.423cc20a.87625.39eb@mx.google.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<b7820438-db18-6c82-37f4-758ccbbcaf77@oracle.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
	<d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
	<5800061e.423cc20a.87625.39eb@mx.google.com>
Message-ID: <6977846B-882C-480B-9F55-776E3DA31EB0@oracle.com>

An interesting comparison of approaches on a Linux server and an OS X client is given here:

http://adityaramesh.com/io_benchmark/

Brian

On Oct 13, 2016, at 3:09 PM, ecki at zusammenkunft.net wrote:

> The question is, if support for fadvice() would be more flexible (and somewhat saner).
> 
> And of course real async fio .)


From mandy.chung at oracle.com  Fri Oct 14 00:24:13 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 13 Oct 2016 17:24:13 -0700
Subject: [9] RfR: 8165271: Fix use of reflection to gain access to private
	fields
In-Reply-To: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>
References: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>
Message-ID: <F45E69FA-DDE3-48BA-9EBC-BA753F9C1997@oracle.com>


> On Oct 13, 2016, at 4:08 PM, David DeHaven <david.dehaven at oracle.com> wrote:
> 
> 
> JBS Issue:
> https://bugs.openjdk.java.net/browse/JDK-8165271
> 
> Webrev:
> http://cr.openjdk.java.net/~ddehaven/8165271/jdk.0/

This change looks fine.  This hack should have been replaced long ago and happy to see them gone.

Mandy


From vladimir.kozlov at oracle.com  Fri Oct 14 01:13:47 2016
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Thu, 13 Oct 2016 18:13:47 -0700
Subject: RFR: JEP draft for Linux/s3990x port
In-Reply-To: <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap>
References: <CA+3eh11Y8kP6AQ6tBsuggOx=dGHt6dyS08dmn09AcFSa1X8q3A@mail.gmail.com>
	<72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com>
	<CA+3eh10XP=-P-=jarS_RJWgLa+WkC+gWEAV8GYsUh4cWCbZNxA@mail.gmail.com>
	<6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com>
	<69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com>
	<b1f4d6b4f6cb40498d38d62603bcd2b2@DEWDFE13DE50.global.corp.sap>
	<f5d2425a-0008-c7c8-69e1-23ec098cb07c@oracle.com>
	<2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap>
Message-ID: <5800314B.5050206@oracle.com>

Thank you, Goetz

I submitted testing.

Vladimir

On 10/13/16 9:22 AM, Lindenmaier, Goetz wrote:
> Hi Vladimir,
>
> I made a new webrev containing all outstanding changes merged into one patch
> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/
>
> You probably saw my RFR with the s390 files.
>
> Best regards,
>    Goetz.
>
>> -----Original Message-----
>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com]
>> Sent: Thursday, October 13, 2016 1:09 AM
>> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Volker Simonis
>> <volker.simonis at gmail.com>
>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build-
>> dev <build-dev at openjdk.java.net>; HotSpot Open Source Developers
>> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
>> dev at openjdk.java.net>
>> Subject: Re: RFR: JEP draft for Linux/s3990x port
>>
>> Hi Goetz and Volker,
>>
>> Positive news is that JEP status is moving, changes are reviewed and some
>> changes were already pushed.
>>
>> But because of our testing issues during past week I was not able to execute
>> the testing.
>>
>> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge
>> hs-comp repository first. hs/ repo will be
>> opened for other pushes soon after that.
>>
>> I added estimated integration date to the JEP (Oct 28). We would like to test
>> and integrate this port before JDK 10
>> forest is forked. Do you think all s390 changes and new code will be ready by
>> that date?
>>
>> Do you have all shared changes reviewed and approved for push?
>>
>> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare
>> changeset based on hs/ repo for changes which
>> are not pushed yet? I will try to submit testing over weekend.
>>
>> Regards,
>> Vladimir
>>
>> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote:
>>> Hi Vladimir,
>>>
>>> This webrev contains all the changes to hotspot needed for the port:
>>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-
>> all/hotspot.wr01/
>>>
>>> It includes
>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>> basic_s390/hotspot.wr03/
>>> http://cr.openjdk.java.net/~goetz/wr16/8166561-
>> basic_C1C2_s390/webrev.01/
>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>> scratch_emit/webrev.01/
>>> which are out for review. Further it includes
>>> the one change to relocate the pc-relative instructions where we didn't
>> open
>>> a bug for yet, and the new s390-files.
>>>
>>> Altogether this passed all our tests that were running on the weekend
>>> on linuxs390.
>>>
>>> The s390-files though are not yet fully in shape, I'm still editing them to get
>>> rid of legacy stuff and SAP JVM specific code.  E.g. all the code guarded by
>>> #ifdef SAPJVM  will go away in the end.
>>>
>>> I hope to have the final versions by end of this week.
>>>
>>> Best regards,
>>>    Goetz.
>>>
>>>
>>>> -----Original Message-----
>>>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net]
>>>> On Behalf Of Vladimir Kozlov
>>>> Sent: Montag, 3. Oktober 2016 23:50
>>>> To: Volker Simonis <volker.simonis at gmail.com>
>>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net;
>>>> build-dev <build-dev at openjdk.java.net>; HotSpot Open Source
>> Developers
>>>> <hotspot-dev at openjdk.java.net>; Java Core Libs <core-libs-
>>>> dev at openjdk.java.net>
>>>> Subject: Re: RFR: JEP draft for Linux/s3990x port
>>>>
>>>> Hi Volker,
>>>>
>>>> Can you prepare combined patch (or set of patches) based on latest
>>>> reviews together with s390 code as it will be in final push?
>>>>
>>>> We want to run it through our pre-integration testing to verify that it
>>>> does not have problems.
>>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote:
>>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or
>>>>> other) state:
>>>>>
>>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html
>>>>>
>>>>> Vladimir
>>>>>
>>>>> On 9/29/16 9:55 AM, Volker Simonis wrote:
>>>>>> Hi Vladimir,
>>>>>>
>>>>>> thanks a lot for reviewing and endorsing the JEP.
>>>>>>
>>>>>> I've linked all the relevant issues to the JEP  (they all have a link
>>>>>> to a webrev) and change the state to "Submitted".
>>>>>>
>>>>>> There's just one more small shared change we need for the port for
>>>>>> which we haven't opened a bug now because we are still working on
>>>>>> simplifying it. The current version looks as follows:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-
>>>> constant_table_offset.patch
>>>>>>
>>>>>>
>>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t
>>>>>> he JEP and add a corresponding "FC Extension Request" comment to it?
>>>>>> Or will this be done automatically once I move it to "Candidate"?
>>>>>>
>>>>>> Is there anything left to do before I can move it to "Candidate" state?
>>>>>>
>>>>>> Thanks a lot and best regards,
>>>>>> Volker
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov
>>>>>> <vladimir.kozlov at oracle.com> wrote:
>>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> can you please review and endorse the following draft JEP for the
>>>>>>>> integration of the Linux/s390x port into the jkd9 master repository:
>>>>>>>>
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730
>>>>>>>
>>>>>>>
>>>>>>> Good.
>>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC
>>>>>>> extension
>>>>>>> approval.
>>>>>>>
>>>>>>>>
>>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared
>>>>>>>> changes and we therefore don't foresee any impact on the existing
>>>>>>>> platforms at all. Following you can find a short description of the
>>>>>>>> planned changes:
>>>>>>>>
>>>>>>>> hotspot:
>>>>>>>> =======
>>>>>>>>
>>>>>>>> Out for review:
>>>>>>>> 8166560: [s390] Basic enablement of s390 port.
>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-
>>>> basic_s390/hotspot.wr01/
>>>>>>>>
>>>>>>>> Reviewed:
>>>>>>>> 8166562: C2: Suppress relocations in scratch emit.
>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>>>> scratch_emit/webrev.01/
>>>>>>>>
>>>>>>>> Will send RFR soon (depends on 8166560):
>>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2.
>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-
>>>> scratch_emit/webrev.01
>>>>>>>
>>>>>>>
>>>>>>> Wrong link.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Vladimir
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> We are still investigating the need of these shared changes:
>>>>>>>>
>>>>>>>>
>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>>> 011-pass_PC_to_retAddrOffset.patch
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>>> 016-constant_table_offset.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> And finally the patch with the s390x-only platform files. We are still
>>>>>>>> editing these to get them into OpenJdk style and shape.
>>>>>>>> Hotspot passes most jck, jtreg and spec tests with these.
>>>>>>>>
>>>>>>>>
>>>>
>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000
>>>> 101-zFiles.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> top-level repository:
>>>>>>>> ===============
>>>>>>>>
>>>>>>>> The following is just adding some s390x specific compiler flags to
>>>>>>>> flags.m4
>>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800
>>>>>>>>
>>>>>>>> jdk repository:
>>>>>>>> ============
>>>>>>>>
>>>>>>>> This one just adds a new jvm.cfg file for s390x
>>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801
>>>>>>>>
>>>>>>>>
>>>>>>>> And finally we plan to do one more change which fixes the jtreg test
>>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the
>>>>>>>> platform and for excluding the tests which are not appropriate for
>>>>>>>> s390x.
>>>>>>>>
>>>>>>>> Thank you and best regards,
>>>>>>>> Volker
>>>>>>>>
>>>>>>>

From frank.yuan at oracle.com  Fri Oct 14 02:10:20 2016
From: frank.yuan at oracle.com (Frank Yuan)
Date: Fri, 14 Oct 2016 10:10:20 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <12733BE7-D427-43F2-891B-4CE3BF291791@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<12733BE7-D427-43F2-891B-4CE3BF291791@oracle.com>
Message-ID: <00bb01d225c0$1f1da230$5d58e690$@oracle.com>

Thank you, will change it.

Frank

> -----Original Message-----
> From: Felix Yang [mailto:felix.yang at oracle.com]
> Subject: Re: RFR (JAXP) JDK-8167478 javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
"java.security.AccessControlException:
> access denied ("java.io.FilePermission" "sko?ice")"
> 
> A comment on naming, alpha to ALPHA ?
> -Felix
> 
> > ? 2016?10?13??17:05?Frank Yuan <frank.yuan at oracle.com> ???
> >
> > Hi all
> >
> >
> >
> > Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/ ?
> >
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
> >
> >
> >
> > This is a test bug, because Bug6341770.java is invalid when the system environment doesn't support non-ascii characters, the
test
> > will exit immediately in this condition.
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Frank
> >
> >
> >



From li.jiang at oracle.com  Fri Oct 14 02:53:55 2016
From: li.jiang at oracle.com (Leo Jiang)
Date: Fri, 14 Oct 2016 10:53:55 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <58000F81.9010704@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<58000F81.9010704@oracle.com>
Message-ID: <09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>

Hi Frank,

I suggest you to use the following code snippet to make your code more readable.

         CharsetEncoder charsetEncoder = Charset.defaultCharset().newEncoder();
         boolean b = charsetEncoder.canEncode(path);// alpha

Any character in Unicode range would be valid as path if current OS charset is UTF-8 (Unicode series encoding). But if 
current encoding setting (code page in Windows) is not UTF, e.g. cp936, the string in Java with inside UTF-16 
representation  need to be mapped to the local encoding, that might be failed.

After modification, the test should be passed on UTF-8 encoding, and the encoding which support the character '\u0159'.

Thanks,
Leo

On 10/14/2016 06:49 AM, Joe Wang wrote:
> Hi Frank,
>
> Does this work as expected? The method doesn't seem to validate whether a character is legal as a file name. For
> example, on Windows, the original char (e.g. \u0159) used in the test is legal in a file name, but it didn't pass that
> decode/encode test by the Windows' default encoding (Windows-1252). Does that mean this test will no longer run on Windows?
>
> Thanks,
> Joe
>
> On 10/13/16, 2:05 AM, Frank Yuan wrote:
>>
>> Hi all
>>
>>
>>
>> Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/
>> <http://cr.openjdk.java.net/%7Efyuan/8167478/webrev.00/> ?
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
>>
>>
>>
>> This is a test bug, because Bug6341770.java is invalid when the system environment doesn?t support non-ascii
>> characters, the test will exit immediately in this condition.
>>
>>
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Frank
>>
>>
>>

From Alan.Bateman at oracle.com  Fri Oct 14 08:24:35 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 14 Oct 2016 09:24:35 +0100
Subject: Problem with multi-release jars, agents and the bootstrap class
	path
In-Reply-To: <f9db6e82-3769-fb39-65eb-9a0066a2c205@redhat.com>
References: <06f196fb-d9d8-f5f8-5363-0453fce96e53@redhat.com>
	<831b14f2-49e1-3487-f7c5-13d307dea5e0@oracle.com>
	<f9db6e82-3769-fb39-65eb-9a0066a2c205@redhat.com>
Message-ID: <31775319-80ca-ff26-0749-a19691c1459e@oracle.com>

On 13/10/2016 21:43, Andrew Dinn wrote:

> :
> I am aware that the code I add to the bootstrap can only rely on a core
> subset of the JDK runtime classes being available. That ought not to be
> an issue -- I have deliberately kept the agent's footprint as small as
> possible because every time Byteman uses an API it makes it trickier to
> instrument the API class. I'd be interested to know what you are hoping
> to exclude though, just in case. Am I ok with print stream? file
> streams? localhost server sockets? Just checking :-)
The granularity is module. It would be great if we could somehow get to 
the point where the only module defined to the boot loader is java.base 
but that is a long way off (java.desktop is the biggie that would need 
to move before its dependences can move). If it were just java.base then 
the "core" APIs would be visible, this includes file and networking.

> :
> Yes, by hoisting I mean precisely that. The minor wrinkle that the class
> which adds the agent jar to the bootstrap path actually gets loaded from
> that same jar (but via the system classpath) is what led me to use  the
> term 'hoist'. There is probably a joke in here somewhere about 'my own
> petard' but I'll demur.
LOL but yes, major hazards here as presumably you have types of the same 
name defined to both the system and boot loaders.


> :
>
> It's a shame multi-release won't work for all agents but never mind.
>
I see Paul has updated the text in JEP 238 to make this clear.

-Alan

From Alan.Burlison at oracle.com  Fri Oct 14 08:28:12 2016
From: Alan.Burlison at oracle.com (Alan Burlison)
Date: Fri, 14 Oct 2016 09:28:12 +0100
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA5146652@ORSMSX113.amr.corp.intel.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
	<d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5146652@ORSMSX113.amr.corp.intel.com>
Message-ID: <715e7654-8037-8688-73b2-35b002307a23@oracle.com>

As has been pointed out already, platforms other than Linux provide 
similar functionality, yet this proposal does not provide any support 
for them. Even if such support isn't added initially, the APIs added for 
the Linux implementation should be generic enough that direct IO support 
could be added for other platforms in the future, and that Java code 
using direct IO on Linux would work on other direct IO enabled platforms 
as well. My concern is that the proposed APIs are too Linux-specific to 
make that possible.

I'm well aware of the issues in things like Hadoop and Apache Spark, 
their over-reliance on platform-specific functionality and native code 
has made it extremely difficult to get them to work well on platforms 
other than Linux/x86. We should not make the same mistake in Java itself.

-- 
Alan Burlison
--

From frank.yuan at oracle.com  Fri Oct 14 09:28:20 2016
From: frank.yuan at oracle.com (Frank Yuan)
Date: Fri, 14 Oct 2016 17:28:20 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<58000F81.9010704@oracle.com>
	<09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>
Message-ID: <01a201d225fd$4f5c7a60$ee156f20$@oracle.com>

Hi Joe

After some testing, I am sure current Windows platforms support Unicode although the default charset is Windows-1252. So now I check it by trying Paths.get method, which was also suggested by Max.

Please check the new webrev http://cr.openjdk.java.net/~fyuan/8167478/webrev.01/

Thanks
Frank

> -----Original Message-----
> From: Leo Jiang [mailto:li.jiang at oracle.com]
> Subject: Re: RFR (JAXP) JDK-8167478 javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with "java.security.AccessControlException:
> access denied ("java.io.FilePermission" "sko?ice")"
> 
> Hi Frank,
> 
> I suggest you to use the following code snippet to make your code more readable.
> 
>          CharsetEncoder charsetEncoder = Charset.defaultCharset().newEncoder();
>          boolean b = charsetEncoder.canEncode(path);// alpha
> 
> Any character in Unicode range would be valid as path if current OS charset is UTF-8 (Unicode series encoding). But if
> current encoding setting (code page in Windows) is not UTF, e.g. cp936, the string in Java with inside UTF-16
> representation  need to be mapped to the local encoding, that might be failed.
> 
> After modification, the test should be passed on UTF-8 encoding, and the encoding which support the character '\u0159'.
> 
> Thanks,
> Leo
> 
> On 10/14/2016 06:49 AM, Joe Wang wrote:
> > Hi Frank,
> >
> > Does this work as expected? The method doesn't seem to validate whether a character is legal as a file name. For
> > example, on Windows, the original char (e.g. \u0159) used in the test is legal in a file name, but it didn't pass that
> > decode/encode test by the Windows' default encoding (Windows-1252). Does that mean this test will no longer run on Windows?
> >
> > Thanks,
> > Joe
> >
> > On 10/13/16, 2:05 AM, Frank Yuan wrote:
> >>
> >> Hi all
> >>
> >>
> >>
> >> Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/
> >> <http://cr.openjdk.java.net/%7Efyuan/8167478/webrev.00/> ?
> >>
> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
> >>
> >>
> >>
> >> This is a test bug, because Bug6341770.java is invalid when the system environment doesn?t support non-ascii
> >> characters, the test will exit immediately in this condition.
> >>
> >>
> >>
> >>
> >>
> >> Thanks,
> >>
> >>
> >>
> >> Frank
> >>
> >>
> >>


From Alan.Bateman at oracle.com  Fri Oct 14 12:16:38 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 14 Oct 2016 13:16:38 +0100
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <1a075ccc-bdd9-ffe9-6a7b-2e734c6233e8@gmail.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<1F5607C1-9EFC-4881-A3A9-31CB948FE67C@oracle.com>
	<1a075ccc-bdd9-ffe9-6a7b-2e734c6233e8@gmail.com>
Message-ID: <0c67ae84-7bab-5944-1665-129dc451f3da@oracle.com>

On 13/10/2016 18:30, Peter Levart wrote:

> Hi Paul, Alan,
>
> I incorporated Paul's suggestions into new webrev:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.05/
>
> This iteration also contains a nearly-exhaustive test of 
> Class.getMethods(): PublicMethodsTest. It is actually a test 
> generator. Given a Case template, it generates all variants of methods 
> for each of the types in the case. Case1 contains 4 interface method 
> variants ^ 3 interfaces * 4 class method variants ^ 3 classes = 4^6 = 
> 4096 different sub-cases of which only 1379 compile. The results of 
> those 1379 sub-cases are persisted in the Case1.results file. Running 
> the test compares the persisted results with actual result of 
> executing each sub-case. When running this test on plain JDK 9 
> (without patch), the test finds 218 sub-cases where results differ:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/PublicMethodsTest.jtr
>
> Looking at those differences gives the impression of the effects of 
> the patch.
Overall I think this looks very good. I mostly focused on the 
PublicMethods implementation to satisfy myself that it does selects the 
most specific methods. In passing I wonder if "combine" or "merge" might 
be better than "consolidate" for method name and terminology, a minor 
point of course.

Given the behavior change then I think we'll need to capture it in a 
release notes. I can't think of any libraries or frameworks that might 
have see but something might come out of the woodwork and would be nice 
to be able to point to a summary.

I assume that copyright headers will be added before this is pushed. 
Also there is a @SuppressWarnings arguments that I don't recognize (IDE 
specific)? It would good to trim back some of the really long times in 
the test too (there are some >150 char lines that will be a pain to 
review side-by-side when there are future changes).

-Alan

From vincent.x.ryan at oracle.com  Fri Oct 14 14:20:21 2016
From: vincent.x.ryan at oracle.com (Vincent Ryan)
Date: Fri, 14 Oct 2016 15:20:21 +0100
Subject: Process to add some default methods to javax.naming.Context?
In-Reply-To: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
References: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
Message-ID: <6D80774B-75DF-419E-A3A6-9DF2C7AFD91A@oracle.com>

Hello Laird,

The problem of having to cast the result of Context.lookup <http://download.java.net/java/jdk9/docs/api/javax/naming/Context.html#lookup-javax.naming.Name-> can be solved using an existing method:
    InitialContext.doLookup <http://download.java.net/java/jdk9/docs/api/javax/naming/InitialContext.html#doLookup-javax.naming.Name->

Although it might not suit all uses because no JNDI environment properties are passed to the InitialContext
constructor that gets created within the method.



> On 13 Oct 2016, at 18:14, Laird Nelson <ljnelson at gmail.com> wrote:
> 
> Hello; I am new to this mailing list and its conventions; if I misstep I am
> happy to be pointed in the right direction.  I was directed here by Mark
> Reinhold.
> 
> Disclaimer in case it matters: I work for Oracle.
> 
> I have long wanted there to be methods in javax.naming.Context like the
> following:
> 
> // typed from memory off the cuff
> default <T> T lookup(final String name, final Class<T> type) {
>  return type.cast(this.lookup(name));
> }
> // do similar things for other lookup* methods here
> 
> What would be the steps I would next need to take to help with this?
> 
> Thanks in advance for help, time and guidance,
> Best,
> Laird
> --
> http://about.me/lairdnelson


From yingqi.lu at intel.com  Fri Oct 14 17:16:21 2016
From: yingqi.lu at intel.com (Lu, Yingqi)
Date: Fri, 14 Oct 2016 17:16:21 +0000
Subject: Proposal for adding O_DIRECT support into JDK 9
In-Reply-To: <715e7654-8037-8688-73b2-35b002307a23@oracle.com>
References: <9ACD5B67AAC5594CB6268234CF29CF9AA510A5DA@ORSMSX113.amr.corp.intel.com>
	<D59AA10C-E9AB-4F70-A209-3C313F1340C6@intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5134F66@ORSMSX113.amr.corp.intel.com>
	<062c589c-fa54-1652-683a-4e779816391d@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513500C@ORSMSX113.amr.corp.intel.com>
	<542288c7-b0b1-5b41-4c45-d07730e3cddf@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5135D27@ORSMSX113.amr.corp.intel.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5139A26@ORSMSX113.amr.corp.intel.com>
	<b9c37057f6c74193abef35f59c8eafec@DEWDFE13DE11.global.corp.sap>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513CE96@ORSMSX113.amr.corp.intel.com>
	<45C5D272-E70A-40D9-8166-48ADD43763D9@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA513D886@ORSMSX113.amr.corp.intel.com>
	<DAC23BAE-E887-4621-897E-AA22617FE09A@oracle.com>
	<d365df5b-6f3f-4ac1-0d7d-9c8f31e199f1@oracle.com>
	<9ACD5B67AAC5594CB6268234CF29CF9AA5146652@ORSMSX113.amr.corp.intel.com>
	<715e7654-8037-8688-73b2-35b002307a23@oracle.com>
Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA514738B@ORSMSX113.amr.corp.intel.com>

Alan,

Thank you for the feedback. We will come up with a more generic API that is flexible for supporting all the platforms that have DirectIO. Will send it here for review soon.

Thanks,
Lucy

>-----Original Message-----
>From: Alan Burlison [mailto:Alan.Burlison at oracle.com]
>Sent: Friday, October 14, 2016 1:28 AM
>To: Lu, Yingqi <yingqi.lu at intel.com>; Brian Burkhalter
><brian.burkhalter at oracle.com>
>Cc: Kharbas, Kishor <kishor.kharbas at intel.com>; nio-dev at openjdk.java.net;
>core-libs-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>
>Subject: Re: Proposal for adding O_DIRECT support into JDK 9
>
>As has been pointed out already, platforms other than Linux provide similar
>functionality, yet this proposal does not provide any support for them. Even if
>such support isn't added initially, the APIs added for the Linux implementation
>should be generic enough that direct IO support could be added for other
>platforms in the future, and that Java code using direct IO on Linux would work
>on other direct IO enabled platforms as well. My concern is that the proposed
>APIs are too Linux-specific to make that possible.
>
>I'm well aware of the issues in things like Hadoop and Apache Spark, their over-
>reliance on platform-specific functionality and native code has made it extremely
>difficult to get them to work well on platforms other than Linux/x86. We should
>not make the same mistake in Java itself.
>
>--
>Alan Burlison
>--

From huizhe.wang at oracle.com  Fri Oct 14 17:36:43 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Fri, 14 Oct 2016 10:36:43 -0700
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java
	failed with "java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <01a201d225fd$4f5c7a60$ee156f20$@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<58000F81.9010704@oracle.com>
	<09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>
	<01a201d225fd$4f5c7a60$ee156f20$@oracle.com>
Message-ID: <580117AB.9020603@oracle.com>

Looks fine. Paths.get is the way to go.

-Joe

On 10/14/16, 2:28 AM, Frank Yuan wrote:
> Hi Joe
>
> After some testing, I am sure current Windows platforms support Unicode although the default charset is Windows-1252. So now I check it by trying Paths.get method, which was also suggested by Max.
>
> Please check the new webrev http://cr.openjdk.java.net/~fyuan/8167478/webrev.01/
>
> Thanks
> Frank
>
>> -----Original Message-----
>> From: Leo Jiang [mailto:li.jiang at oracle.com]
>> Subject: Re: RFR (JAXP) JDK-8167478 javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with "java.security.AccessControlException:
>> access denied ("java.io.FilePermission" "sko?ice")"
>>
>> Hi Frank,
>>
>> I suggest you to use the following code snippet to make your code more readable.
>>
>>           CharsetEncoder charsetEncoder = Charset.defaultCharset().newEncoder();
>>           boolean b = charsetEncoder.canEncode(path);// alpha
>>
>> Any character in Unicode range would be valid as path if current OS charset is UTF-8 (Unicode series encoding). But if
>> current encoding setting (code page in Windows) is not UTF, e.g. cp936, the string in Java with inside UTF-16
>> representation  need to be mapped to the local encoding, that might be failed.
>>
>> After modification, the test should be passed on UTF-8 encoding, and the encoding which support the character '\u0159'.
>>
>> Thanks,
>> Leo
>>
>> On 10/14/2016 06:49 AM, Joe Wang wrote:
>>> Hi Frank,
>>>
>>> Does this work as expected? The method doesn't seem to validate whether a character is legal as a file name. For
>>> example, on Windows, the original char (e.g. \u0159) used in the test is legal in a file name, but it didn't pass that
>>> decode/encode test by the Windows' default encoding (Windows-1252). Does that mean this test will no longer run on Windows?
>>>
>>> Thanks,
>>> Joe
>>>
>>> On 10/13/16, 2:05 AM, Frank Yuan wrote:
>>>> Hi all
>>>>
>>>>
>>>>
>>>> Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/
>>>> <http://cr.openjdk.java.net/%7Efyuan/8167478/webrev.00/>  ?
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
>>>>
>>>>
>>>>
>>>> This is a test bug, because Bug6341770.java is invalid when the system environment doesn?t support non-ascii
>>>> characters, the test will exit immediately in this condition.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>>
>>>> Frank
>>>>
>>>>
>>>>

From Roger.Riggs at Oracle.com  Fri Oct 14 21:03:05 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 14 Oct 2016 17:03:05 -0400
Subject: RFR 9: 8167166 : Java API docs mention a non-existent method
	getNanosOfSecond
Message-ID: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>

Please review a corrected javadoc link correction in java.time.Instant.

Webrev:
http://bussund0416.us.oracle.com/~rriggs/webrev/webrev-nano-8167166/

Thanks, Roger


From naoto.sato at oracle.com  Fri Oct 14 21:07:28 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Fri, 14 Oct 2016 14:07:28 -0700
Subject: RFR 9: 8167166 : Java API docs mention a non-existent method
	getNanosOfSecond
In-Reply-To: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>
References: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>
Message-ID: <e9a27fa1-e82a-68a5-30d6-c51492c68121@oracle.com>

Looks good, Roger.

Naoto

On 10/14/16 2:03 PM, Roger Riggs wrote:
> Please review a corrected javadoc link correction in java.time.Instant.
>
> Webrev:
> http://bussund0416.us.oracle.com/~rriggs/webrev/webrev-nano-8167166/
>
> Thanks, Roger
>

From mark.reinhold at oracle.com  Fri Oct 14 21:10:09 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 14 Oct 2016 14:10:09 -0700
Subject: Process to add some default methods to javax.naming.Context?
In-Reply-To: <1513bcdf-af70-c0ef-ae79-72fdc37ad608@oracle.com>
References: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
	<1513bcdf-af70-c0ef-ae79-72fdc37ad608@oracle.com>
Message-ID: <20161014141009.948397736eggemoggin.niobe.net>

2016/10/13 13:39:44 -0700, brian.goetz at oracle.com:
> javax.naming is a JCP-controlled API.  Adding to it would require a 
> maintenance release of whichever JSR covers JNDI (probably the Java EE 
> umbrella JSR.)  You'll need to track down the spec lead and see if a MR 
> is in the works, and if you can convince them that this is a desirable 
> addition.

JNDI was merged into the JDK before the platform as a whole was governed
by the JCP, so there's never been a separate JSR for it.  Any changes to
the JNDI API would therefore be covered by the relevant Java SE Umbrella
JSR unless they were so significant as to require their own JSR -- which
at this stage seems pretty unlikely.

- Mark

From Roger.Riggs at Oracle.com  Fri Oct 14 21:13:29 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 14 Oct 2016 17:13:29 -0400
Subject: RFR 9: 8167166 : Java API docs mention a non-existent method
	getNanosOfSecond
In-Reply-To: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>
References: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>
Message-ID: <aed27f71-c184-45dc-2c28-8eefd2017f03@Oracle.com>

(With correct cr.openjdk.java.net link).

Please review a corrected javadoc link correction in java.time.Instant.

Webrev:
   http://cr.openjdk.java.net/~rriggs/webrev-nano-8167166/

Thanks, Roger

From ljnelson at gmail.com  Fri Oct 14 21:21:55 2016
From: ljnelson at gmail.com (Laird Nelson)
Date: Fri, 14 Oct 2016 21:21:55 +0000
Subject: Process to add some default methods to javax.naming.Context?
In-Reply-To: <6D80774B-75DF-419E-A3A6-9DF2C7AFD91A@oracle.com>
References: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
	<6D80774B-75DF-419E-A3A6-9DF2C7AFD91A@oracle.com>
Message-ID: <CAHNdxu=JAwUHwUfjkiV5RMA25u7jR-VXPKMDTxHLH=UH2Pyirg@mail.gmail.com>

On Fri, Oct 14, 2016 at 7:19 AM Vincent Ryan <vincent.x.ryan at oracle.com>
wrote:

> The problem of having to cast the result of Context.lookup
> <http://download.java.net/java/jdk9/docs/api/javax/naming/Context.html#lookup-javax.naming.Name-> can
> be solved using an existing method:
>     InitialContext.doLookup
> <http://download.java.net/java/jdk9/docs/api/javax/naming/InitialContext.html#doLookup-javax.naming.Name->
>

That solves the simple casting problem from an initial Context, but not for
all Contexts, and the simple addition of a relatively trivial default
method like this in the Context interface it seems to me would open the
door for Context implementers to reimplement it to do more
sophisticated?and deliberate?type conversion.

Best,
Laird

From naoto.sato at oracle.com  Fri Oct 14 21:23:28 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Fri, 14 Oct 2016 14:23:28 -0700
Subject: RFR 9: 8167166 : Java API docs mention a non-existent method
	getNanosOfSecond
In-Reply-To: <aed27f71-c184-45dc-2c28-8eefd2017f03@Oracle.com>
References: <62c0201a-2bdd-9f74-064e-921658d6c59e@Oracle.com>
	<aed27f71-c184-45dc-2c28-8eefd2017f03@Oracle.com>
Message-ID: <92dfc4b7-30d7-d3da-3b7a-fed180dc899d@oracle.com>

+1

Naoto

On 10/14/16 2:13 PM, Roger Riggs wrote:
> (With correct cr.openjdk.java.net link).
>
> Please review a corrected javadoc link correction in java.time.Instant.
>
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-nano-8167166/
>
> Thanks, Roger

From paul.sandoz at oracle.com  Fri Oct 14 22:08:30 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 14 Oct 2016 15:08:30 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
Message-ID: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>

Hi,

Please review:

  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/
  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/

The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction:

https://bugs.openjdk.java.net/browse/JDK-8164693

"If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction
<add>throws E if the type of E is Error or a subclass, else it</add>
throws a BootstrapMethodError that wraps E."

"Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction
<add>throws E if the type of E is Error or a subclass, else it</add>
throws a BootstrapMethodError that wraps E.?

Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped.

This is a behavioural change but we anticipate one that should have minimal impact.


This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change.

Thanks,
Paul.

From mandy.chung at oracle.com  Sat Oct 15 21:31:37 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 15 Oct 2016 14:31:37 -0700
Subject: =?utf-8?Q?Re=3A_RFR=3A_6378384_=28reflect=29_subclass_can?=
	=?utf-8?Q?=E2=80=99t_access_superclass=E2=80=99s_protected_field?=
	=?utf-8?Q?s_and_methods_by_reflection?=
In-Reply-To: <bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
Message-ID: <CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>

Hi Peter,

> On Oct 2, 2016, at 2:51 PM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.02/


This change looks good to me.  Thanks for adding the test enumerating exhaustive combinations.  I ran JCK tests and verified that no new failure.

With this change, sun.reflect.misc.ReflectUtil.ensureMemberAccess could be removed when Atomic*FieldUpdater are updated to use Reflection::ensureMemberAccess directly. As of now, we will keep ReflectUtil::ensureMemberAccess.

AccessControlTest.java is a great test.  I wonder if AccessControlTest.java could be made less verbose and make it easier to read.  For example, can we add a builder class that takes either the list of allowed or denied MemberFactory  (but not both) to reduce the verbosity.   Builder::allowAll and Builder::denyAll would be useful.  allowAccessMember of a specific modifier can imply both field and method.

The builder will generate a test case that allowed and denied sets are distinct and cover the full set of MemberFactory.

Mandy


From peter.levart at gmail.com  Sun Oct 16 18:18:18 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 16 Oct 2016 20:18:18 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
Message-ID: <23885332-189f-730e-bd2d-165859701d41@gmail.com>

Hi Mandy,

Thanks for looking into this patch.

On 10/15/2016 11:31 PM, Mandy Chung wrote:
> Hi Peter,
>
>> On Oct 2, 2016, at 2:51 PM, Peter Levart <peter.levart at gmail.com> wrote:
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.02/
> This change looks good to me.  Thanks for adding the test enumerating exhaustive combinations.  I ran JCK tests and verified that no new failure.
>
> With this change, sun.reflect.misc.ReflectUtil.ensureMemberAccess could be removed when Atomic*FieldUpdater are updated to use Reflection::ensureMemberAccess directly. As of now, we will keep ReflectUtil::ensureMemberAccess.
>
> AccessControlTest.java is a great test.  I wonder if AccessControlTest.java could be made less verbose and make it easier to read.  For example, can we add a builder class that takes either the list of allowed or denied MemberFactory  (but not both) to reduce the verbosity.

I think specifying both is more verbose on one hand, but OTOH it allows 
one to visually inspect all the cases and think of each and every one in 
isolation to see if it is valid in a particular caller/member/target 
arrangement. The test infrastructure verifies that the test case covers 
all the MemberFactory(s) so one must only verify each individual allowed 
/ denied MemberFactory.

>     Builder::allowAll and Builder::denyAll would be useful.  allowAccessMember of a specific modifier can imply both field and method.

This is a good idea. Here's a modified test (no changes to patched 
files, just tests):

http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.03/


Is it easier on the eye now?

I also added Copyright headers to all the test sources.


Regards, Peter


From frank.yuan at oracle.com  Mon Oct 17 02:13:19 2016
From: frank.yuan at oracle.com (Frank Yuan)
Date: Mon, 17 Oct 2016 10:13:19 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <580117AB.9020603@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<58000F81.9010704@oracle.com>
	<09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>
	<01a201d225fd$4f5c7a60$ee156f20$@oracle.com>
	<580117AB.9020603@oracle.com>
Message-ID: <006f01d2281c$0d450c70$27cf2550$@oracle.com>

Alright, thank you! Then I will check it in.

Frank

> -----Original Message-----
> From: Joe Wang [mailto:huizhe.wang at oracle.com]
> Subject: Re: RFR (JAXP) JDK-8167478 javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with "java.security.AccessControlException:
> access denied ("java.io.FilePermission" "sko?ice")"
> 
> Looks fine. Paths.get is the way to go.
> 
> -Joe
> 
> On 10/14/16, 2:28 AM, Frank Yuan wrote:
> > Hi Joe
> >
> > After some testing, I am sure current Windows platforms support Unicode although the default charset is Windows-1252. So now I check it by
> trying Paths.get method, which was also suggested by Max.
> >
> > Please check the new webrev http://cr.openjdk.java.net/~fyuan/8167478/webrev.01/
> >
> > Thanks
> > Frank
> >
> >> -----Original Message-----
> >> From: Leo Jiang [mailto:li.jiang at oracle.com]
> >> Subject: Re: RFR (JAXP) JDK-8167478 javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with "java.security.AccessControlException:
> >> access denied ("java.io.FilePermission" "sko?ice")"
> >>
> >> Hi Frank,
> >>
> >> I suggest you to use the following code snippet to make your code more readable.
> >>
> >>           CharsetEncoder charsetEncoder = Charset.defaultCharset().newEncoder();
> >>           boolean b = charsetEncoder.canEncode(path);// alpha
> >>
> >> Any character in Unicode range would be valid as path if current OS charset is UTF-8 (Unicode series encoding). But if
> >> current encoding setting (code page in Windows) is not UTF, e.g. cp936, the string in Java with inside UTF-16
> >> representation  need to be mapped to the local encoding, that might be failed.
> >>
> >> After modification, the test should be passed on UTF-8 encoding, and the encoding which support the character '\u0159'.
> >>
> >> Thanks,
> >> Leo
> >>
> >> On 10/14/2016 06:49 AM, Joe Wang wrote:
> >>> Hi Frank,
> >>>
> >>> Does this work as expected? The method doesn't seem to validate whether a character is legal as a file name. For
> >>> example, on Windows, the original char (e.g. \u0159) used in the test is legal in a file name, but it didn't pass that
> >>> decode/encode test by the Windows' default encoding (Windows-1252). Does that mean this test will no longer run on Windows?
> >>>
> >>> Thanks,
> >>> Joe
> >>>
> >>> On 10/13/16, 2:05 AM, Frank Yuan wrote:
> >>>> Hi all
> >>>>
> >>>>
> >>>>
> >>>> Would you like to review http://cr.openjdk.java.net/~fyuan/8167478/webrev.00/
> >>>> <http://cr.openjdk.java.net/%7Efyuan/8167478/webrev.00/>  ?
> >>>>
> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8167478
> >>>>
> >>>>
> >>>>
> >>>> This is a test bug, because Bug6341770.java is invalid when the system environment doesn?t support non-ascii
> >>>> characters, the test will exit immediately in this condition.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>>
> >>>>
> >>>> Frank
> >>>>
> >>>>
> >>>>


From weijun.wang at oracle.com  Mon Oct 17 05:30:08 2016
From: weijun.wang at oracle.com (Wang Weijun)
Date: Mon, 17 Oct 2016 13:30:08 +0800
Subject: RFR (JAXP) JDK-8167478
	javax/xml/jaxp/unittest/parsers/Bug6341770.java failed with
	"java.security.AccessControlException: access denied
	("java.io.FilePermission" "sko?ice")"
In-Reply-To: <006f01d2281c$0d450c70$27cf2550$@oracle.com>
References: <021f01d22530$ed32c0f0$c79842d0$@oracle.com>
	<58000F81.9010704@oracle.com>
	<09b94f7a-1833-d543-498b-3e6e93405064@oracle.com>
	<01a201d225fd$4f5c7a60$ee156f20$@oracle.com>
	<580117AB.9020603@oracle.com>
	<006f01d2281c$0d450c70$27cf2550$@oracle.com>
Message-ID: <66B5E020-867F-4B5C-A907-433D1B67C839@oracle.com>

http://hg.openjdk.java.net/jdk9/dev/jaxp/rev/037c095ba0c345edbeaaab52fda913a76c3930c0

My understanding is that if the changeset already has your name as the author then there is no need to add your name again into Contributed-by.

--Max

> On Oct 17, 2016, at 10:13 AM, Frank Yuan <frank.yuan at oracle.com> wrote:
> 
> Alright, thank you! Then I will check it in.
> 
> Frank


From srinivasan.raghavan at oracle.com  Mon Oct 17 08:51:09 2016
From: srinivasan.raghavan at oracle.com (Srinivasan Raghavan)
Date: Mon, 17 Oct 2016 14:21:09 +0530
Subject: RFR:JDK-8075205 java/net/URLClassLoader/closetest/CloseTest.java and
	GetResourceAsStream.java failed with existing dir
Message-ID: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>

Hi all

Please review the fix for the bug 

Bug :https://bugs.openjdk.java.net/browse/JDK-8075205

The tests uses classes directory for the output files. This can result in the files being left over after the test is complete which can result in instability. The tests copies the files to be compiled form test src to test classes which can result in copy of permission and result in instability because the test has delete operations. The test fails randomly mostly in copy or delete operation. I propose the test to be refactored to make the use scratch directory as its output directory and eliminate shell by using testlibrary utils.

fix : http://cr.openjdk.java.net/~sraghavan/8075205/webrev.00/

Regards
Srinivasan Raghavan


From vincent.x.ryan at oracle.com  Mon Oct 17 10:24:29 2016
From: vincent.x.ryan at oracle.com (Vincent Ryan)
Date: Mon, 17 Oct 2016 11:24:29 +0100
Subject: Process to add some default methods to javax.naming.Context?
In-Reply-To: <CAHNdxu=JAwUHwUfjkiV5RMA25u7jR-VXPKMDTxHLH=UH2Pyirg@mail.gmail.com>
References: <CAHNdxunme18iBBcAOmAriLTR62+bVEepafmF3f3fmPUj+g2Ppw@mail.gmail.com>
	<6D80774B-75DF-419E-A3A6-9DF2C7AFD91A@oracle.com>
	<CAHNdxu=JAwUHwUfjkiV5RMA25u7jR-VXPKMDTxHLH=UH2Pyirg@mail.gmail.com>
Message-ID: <B156A626-F8EE-462C-9975-82A5A74D1644@oracle.com>

Please file an RFE for this issue so that the core-libs team can evaluate.
While this seems like a good use for default methods, it may be difficult to justify including a convenience method
at this late stage in the JDK 9 schedule.

Thanks.


> On 14 Oct 2016, at 22:21, Laird Nelson <ljnelson at gmail.com> wrote:
> 
> On Fri, Oct 14, 2016 at 7:19 AM Vincent Ryan <vincent.x.ryan at oracle.com <mailto:vincent.x.ryan at oracle.com>> wrote:
> The problem of having to cast the result of Context.lookup <http://download.java.net/java/jdk9/docs/api/javax/naming/Context.html#lookup-javax.naming.Name-> can be solved using an existing method:
>     InitialContext.doLookup <http://download.java.net/java/jdk9/docs/api/javax/naming/InitialContext.html#doLookup-javax.naming.Name->
> 
> That solves the simple casting problem from an initial Context, but not for all Contexts, and the simple addition of a relatively trivial default method like this in the Context interface it seems to me would open the door for Context implementers to reimplement it to do more sophisticated?and deliberate?type conversion.
> 
> Best,
> Laird


From claes.redestad at oracle.com  Mon Oct 17 11:17:33 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 17 Oct 2016 13:17:33 +0200
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
Message-ID: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>

Hi,

one partial cause for startup regressions due to jigsaw is related to 
creating
URIs for the location of each module.

By providing a package-private constructor we can avoid the time to scan 
and
validate the URI, which takes a little time (executes ~80K bytecodes) 
but also
pushes various methods over compilation thresholds during early startup,
which more notably interferes with perceived startup.

http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8168073

Notes:

When looking at this I discovered that the existing methods in JavaNetAccess
is no longer in use, and keeping the initialization in URLClassLoader 
forces
added classloading and costly initialization no longer needed during normal
bootstrap. Moving JavaNetAccess to URI seems like the natural choice.

By exploiting the lazyness of URI.toString(), this patch also saves a 
few Kbs of
retained heap.

Thanks!

/Claes

From paul.sandoz at oracle.com  Mon Oct 17 17:39:48 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 10:39:48 -0700
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
Message-ID: <FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>

Hi Claes,

This looks good.

Did you consider adding asserts to the package private constructor?

Paul.

> On 17 Oct 2016, at 04:17, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> Hi,
> 
> one partial cause for startup regressions due to jigsaw is related to creating
> URIs for the location of each module.
> 
> By providing a package-private constructor we can avoid the time to scan and
> validate the URI, which takes a little time (executes ~80K bytecodes) but also
> pushes various methods over compilation thresholds during early startup,
> which more notably interferes with perceived startup.
> 
> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8168073
> 
> Notes:
> 
> When looking at this I discovered that the existing methods in JavaNetAccess
> is no longer in use, and keeping the initialization in URLClassLoader forces
> added classloading and costly initialization no longer needed during normal
> bootstrap. Moving JavaNetAccess to URI seems like the natural choice.
> 
> By exploiting the lazyness of URI.toString(), this patch also saves a few Kbs of
> retained heap.
> 
> Thanks!
> 
> /Claes


From claes.redestad at oracle.com  Mon Oct 17 17:52:51 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 17 Oct 2016 19:52:51 +0200
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>
Message-ID: <58050FF3.9070207@oracle.com>

Hi Paul,

On 2016-10-17 19:39, Paul Sandoz wrote:
> Hi Claes,
>
> This looks good.

Thanks!

>
> Did you consider adding asserts to the package private constructor?

No, might be reasonable. Do you insist? :-)

/Claes

>
> Paul.
>
>> On 17 Oct 2016, at 04:17, Claes Redestad <claes.redestad at oracle.com> wrote:
>>
>> Hi,
>>
>> one partial cause for startup regressions due to jigsaw is related to creating
>> URIs for the location of each module.
>>
>> By providing a package-private constructor we can avoid the time to scan and
>> validate the URI, which takes a little time (executes ~80K bytecodes) but also
>> pushes various methods over compilation thresholds during early startup,
>> which more notably interferes with perceived startup.
>>
>> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8168073
>>
>> Notes:
>>
>> When looking at this I discovered that the existing methods in JavaNetAccess
>> is no longer in use, and keeping the initialization in URLClassLoader forces
>> added classloading and costly initialization no longer needed during normal
>> bootstrap. Moving JavaNetAccess to URI seems like the natural choice.
>>
>> By exploiting the lazyness of URI.toString(), this patch also saves a few Kbs of
>> retained heap.
>>
>> Thanks!
>>
>> /Claes
>

From claes.redestad at oracle.com  Mon Oct 17 17:53:02 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 17 Oct 2016 19:53:02 +0200
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>
Message-ID: <58050FFE.4080009@oracle.com>

Hi Paul,

On 2016-10-17 19:39, Paul Sandoz wrote:
> Hi Claes,
>
> This looks good.

Thanks!

>
> Did you consider adding asserts to the package private constructor?

No, might be reasonable. Do you insist? :-)

/Claes

>
> Paul.
>
>> On 17 Oct 2016, at 04:17, Claes Redestad <claes.redestad at oracle.com> wrote:
>>
>> Hi,
>>
>> one partial cause for startup regressions due to jigsaw is related to creating
>> URIs for the location of each module.
>>
>> By providing a package-private constructor we can avoid the time to scan and
>> validate the URI, which takes a little time (executes ~80K bytecodes) but also
>> pushes various methods over compilation thresholds during early startup,
>> which more notably interferes with perceived startup.
>>
>> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8168073
>>
>> Notes:
>>
>> When looking at this I discovered that the existing methods in JavaNetAccess
>> is no longer in use, and keeping the initialization in URLClassLoader forces
>> added classloading and costly initialization no longer needed during normal
>> bootstrap. Moving JavaNetAccess to URI seems like the natural choice.
>>
>> By exploiting the lazyness of URI.toString(), this patch also saves a few Kbs of
>> retained heap.
>>
>> Thanks!
>>
>> /Claes
>

From Alan.Bateman at oracle.com  Mon Oct 17 18:07:20 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 17 Oct 2016 19:07:20 +0100
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
Message-ID: <53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>



On 17/10/2016 12:17, Claes Redestad wrote:
> Hi,
>
> one partial cause for startup regressions due to jigsaw is related to 
> creating
> URIs for the location of each module.
>
> By providing a package-private constructor we can avoid the time to 
> scan and
> validate the URI, which takes a little time (executes ~80K bytecodes) 
> but also
> pushes various methods over compilation thresholds during early startup,
> which more notably interferes with perceived startup.
>
> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8168073
This looks okay, I just wondering if JavaNetAccess should be renamed to 
JavaNetUriAccess to avoid it being used to access other classes in 
java.net. If renamed then I guess createUri could be renamed to create too.

-Alan

From claes.redestad at oracle.com  Mon Oct 17 18:45:04 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 17 Oct 2016 20:45:04 +0200
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
Message-ID: <58051C30.5040106@oracle.com>



On 2016-10-17 20:07, Alan Bateman wrote:
>
>
> On 17/10/2016 12:17, Claes Redestad wrote:
>> Hi,
>>
>> one partial cause for startup regressions due to jigsaw is related to
>> creating
>> URIs for the location of each module.
>>
>> By providing a package-private constructor we can avoid the time to
>> scan and
>> validate the URI, which takes a little time (executes ~80K bytecodes)
>> but also
>> pushes various methods over compilation thresholds during early startup,
>> which more notably interferes with perceived startup.
>>
>> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8168073
> This looks okay, I just wondering if JavaNetAccess should be renamed to
> JavaNetUriAccess to avoid it being used to access other classes in
> java.net. If renamed then I guess createUri could be renamed to create too.

Most other SharedSecrets classes seems to be per-package, so not sure
if this case is special enough to start making them per-class, other 
than the slightly more natural naming.

/Claes

>
> -Alan

From Alan.Bateman at oracle.com  Mon Oct 17 19:35:32 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 17 Oct 2016 20:35:32 +0100
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <58051C30.5040106@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
Message-ID: <4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>



On 17/10/2016 19:45, Claes Redestad wrote:
>
> Most other SharedSecrets classes seems to be per-package, so not sure
> if this case is special enough to start making them per-class, other 
> than the slightly more natural naming.
JavaNetHttpCookieAccess, JavaNetInetAddressAccess and 
JavaNetSocketAccess are the other 3 that are used to get at non-public 
types in java.net so I don't think JavaNetUriAccess would be out of place.

-Alan


From paul.sandoz at oracle.com  Mon Oct 17 19:58:18 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 12:58:18 -0700
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <58050FFE.4080009@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<FD1A9098-30A3-4B62-8455-136F19D6105F@oracle.com>
	<58050FFE.4080009@oracle.com>
Message-ID: <72A41CB5-6DEF-4292-A277-BAD77439E975@oracle.com>


> On 17 Oct 2016, at 10:53, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> Hi Paul,
> 
> On 2016-10-17 19:39, Paul Sandoz wrote:
>> Hi Claes,
>> 
>> This looks good.
> 
> Thanks!
> 
>> 
>> Did you consider adding asserts to the package private constructor?
> 
> No, might be reasonable. Do you insist? :-)
> 

A gentle insistence. It could stop a nasty bug in the future if this method gets used in other places and incorrect URI instances are produced e.g.:

  assert foo(scheme, path);

  static boolean foo(String scheme, String path) {
    URI u = new URI(scheme + ?:/? + path);
    return scheme.equals(u.scheme) && path.equals(u.path);
  }

Paul.

From mandy.chung at oracle.com  Mon Oct 17 19:59:03 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 17 Oct 2016 12:59:03 -0700
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
Message-ID: <833329FA-583C-4D56-8FBC-04D238DF5634@oracle.com>


> On Oct 17, 2016, at 11:07 AM, Alan Bateman <alan.bateman at oracle.com> wrote:
> 
> 
> 
> On 17/10/2016 12:17, Claes Redestad wrote:
>> Hi,
>> 
>> one partial cause for startup regressions due to jigsaw is related to creating
>> URIs for the location of each module.
>> 
>> By providing a package-private constructor we can avoid the time to scan and
>> validate the URI, which takes a little time (executes ~80K bytecodes) but also
>> pushes various methods over compilation thresholds during early startup,
>> which more notably interferes with perceived startup.
>> 
>> http://cr.openjdk.java.net/~redestad/8168073/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8168073
> This looks okay, I just wondering if JavaNetAccess should be renamed to JavaNetUriAccess to avoid it being used to access other classes in java.net. If renamed then I guess createUri could be renamed to create too.
> 

I agree that JavaNetUriAccess is a better name for this create method, as the shared secret is specific to URI.

One thing to note is that Dave DeHaven has a patch being reviewed to add a getAccessControlContext(URLClassLoader u) method to get ACC of a given URLClassLoader.  I suppose your patch will be pushed soon.  Dave should adjust his patch to add JavaNetURLClassLoaderAccess interface.

http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-October/044149.html

Mandy

From kumar.x.srinivasan at oracle.com  Mon Oct 17 20:39:07 2016
From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan)
Date: Mon, 17 Oct 2016 13:39:07 -0700
Subject: RFR: 8168093: Need a way for the launcher to query the JRE location
	using Windows registry.
Message-ID: <580536EB.7070204@oracle.com>

Hello,

I am sponsoring this patch for the client team, the issue here
is that the Windows installer copies the java launcher binaries to a
system folder, and those binaries need to locate the Java Runtime
using Windows registry. These changes are to allow calling an
internal (Registry locator) method for the product build.

http://cr.openjdk.java.net/~ksrini/8168093/webrev.00/

Thanks
Kumar


From mandy.chung at oracle.com  Mon Oct 17 20:52:25 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 17 Oct 2016 13:52:25 -0700
Subject: =?windows-1252?Q?Re=3A_RFR=3A_6378384_=28reflect=29_subclass_can?=
	=?windows-1252?Q?=92t_access_superclass=92s_protected_fields_and?=
	=?windows-1252?Q?_methods_by_reflection?=
In-Reply-To: <23885332-189f-730e-bd2d-165859701d41@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
Message-ID: <A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>


> On Oct 16, 2016, at 11:18 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> 
> I think specifying both is more verbose on one hand, but OTOH it allows one to visually inspect all the cases and think of each and every one in isolation to see if it is valid in a particular caller/member/target arrangement. The test infrastructure verifies that the test case covers all the MemberFactory(s) so one must only verify each individual allowed / denied MemberFactory.

I understand the intention there.  But when each test case enumerates 20 constants, it?s getting harder to review what?s going on and catch any issue.

> 
>>    Builder::allowAll and Builder::denyAll would be useful.  allowAccessMember of a specific modifier can imply both field and method.
> 
> This is a good idea. Here's a modified test (no changes to patched files, just tests):
> 
> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.03/


I like these grouping and it does help.  One more nit: would be good to replace the class name strings with constant variables.  I don?t need a new webrev.

thanks
Mandy

From claes.redestad at oracle.com  Mon Oct 17 21:18:55 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 17 Oct 2016 23:18:55 +0200
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
	<4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
Message-ID: <5805403F.3090809@oracle.com>



On 2016-10-17 21:35, Alan Bateman wrote:
>
>
> On 17/10/2016 19:45, Claes Redestad wrote:
>>
>> Most other SharedSecrets classes seems to be per-package, so not sure
>> if this case is special enough to start making them per-class, other
>> than the slightly more natural naming.
> JavaNetHttpCookieAccess, JavaNetInetAddressAccess and
> JavaNetSocketAccess are the other 3 that are used to get at non-public
> types in java.net so I don't think JavaNetUriAccess would be out of place.

TIL!

Ok, renamed and added an assert by Paul's request:

http://cr.openjdk.java.net/~redestad/8168073/webrev.02/

/Claes

From paul.sandoz at oracle.com  Mon Oct 17 21:44:26 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 14:44:26 -0700
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <5805403F.3090809@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
	<4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
	<5805403F.3090809@oracle.com>
Message-ID: <BB0AFD6A-DC03-4494-83E4-3753C156A4D8@oracle.com>


> On 17 Oct 2016, at 14:18, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> 
> 
> On 2016-10-17 21:35, Alan Bateman wrote:
>> 
>> 
>> On 17/10/2016 19:45, Claes Redestad wrote:
>>> 
>>> Most other SharedSecrets classes seems to be per-package, so not sure
>>> if this case is special enough to start making them per-class, other
>>> than the slightly more natural naming.
>> JavaNetHttpCookieAccess, JavaNetInetAddressAccess and
>> JavaNetSocketAccess are the other 3 that are used to get at non-public
>> types in java.net so I don't think JavaNetUriAccess would be out of place.
> 
> TIL!
> 
> Ok, renamed and added an assert by Paul's request:
> 
> http://cr.openjdk.java.net/~redestad/8168073/webrev.02/
> 

+1

Paul.

From mandy.chung at oracle.com  Mon Oct 17 21:59:35 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 17 Oct 2016 14:59:35 -0700
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <5805403F.3090809@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
	<4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
	<5805403F.3090809@oracle.com>
Message-ID: <7AD1D598-0F39-4233-A4AE-5915F43C1C77@oracle.com>


> On Oct 17, 2016, at 2:18 PM, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~redestad/8168073/webrev.02/

+1

Mandy


From stuart.marks at oracle.com  Mon Oct 17 22:01:02 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 17 Oct 2016 15:01:02 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
Message-ID: <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>

Hi Paul,

I took a look at the jdk changes. They look good to me.

One section of code gave me pause, which is the throw of ClassCastException at 
339 of CallSite.java, and the throw of the exception returned from 
wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow 
any Error and wrap anything else in BSME" rule. But these throws are caught by 
the catch of Throwable below, which does the wrap and throw of BSME.

This arrangement was already present in the code.

Usually I wrinkle my nose at a throw that's caught by a catch clause later on, 
but in this case it's not obvious what would be better. Maybe a comment is 
warranted?

I didn't look too closely at the hotspot changes; somebody else should review them.

s'marks



On 10/14/16 3:08 PM, Paul Sandoz wrote:
> Hi,
>
> Please review:
>
>   http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/
>   http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/
>
> The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction:
>
> https://bugs.openjdk.java.net/browse/JDK-8164693
>
> "If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction
> <add>throws E if the type of E is Error or a subclass, else it</add>
> throws a BootstrapMethodError that wraps E."
>
> "Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction
> <add>throws E if the type of E is Error or a subclass, else it</add>
> throws a BootstrapMethodError that wraps E.?
>
> Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped.
>
> This is a behavioural change but we anticipate one that should have minimal impact.
>
>
> This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change.
>
> Thanks,
> Paul.
>

From david.holmes at oracle.com  Mon Oct 17 22:33:40 2016
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 18 Oct 2016 08:33:40 +1000
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
Message-ID: <d136fad2-e9be-1ef4-1394-e1f26f500496@oracle.com>

Hi Paul,

Looking at hotspot changes only ...

On 15/10/2016 8:08 AM, Paul Sandoz wrote:
> Hi,
>
> Please review:
>
>   http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/
>   http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/

src/share/vm/interpreter/linkResolver.cpp

Changes seem fine, but the existing tracing code - which was already 
somewhat incorrect - now appears even more incorrect. It either needs to 
be moved to after the return for the Error case - so that we really are 
going to throw BSME - or else two different trace statements are needed.

--

test/runtime/invokedynamic/BootstrapMethodErrorTest.java

  2  * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights 
reserved.

Need comma after 2016.

Otherwise all good.

Thanks,
David

> The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction:
>
> https://bugs.openjdk.java.net/browse/JDK-8164693
>
> "If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction
> <add>throws E if the type of E is Error or a subclass, else it</add>
> throws a BootstrapMethodError that wraps E."
>
> "Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction
> <add>throws E if the type of E is Error or a subclass, else it</add>
> throws a BootstrapMethodError that wraps E.?
>
> Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped.
>
> This is a behavioural change but we anticipate one that should have minimal impact.
>
>
> This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change.
>
> Thanks,
> Paul.
>

From paul.sandoz at oracle.com  Mon Oct 17 22:38:17 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 15:38:17 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>
Message-ID: <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com>


> On 17 Oct 2016, at 15:01, Stuart Marks <stuart.marks at oracle.com> wrote:
> 
> Hi Paul,
> 
> I took a look at the jdk changes. They look good to me.
> 
> One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME.
> 
> This arrangement was already present in the code.
> 
> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted?

In addition to the

  // See the "Linking Exceptions" section for the invokedynamic
  // instruction in JVMS 6.5.

I can add something like:

  // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError

?

Thanks,
Paul.

From stuart.marks at oracle.com  Mon Oct 17 23:20:13 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 17 Oct 2016 16:20:13 -0700
Subject: RFR(xs): 8168096: markup error in "since" element spec of @Deprecated
Message-ID: <f4f30223-0dc5-e836-eee7-e7c9e2e3d78b@oracle.com>

Hi all, please review this tiny fix to the javadoc markup in the Deprecated 
annotation type.

Thanks,

s'marks




--- a/src/java.base/share/classes/java/lang/Deprecated.java	Thu Oct 13 23:02:35 
2016 +0000
+++ b/src/java.base/share/classes/java/lang/Deprecated.java	Mon Oct 17 16:18:26 
2016 -0700
@@ -79,7 +79,7 @@
      /**
       * Returns the version in which the annotated element became deprecated.
       * The version string is in the same format and namespace as the value of
-     * the {@code &#64;since} javadoc tag. The default value is the empty
+     * the {@code @since} javadoc tag. The default value is the empty
       * string.
       *
       * @return the version string

From joe.darcy at oracle.com  Mon Oct 17 23:21:40 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 17 Oct 2016 16:21:40 -0700
Subject: RFR(xs): 8168096: markup error in "since" element spec of
	@Deprecated
In-Reply-To: <f4f30223-0dc5-e836-eee7-e7c9e2e3d78b@oracle.com>
References: <f4f30223-0dc5-e836-eee7-e7c9e2e3d78b@oracle.com>
Message-ID: <58055D04.4020802@oracle.com>

+1

-Joe

On 10/17/2016 4:20 PM, Stuart Marks wrote:
> Hi all, please review this tiny fix to the javadoc markup in the 
> Deprecated annotation type.
>
> Thanks,
>
> s'marks
>
>
>
>
> --- a/src/java.base/share/classes/java/lang/Deprecated.java    Thu Oct 
> 13 23:02:35 2016 +0000
> +++ b/src/java.base/share/classes/java/lang/Deprecated.java    Mon Oct 
> 17 16:18:26 2016 -0700
> @@ -79,7 +79,7 @@
>      /**
>       * Returns the version in which the annotated element became 
> deprecated.
>       * The version string is in the same format and namespace as the 
> value of
> -     * the {@code &#64;since} javadoc tag. The default value is the 
> empty
> +     * the {@code @since} javadoc tag. The default value is the empty
>       * string.
>       *
>       * @return the version string


From peter.levart at gmail.com  Mon Oct 17 23:30:40 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 18 Oct 2016 01:30:40 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
Message-ID: <bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>

Hi Mandy,


On 10/17/2016 10:52 PM, Mandy Chung wrote:
>> On Oct 16, 2016, at 11:18 AM, Peter Levart <peter.levart at gmail.com> wrote:
>>
>>
>> I think specifying both is more verbose on one hand, but OTOH it allows one to visually inspect all the cases and think of each and every one in isolation to see if it is valid in a particular caller/member/target arrangement. The test infrastructure verifies that the test case covers all the MemberFactory(s) so one must only verify each individual allowed / denied MemberFactory.
> I understand the intention there.  But when each test case enumerates 20 constants, it?s getting harder to review what?s going on and catch any issue.

It now enumerates only 12 constants and I think it's quite the opposite. 
Take the following for example. If only the allowed members were listed:

          ok &= new Test()
.current("b.PublicSub").member("a.PublicSuper").target("b.PublicSub")
              .allowed(PROTECTED_INSTANCE_F_M, PUBLIC_INSTANCE_F_M, 
PROTECTED_STATIC_F_M,
                       PUBLIC_STATIC_F_M, PUBLIC_C)
              .perform();

...you would review the members that are allowed and then you should ask 
yourself: "Which are the ones that are denied? All the rest. What are 
they?", or: "Could there be any other that should be allowed? Which one?"

It's much easier if they are explicitly listed:

          ok &= new Test()
.current("b.PublicSub").member("a.PublicSuper").target("b.PublicSub")
              .allowed(PROTECTED_INSTANCE_F_M, PUBLIC_INSTANCE_F_M, 
PROTECTED_STATIC_F_M,
                       PUBLIC_STATIC_F_M, PUBLIC_C)
              .denied (PRIVATE_INSTANCE_F_M, PACKAGE_INSTANCE_F_M, 
PRIVATE_STATIC_F_M,
                       PACKAGE_STATIC_F_M, PRIVATE_C, PACKAGE_C,
                       PROTECTED_C)
              .perform();


And besides, you don't really have to review them all. The fact that 
running the test on unpatched JDK 9 finds just two differences:

- access to protected static method from subclass in another package
- access to protected static field from subclass in another package

...is a reassurance that the patch does exactly what it should. No less, 
no more.

>>>     Builder::allowAll and Builder::denyAll would be useful.  allowAccessMember of a specific modifier can imply both field and method.
>> This is a good idea. Here's a modified test (no changes to patched files, just tests):
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.03/
> I like these grouping and it does help.  One more nit: would be good to replace the class name strings with constant variables.  I don?t need a new webrev.

Then I would have to have a mapping from class name -> constant name for 
the generator. Besides, constant names would not be any prettier than 
class name string literals. At least now it is obvious to anyone what 
package a particular class belongs to:

     "a.Package" vs. A_PACKAGE ?

Note that I can't use a.Package.class literal(s) here (thought I would 
like to) as they don't compile if they refer to a package-private class 
from another package.

I would like to keep those things unchanged, If you don't mind.

> thanks
> Mandy

Regards, Peter


From john.r.rose at oracle.com  Mon Oct 17 23:36:16 2016
From: john.r.rose at oracle.com (John Rose)
Date: Mon, 17 Oct 2016 16:36:16 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>
	<36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com>
Message-ID: <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com>

On Oct 17, 2016, at 3:38 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
> 
> 
>> On 17 Oct 2016, at 15:01, Stuart Marks <stuart.marks at oracle.com> wrote:
>> 
>> Hi Paul,
>> 
>> I took a look at the jdk changes. They look good to me.
>> 
>> One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME.
>> 
>> This arrangement was already present in the code.
>> 
>> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted?
> 
> In addition to the
> 
>  // See the "Linking Exceptions" section for the invokedynamic
>  // instruction in JVMS 6.5.
> 
> I can add something like:
> 
>  // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError

I agree.  Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific.

I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped
exceptions through one point.  That may make somebody's day easier when placing breakpoints.

The upgraded BootstrapMethodErrorTest.java is really, really good.

Overall, I am *delighted* to see this little mess cleaned up.  It makes indy linkage approximately as
transparent to errors as the other invocation instructions, making it a better replacement for them.

Thanks a million!  Reviewed.

? John

From mandy.chung at oracle.com  Mon Oct 17 23:51:24 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 17 Oct 2016 16:51:24 -0700
Subject: =?windows-1252?Q?Re=3A_RFR=3A_6378384_=28reflect=29_subclass_can?=
	=?windows-1252?Q?=92t_access_superclass=92s_protected_fields_and?=
	=?windows-1252?Q?_methods_by_reflection?=
In-Reply-To: <bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
	<bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
Message-ID: <B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>


> On Oct 17, 2016, at 4:30 PM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> ...you would review the members that are allowed and then you should ask yourself: "Which are the ones that are denied? All the rest. What are they?", or: "Could there be any other that should be allowed? Which one??

I?m happy with your revised patch that group field/method together and adding allowAll/denyAll that makes it easier to understand while it does not lose any information.

> It's much easier if they are explicitly listed:
> 
>          ok &= new Test()
>              .current("b.PublicSub").member("a.PublicSuper").target("b.PublicSub")
>              .allowed(PROTECTED_INSTANCE_F_M, PUBLIC_INSTANCE_F_M,     PROTECTED_STATIC_F_M,
>                       PUBLIC_STATIC_F_M, PUBLIC_C)
>              .denied (PRIVATE_INSTANCE_F_M, PACKAGE_INSTANCE_F_M, PRIVATE_STATIC_F_M,
>                       PACKAGE_STATIC_F_M, PRIVATE_C, PACKAGE_C,
>                       PROTECTED_C)
>              .perform();
> 
> 
> And besides, you don't really have to review them all.

I do review them as this test is one important part of this patch :) 

> The fact that running the test on unpatched JDK 9 finds just two differences:
> 
> - access to protected static method from subclass in another package
> - access to protected static field from subclass in another package
> 
> ...is a reassurance that the patch does exactly what it should. No     less, no more.
> 

Indeed.

> 
> Then I would have to have a mapping from class name -> constant name for the generator.

This is one time thing.

> Besides, constant names would not be any prettier than class name string literals. At least now it is obvious to anyone what package a particular class belongs to:
> 
>     "a.Package" vs. A_PACKAGE ?
> 

PACKAGE_CLASS_IN_PKG_A
PUBLIC_SUPERCLASS_IN_PKG_A
PUBLIC_SUBCLASS_IN_PKG_A


> Note that I can't use a.Package.class literal(s) here (thought I would like to) as they don't compile if they refer to a package-private class from another package.
> 
> I would like to keep those things unchanged, If you don't mind.

Do the suggested variable names help?

Mandy


From paul.sandoz at oracle.com  Tue Oct 18 00:09:13 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 17:09:13 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <d136fad2-e9be-1ef4-1394-e1f26f500496@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<d136fad2-e9be-1ef4-1394-e1f26f500496@oracle.com>
Message-ID: <7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com>


> On 17 Oct 2016, at 15:33, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Paul,
> 
> Looking at hotspot changes only ...
> 
> On 15/10/2016 8:08 AM, Paul Sandoz wrote:
>> Hi,
>> 
>> Please review:
>> 
>>  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/
>>  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/
> 
> src/share/vm/interpreter/linkResolver.cpp
> 
> Changes seem fine, but the existing tracing code - which was already somewhat incorrect - now appears even more incorrect. It either needs to be moved to after the return for the Error case - so that we really are going to throw BSME - or else two different trace statements are needed.
> 

Good catch. Updated in place.

http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/src/share/vm/interpreter/linkResolver.cpp.sdiff.html <http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/src/share/vm/interpreter/linkResolver.cpp.sdiff.html>

Given how CallSite behaves the VM wrapping in BSME is likely to be rare, i am unsure how it could actually occur.


> --
> 
> test/runtime/invokedynamic/BootstrapMethodErrorTest.java
> 
> 2  * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights reserved.
> 
> Need comma after 2016.
> 

Updated.

Thanks,
Paul.

> Otherwise all good.
> 
> Thanks,
> David
> 

From paul.sandoz at oracle.com  Tue Oct 18 00:16:49 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 17 Oct 2016 17:16:49 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>
	<36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com>
	<8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com>
Message-ID: <4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com>


> On 17 Oct 2016, at 16:36, John Rose <john.r.rose at oracle.com> wrote:
> 
> On Oct 17, 2016, at 3:38 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>> 
>> 
>>> On 17 Oct 2016, at 15:01, Stuart Marks <stuart.marks at oracle.com> wrote:
>>> 
>>> Hi Paul,
>>> 
>>> I took a look at the jdk changes. They look good to me.
>>> 
>>> One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME.
>>> 
>>> This arrangement was already present in the code.
>>> 
>>> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted?
>> 
>> In addition to the
>> 
>> // See the "Linking Exceptions" section for the invokedynamic
>> // instruction in JVMS 6.5.
>> 
>> I can add something like:
>> 
>> // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError
> 
> I agree.  Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific.
> 

Thanks, updated in place.


> I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped
> exceptions through one point.  That may make somebody's day easier when placing breakpoints.
> 

> The upgraded BootstrapMethodErrorTest.java is really, really good.
> 
> Overall, I am *delighted* to see this little mess cleaned up.  It makes indy linkage approximately as
> transparent to errors as the other invocation instructions, making it a better replacement for them.
> 

Another patch will follow soon for the MH/VH linking (and general j.l.invoke code) unduly wrapping Errors.

Thanks,
Paul.

> Thanks a million!  Reviewed.
> 
> ? John


From david.holmes at oracle.com  Tue Oct 18 00:27:15 2016
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 18 Oct 2016 10:27:15 +1000
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<d136fad2-e9be-1ef4-1394-e1f26f500496@oracle.com>
	<7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com>
Message-ID: <ba4d4102-02d7-422e-6721-683af806270a@oracle.com>

On 18/10/2016 10:09 AM, Paul Sandoz wrote:
>
>> On 17 Oct 2016, at 15:33, David Holmes <david.holmes at oracle.com
>> <mailto:david.holmes at oracle.com>> wrote:
>>
>> Hi Paul,
>>
>> Looking at hotspot changes only ...
>>
>> On 15/10/2016 8:08 AM, Paul Sandoz wrote:
>>> Hi,
>>>
>>> Please review:
>>>
>>>  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/
>>>  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/
>>
>> src/share/vm/interpreter/linkResolver.cpp
>>
>> Changes seem fine, but the existing tracing code - which was already
>> somewhat incorrect - now appears even more incorrect. It either needs
>> to be moved to after the return for the Error case - so that we really
>> are going to throw BSME - or else two different trace statements are
>> needed.
>>
>
> Good catch. Updated in place.
>
> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/src/share/vm/interpreter/linkResolver.cpp.sdiff.html

Looks good.

Thanks,
David

> Given how CallSite behaves the VM wrapping in BSME is likely to be rare,
> i am unsure how it could actually occur.
>
>
>> --
>>
>> test/runtime/invokedynamic/BootstrapMethodErrorTest.java
>>
>> 2  * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights
>> reserved.
>>
>> Need comma after 2016.
>>
>
> Updated.
>
> Thanks,
> Paul.
>
>> Otherwise all good.
>>
>> Thanks,
>> David
>>

From stuart.marks at oracle.com  Tue Oct 18 00:52:10 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 17 Oct 2016 17:52:10 -0700
Subject: RFR 8166974: invokedynamic implementation should not wrap Errors
In-Reply-To: <4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com>
References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com>
	<96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com>
	<36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com>
	<8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com>
	<4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com>
Message-ID: <6212e871-fa7f-419e-383f-e008d1e2b214@oracle.com>



On 10/17/16 5:16 PM, Paul Sandoz wrote:
>> On 17 Oct 2016, at 16:36, John Rose <john.r.rose at oracle.com> wrote:
>> On Oct 17, 2016, at 3:38 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>>>> On 17 Oct 2016, at 15:01, Stuart Marks <stuart.marks at oracle.com> wrote:
>>>>
>>>> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted?
>>>
>>> In addition to the
>>>
>>> // See the "Linking Exceptions" section for the invokedynamic
>>> // instruction in JVMS 6.5.
>>>
>>> I can add something like:
>>>
>>> // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError
>>
>> I agree.  Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific.
>
> Thanks, updated in place.

The revised comment is a bit more verbose than my taste, but it's fine as it is. 
The main point is to alert the reader that something unusual is going on.

[from John]
>> I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped
>> exceptions through one point.  That may make somebody's day easier when placing breakpoints.

Agreed. This approach is indeed unusual, but I think the alternatives are worse: 
either the BSME wrapping can be replicated at the point of the throw, or it 
could be refactored into a common method. Either seems like a step in the wrong 
direction. Once you figure out this code, it isn't so bad; it's just unusual.

s'marks

From lance.andersen at oracle.com  Tue Oct 18 01:15:28 2016
From: lance.andersen at oracle.com (Lance @ Oracle)
Date: Mon, 17 Oct 2016 21:15:28 -0400
Subject: RFR(xs): 8168096: markup error in "since" element spec of
	@Deprecated
In-Reply-To: <f4f30223-0dc5-e836-eee7-e7c9e2e3d78b@oracle.com>
References: <f4f30223-0dc5-e836-eee7-e7c9e2e3d78b@oracle.com>
Message-ID: <C92C2DAB-94E1-45AE-81D8-093AE20E7D63@oracle.com>

+1


Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive
Burlington, MA 01803
Lance.Andersen at oracle.com
Sent from my iPad

> On Oct 17, 2016, at 7:20 PM, Stuart Marks <stuart.marks at oracle.com> wrote:
> 
> Hi all, please review this tiny fix to the javadoc markup in the Deprecated annotation type.
> 
> Thanks,
> 
> s'marks
> 
> 
> 
> 
> --- a/src/java.base/share/classes/java/lang/Deprecated.java    Thu Oct 13 23:02:35 2016 +0000
> +++ b/src/java.base/share/classes/java/lang/Deprecated.java    Mon Oct 17 16:18:26 2016 -0700
> @@ -79,7 +79,7 @@
>     /**
>      * Returns the version in which the annotated element became deprecated.
>      * The version string is in the same format and namespace as the value of
> -     * the {@code &#64;since} javadoc tag. The default value is the empty
> +     * the {@code @since} javadoc tag. The default value is the empty
>      * string.
>      *
>      * @return the version string

From martinrb at google.com  Tue Oct 18 01:34:34 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 17 Oct 2016 18:34:34 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
Message-ID: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>

Most of this is for Stuart - very collection-y.

http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/

This includes a rewrite of ArrayDeque to bring it to parity with ArrayList
(except for List features).
The patch includes public methods ensureCapacity, trimToSize, replaceAll as
in ArrayList, but we can defer making them public to a later change (or a
later release), since new public methods are always controversial.  But I'd
really like to get the performance/scalability changes in for jdk 9.

It also includes a redo of ArrayList#removeIf to make it more efficient and
consistent with behavior of other implementations, including ArrayDeque.

The patches are a little tangled because they step on each other's toes.
File CollectionTest is in "miscellaneous", but it tests both the ArrayDeque
and ArrayList changes.

From Alan.Bateman at oracle.com  Tue Oct 18 08:11:34 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 18 Oct 2016 09:11:34 +0100
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <5805403F.3090809@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
	<4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
	<5805403F.3090809@oracle.com>
Message-ID: <da4046ba-1ab1-c6b9-62df-a86e5b8d517c@oracle.com>

On 17/10/2016 22:18, Claes Redestad wrote:

>
>
> On 2016-10-17 21:35, Alan Bateman wrote:
>>
>> JavaNetHttpCookieAccess, JavaNetInetAddressAccess and
>> JavaNetSocketAccess are the other 3 that are used to get at non-public
>> types in java.net so I don't think JavaNetUriAccess would be out of 
>> place.
>
> TIL!
>
> Ok, renamed and added an assert by Paul's request:
>
> http://cr.openjdk.java.net/~redestad/8168073/webrev.02/
Looks good.

-Alan

From chris.hegarty at oracle.com  Tue Oct 18 08:47:36 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Tue, 18 Oct 2016 09:47:36 +0100
Subject: RFR: 8168073: Speed up URI creation during module bootstrap
In-Reply-To: <da4046ba-1ab1-c6b9-62df-a86e5b8d517c@oracle.com>
References: <080c997f-2eca-16ff-d02e-6b2db5f1df06@oracle.com>
	<53106d5d-1dfc-d4ab-1703-cbccb6f02b5f@oracle.com>
	<58051C30.5040106@oracle.com>
	<4d56af1f-563c-637a-24e1-c9bd5a2ea5a3@oracle.com>
	<5805403F.3090809@oracle.com>
	<da4046ba-1ab1-c6b9-62df-a86e5b8d517c@oracle.com>
Message-ID: <06DC99AE-DBEC-4E37-929C-936232136689@oracle.com>

On 17/10/2016 22:18, Claes Redestad wrote:
> 
>> On 2016-10-17 21:35, Alan Bateman wrote:
>>> 
>>> JavaNetHttpCookieAccess, JavaNetInetAddressAccess and
>>> JavaNetSocketAccess are the other 3 that are used to get at non-public
>>> types in java.net so I don't think JavaNetUriAccess would be out of place.
>> 
>> TIL!
>> 
>> Ok, renamed and added an assert by Paul's request:
>> 
>> http://cr.openjdk.java.net/~redestad/8168073/webrev.02/

Thanks Claes, this looks good.

-Chris.


From chris.hegarty at oracle.com  Tue Oct 18 10:46:45 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Tue, 18 Oct 2016 11:46:45 +0100
Subject: RFR:JDK-8075205 java/net/URLClassLoader/closetest/CloseTest.java
	and GetResourceAsStream.java failed with existing dir
In-Reply-To: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>
References: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>
Message-ID: <019E6AAB-85A3-4D4E-A2A0-2DCBCF028EF3@oracle.com>


> On 17 Oct 2016, at 09:51, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
> 
> Hi all
> 
> Please review the fix for the bug 
> 
> Bug :https://bugs.openjdk.java.net/browse/JDK-8075205
> 
> The tests uses classes directory for the output files. This can result in the files being left over after the test is complete which can result in instability. The tests copies the files to be compiled form test src to test classes which can result in copy of permission and result in instability because the test has delete operations. The test fails randomly mostly in copy or delete operation. I propose the test to be refactored to make the use scratch directory as its output directory and eliminate shell by using testlibrary utils.
> 
> fix : http://cr.openjdk.java.net/~sraghavan/8075205/webrev.00/

This looks good to me. Thanks Srinivasan.

-Chris.

From peter.levart at gmail.com  Tue Oct 18 10:55:36 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 18 Oct 2016 12:55:36 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
	<bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
	<B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>
Message-ID: <0f3d14df-00d9-1f52-85e6-c64a3de55362@gmail.com>

Hi Mandy,


On 10/18/2016 01:51 AM, Mandy Chung wrote:
>> >Besides, constant names would not be any prettier than class name string literals. At least now it is obvious to anyone what package a particular class belongs to:
>> >
>> >     "a.Package" vs. A_PACKAGE ?
>> >
> PACKAGE_CLASS_IN_PKG_A
> PUBLIC_SUPERCLASS_IN_PKG_A
> PUBLIC_SUBCLASS_IN_PKG_A
>
> Do the suggested variable names help?

They do. What do you say about this variant (compressionLevel=9):

http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.04/

Regards, Peter


From sundararajan.athijegannathan at oracle.com  Tue Oct 18 11:36:03 2016
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Tue, 18 Oct 2016 17:06:03 +0530
Subject: RFR 8071678: javax.script.ScriptContext setAttribute method should
	clarify behavior when GLOBAL_SCOPE is used and global scope object is
	null
Message-ID: <27740553-a8b2-afde-1cde-4afbdb32b3e2@oracle.com>

Please review http://cr.openjdk.java.net/~sundar/8071678/webrev.00/ for
https://bugs.openjdk.java.net/browse/JDK-8071678

Thanks,

-Sundar


From james.laskey at oracle.com  Tue Oct 18 11:38:21 2016
From: james.laskey at oracle.com (Jim Laskey (Oracle))
Date: Tue, 18 Oct 2016 08:38:21 -0300
Subject: RFR 8071678: javax.script.ScriptContext setAttribute method
	should clarify behavior when GLOBAL_SCOPE is used and global scope
	object is null
In-Reply-To: <27740553-a8b2-afde-1cde-4afbdb32b3e2@oracle.com>
References: <27740553-a8b2-afde-1cde-4afbdb32b3e2@oracle.com>
Message-ID: <0E4B95F2-A59F-481D-ABAA-AA81111E5060@oracle.com>

+1

> On Oct 18, 2016, at 8:36 AM, Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com> wrote:
> 
> Please review http://cr.openjdk.java.net/~sundar/8071678/webrev.00/ for
> https://bugs.openjdk.java.net/browse/JDK-8071678
> 
> Thanks,
> 
> -Sundar
> 


From hannes.wallnoefer at oracle.com  Tue Oct 18 11:44:37 2016
From: hannes.wallnoefer at oracle.com (=?utf-8?Q?Hannes_Walln=C3=B6fer?=)
Date: Tue, 18 Oct 2016 13:44:37 +0200
Subject: RFR 8071678: javax.script.ScriptContext setAttribute method
	should clarify behavior when GLOBAL_SCOPE is used and global
	scope object is null
In-Reply-To: <27740553-a8b2-afde-1cde-4afbdb32b3e2@oracle.com>
References: <27740553-a8b2-afde-1cde-4afbdb32b3e2@oracle.com>
Message-ID: <236D22DD-F13C-4EC8-8D17-1299038EE07B@oracle.com>

+1

Hannes

> Am 18.10.2016 um 13:36 schrieb Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com>:
> 
> Please review http://cr.openjdk.java.net/~sundar/8071678/webrev.00/ for
> https://bugs.openjdk.java.net/browse/JDK-8071678
> 
> Thanks,
> 
> -Sundar
> 


From peter.levart at gmail.com  Tue Oct 18 14:14:33 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 18 Oct 2016 16:14:33 +0200
Subject: RFR: 8062389, 8029459, 8061950: Class.getMethod() is inconsistent
	with Class.getMethods() results + more
In-Reply-To: <0c67ae84-7bab-5944-1665-129dc451f3da@oracle.com>
References: <9ddc7fc5-68fe-bb78-b54d-6ba55880c018@gmail.com>
	<1F5607C1-9EFC-4881-A3A9-31CB948FE67C@oracle.com>
	<1a075ccc-bdd9-ffe9-6a7b-2e734c6233e8@gmail.com>
	<0c67ae84-7bab-5944-1665-129dc451f3da@oracle.com>
Message-ID: <07a6e24e-7dbc-6582-015f-62fd3fda24bd@gmail.com>

Hi Alan,


On 10/14/2016 02:16 PM, Alan Bateman wrote:
> On 13/10/2016 18:30, Peter Levart wrote:
>
>> Hi Paul, Alan,
>>
>> I incorporated Paul's suggestions into new webrev:
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.05/ 
>>
>>
>> This iteration also contains a nearly-exhaustive test of 
>> Class.getMethods(): PublicMethodsTest. It is actually a test 
>> generator. Given a Case template, it generates all variants of 
>> methods for each of the types in the case. Case1 contains 4 interface 
>> method variants ^ 3 interfaces * 4 class method variants ^ 3 classes 
>> = 4^6 = 4096 different sub-cases of which only 1379 compile. The 
>> results of those 1379 sub-cases are persisted in the Case1.results 
>> file. Running the test compares the persisted results with actual 
>> result of executing each sub-case. When running this test on plain 
>> JDK 9 (without patch), the test finds 218 sub-cases where results 
>> differ:
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/PublicMethodsTest.jtr 
>>
>>
>> Looking at those differences gives the impression of the effects of 
>> the patch.
> Overall I think this looks very good. I mostly focused on the 
> PublicMethods implementation to satisfy myself that it does selects 
> the most specific methods. In passing I wonder if "combine" or "merge" 
> might be better than "consolidate" for method name and terminology, a 
> minor point of course.

What about "coalesce" ?

>
> Given the behavior change then I think we'll need to capture it in a 
> release notes. I can't think of any libraries or frameworks that might 
> have see but something might come out of the woodwork and would be 
> nice to be able to point to a summary.

What do you think of specifying new behavior in javadoc(s) of 
getMethod() and getMethods() themselves? The description in those docs 
was never very precise. It is composed of a few seemingly unconnected 
statements which don't give a complete picture of what those methods 
return. It is now plainly wrong in getMethod() and does not give any 
clew about what it means to "inherit" a method from supertype in 
getMethods(). I have tried to capture the precise behavior in the 
changed javadocs that I present here:

http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/webrev.06/

Those javadocs refer to definitions of two other methods: 
Class.getDeclaredMethods() and Class.isAssignableFrom(). Their 
definitions are leveraged so javadocs of getMethod() and getMethods() 
themselves can be simpler - I removed the redundant statements that are 
a direct consequence of the algorithm description.

We could then perhaps point from release notes to the new javadocs or 
even include them.

>
> I assume that copyright headers will be added before this is pushed.

Added in above webrev.

> Also there is a @SuppressWarnings arguments that I don't recognize 
> (IDE specific)? It would good to trim back some of the really long 
> times in the test too (there are some >150 char lines that will be a 
> pain to review side-by-side when there are future changes).

The trimming was done. I have missed the @SuppressWarnings. Will do it 
in next iteration.

I have updated the test which now includes Class.getMethod() testing 
too. It also presents results (differences from expected) in a more 
pleasing way - just those results that differ are presented. Here's a 
result of running the test on unpatched JDK 9:

http://cr.openjdk.java.net/~plevart/jdk9-dev/Class.getMethods.new/PublicMethodsTest.jtr

There are some more differences now that Class.getMethod() is also part 
of the test. Some interesting consequences of the patch (expected=new 
behavior, actual=old behavior):

interface I { void m(); }
interface J {  }
interface K extends I, J { default void m() {} }
abstract class C {  }
abstract class D extends C implements I {  }
abstract class E extends D implements J, K {  }

expected: E.gM: K.m
   actual: E.gM: I.m

...here E.class.getMethods() has the same result between unpatched and 
patched JDK 9, just getMethod() returns different method. Which means 
that in present JDK, getMethod() returns a method that is not included 
in the getMethods() result.


interface I { void m(); }
interface J { void m(); }
interface K extends I, J { void m(); }
abstract class C { public abstract void m(); }
abstract class D extends C implements I {  }
abstract class E extends D implements J, K {  }

expected: E.gMs: [C.m]
   actual: E.gMs: [C.m, I.m, J.m, K.m]

expected: D.gMs: [C.m]
   actual: D.gMs: [C.m, I.m]

...huge difference, huh.


interface I { default void m() {} }
interface J { void m(); }
interface K extends I, J { void m(); }
abstract class C {  }
abstract class D extends C implements I {  }
abstract class E extends D implements J, K {  }

expected: E.gMs: [K.m]
   actual: E.gMs: [I.m, J.m, K.m]

expected: E.gM: K.m
   actual: E.gM: I.m

...difference in both getMethod() and getMethods().


Regards, Peter


From stanislav.smirnov at oracle.com  Tue Oct 18 15:06:22 2016
From: stanislav.smirnov at oracle.com (Stanislav Smirnov)
Date: Tue, 18 Oct 2016 18:06:22 +0300
Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under
	test/lib
In-Reply-To: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
Message-ID: <0404A092-57A8-4EF6-9423-885367B57FFD@oracle.com>

Hi,

I'm still looking for volunteers to review

Best regards,
Stanislav Smirnov





> On 05 Oct 2016, at 19:44, Stanislav Smirnov <stanislav.smirnov at oracle.com> wrote:
> 
> Hi,
> 
> Please review this fix for JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>.
> This one is similar to another one, I have sent earlier.
> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements.
> 
> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>
> Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ <http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/>
> Best regards,
> Stanislav Smirnov
> 
> 
> 
> 
> 


From stuart.marks at oracle.com  Tue Oct 18 17:15:39 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 18 Oct 2016 10:15:39 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
Message-ID: <8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>



On 10/17/16 6:34 PM, Martin Buchholz wrote:
> Most of this is for Stuart - very collection-y.
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/
>
> This includes a rewrite of ArrayDeque to bring it to parity with ArrayList
> (except for List features).
> The patch includes public methods ensureCapacity, trimToSize, replaceAll as in
> ArrayList, but we can defer making them public to a later change (or a later
> release), since new public methods are always controversial.  But I'd really
> like to get the performance/scalability changes in for jdk 9.
>
> It also includes a redo of ArrayList#removeIf to make it more efficient and
> consistent with behavior of other implementations, including ArrayDeque.
>
> The patches are a little tangled because they step on each other's toes.  File
> CollectionTest is in "miscellaneous", but it tests both the ArrayDeque and
> ArrayList changes.

Hi Martin,

ArrayList.removeIf() is nice. I like the way it recovers from the predicate 
having thrown an exception. I note that it uselessly copies the tail of the 
array to itself, if the predicate throws an exception and nothing has been 
deleted yet. You could add a r != w check, or possibly deleted > 0 if you 
prefer, or maybe we don't care because this is a rare (we hope) error recovery case.

***

I have some comments on ArrayDeque:

* Change in allocation/capacity policy.

The removal of the power-of-two restriction, and applying a 1.5x growth factor 
(same as ArrayList) seems sensible. Does this mean that the ability to compute 
the proper array index by using x & (length-1) wasn't worth it? Or at least not 
worth the extra tail wastage?

(Potential future enhancement: allocate the array lazily, like ArrayList)

Note, I haven't digested all the changes yet.

* API additions

I'm somewhat undecided about these.

I've always felt that the ensureCapacity() and trimToSize() methods were a sop 
added to ArrayList aimed at people converting from Vector. The 
ArrayList.ensureCapacity() method seems to be used a lot, but probably not to 
great effect, since addAll() gives exact sizing anyway. The trimToSize() method 
could be useful in some cases, but should we hold out for a generalized one, as 
requested by JDK-4619094?

On the other hand, ArrayDeque is modeling an array, so providing some size 
control seems mostly harmless.

The replaceAll() method is a bit oddly placed here. It's a default method on 
List (which ArrayDeque doesn't, and can't implement) so it's a bit strange to 
have a special method here with the same name and semantics as the one on List, 
but with a "fork" of the specification.

Maybe if JDK-8143850 is implemented to give a list view of an ArrayDeque, the 
replaceAll() method could be implemented on that:

     arrayDeque.asList().replaceAll(clazz::method);

***

I'm not sure what to think of the arrangement of the tests. The "core" 
collections, that is, collections in java.util, exclusive of 
java.util.concurrent, are mostly tested by tests in jdk/test/java/util. This 
patch adds tests for ArrayList and ArrayDeque inside of 
jdk/test/java/util/concurrent/tck. There's a test group jdk_collections_core 
that's intended to test the core collections, so it'll miss the new tests being 
added here.

I'm not sure what to suggest. The new tests use the JSR166TestCase framework, 
which is probably useful, and probably can't be used if the tests are moved 
elsewhere. I don't know what it would take to convert this to jtreg or testng. 
Or, the core test group could be modified to run selected JSR166 test cases, 
which doesn't seem quite right either. Suggestions?

***

I haven't looked at the other j.u.c changes. I haven't done so historically, but 
I can if you need a reviewer.

s'marks


From mandy.chung at oracle.com  Tue Oct 18 17:36:33 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 18 Oct 2016 10:36:33 -0700
Subject: =?windows-1252?Q?Re=3A_RFR=3A_6378384_=28reflect=29_subclass_can?=
	=?windows-1252?Q?=92t_access_superclass=92s_protected_fields_and?=
	=?windows-1252?Q?_methods_by_reflection?=
In-Reply-To: <0f3d14df-00d9-1f52-85e6-c64a3de55362@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
	<bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
	<B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>
	<0f3d14df-00d9-1f52-85e6-c64a3de55362@gmail.com>
Message-ID: <B9A0D252-082A-48C4-A8F3-B6F4B7F9C05B@oracle.com>


> On Oct 18, 2016, at 3:55 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> They do. What do you say about this variant (compressionLevel=9):
> 
>     http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.04/

+1.  Ship it!  Thanks for compressing it!  

Mandy


From paul.sandoz at oracle.com  Tue Oct 18 17:55:58 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 18 Oct 2016 10:55:58 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
Message-ID: <88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>

Hi,

The j.u.c. changes look ok to me as do the misc changes.

The ArrayDeque changes may also change the performance characteristics, a space/time trade-off e.g. in current version array bounds checks are strength reduced to a zero-based test of the array length. Unsure in practice if this really matters.

At this stage in Java 9 i would be inclined not to make ArrayDeque more ArrayList-like with new public methods ensureCapacity/trimToSize/replaceAll. Do you mind if we hold off an that for now and think more about that?

Testing-wise there is always gonna be some overlap. It would be nice to consolidate, although arguably in principle TCK has a slightly different focus. Now that the tck is in the OpenJDK repo perhaps we should add that to the jdk_collections_core?

Paul.

> On 18 Oct 2016, at 10:15, Stuart Marks <stuart.marks at oracle.com> wrote:
> 
> 
> 
> On 10/17/16 6:34 PM, Martin Buchholz wrote:
>> Most of this is for Stuart - very collection-y.
>> 
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/
>> 
>> This includes a rewrite of ArrayDeque to bring it to parity with ArrayList
>> (except for List features).
>> The patch includes public methods ensureCapacity, trimToSize, replaceAll as in
>> ArrayList, but we can defer making them public to a later change (or a later
>> release), since new public methods are always controversial.  But I'd really
>> like to get the performance/scalability changes in for jdk 9.
>> 
>> It also includes a redo of ArrayList#removeIf to make it more efficient and
>> consistent with behavior of other implementations, including ArrayDeque.
>> 
>> The patches are a little tangled because they step on each other's toes.  File
>> CollectionTest is in "miscellaneous", but it tests both the ArrayDeque and
>> ArrayList changes.
> 
> Hi Martin,
> 
> ArrayList.removeIf() is nice. I like the way it recovers from the predicate having thrown an exception. I note that it uselessly copies the tail of the array to itself, if the predicate throws an exception and nothing has been deleted yet. You could add a r != w check, or possibly deleted > 0 if you prefer, or maybe we don't care because this is a rare (we hope) error recovery case.
> 
> ***
> 
> I have some comments on ArrayDeque:
> 
> * Change in allocation/capacity policy.
> 
> The removal of the power-of-two restriction, and applying a 1.5x growth factor (same as ArrayList) seems sensible. Does this mean that the ability to compute the proper array index by using x & (length-1) wasn't worth it? Or at least not worth the extra tail wastage?
> 
> (Potential future enhancement: allocate the array lazily, like ArrayList)
> 
> Note, I haven't digested all the changes yet.
> 
> * API additions
> 
> I'm somewhat undecided about these.
> 
> I've always felt that the ensureCapacity() and trimToSize() methods were a sop added to ArrayList aimed at people converting from Vector. The ArrayList.ensureCapacity() method seems to be used a lot, but probably not to great effect, since addAll() gives exact sizing anyway. The trimToSize() method could be useful in some cases, but should we hold out for a generalized one, as requested by JDK-4619094?
> 
> On the other hand, ArrayDeque is modeling an array, so providing some size control seems mostly harmless.
> 
> The replaceAll() method is a bit oddly placed here. It's a default method on List (which ArrayDeque doesn't, and can't implement) so it's a bit strange to have a special method here with the same name and semantics as the one on List, but with a "fork" of the specification.
> 
> Maybe if JDK-8143850 is implemented to give a list view of an ArrayDeque, the replaceAll() method could be implemented on that:
> 
>    arrayDeque.asList().replaceAll(clazz::method);
> 
> ***
> 
> I'm not sure what to think of the arrangement of the tests. The "core" collections, that is, collections in java.util, exclusive of java.util.concurrent, are mostly tested by tests in jdk/test/java/util. This patch adds tests for ArrayList and ArrayDeque inside of jdk/test/java/util/concurrent/tck. There's a test group jdk_collections_core that's intended to test the core collections, so it'll miss the new tests being added here.
> 
> I'm not sure what to suggest. The new tests use the JSR166TestCase framework, which is probably useful, and probably can't be used if the tests are moved elsewhere. I don't know what it would take to convert this to jtreg or testng. Or, the core test group could be modified to run selected JSR166 test cases, which doesn't seem quite right either. Suggestions?
> 
> ***
> 
> I haven't looked at the other j.u.c changes. I haven't done so historically, but I can if you need a reviewer.
> 
> s'marks
> 


From martinrb at google.com  Tue Oct 18 18:00:33 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 11:00:33 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
Message-ID: <CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>

On Tue, Oct 18, 2016 at 10:15 AM, Stuart Marks <stuart.marks at oracle.com>
wrote:

>
>
> On 10/17/16 6:34 PM, Martin Buchholz wrote:
>
>> Most of this is for Stuart - very collection-y.
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-
>> jdk9-integration/
>>
>> This includes a rewrite of ArrayDeque to bring it to parity with ArrayList
>> (except for List features).
>> The patch includes public methods ensureCapacity, trimToSize, replaceAll
>> as in
>> ArrayList, but we can defer making them public to a later change (or a
>> later
>> release), since new public methods are always controversial.  But I'd
>> really
>> like to get the performance/scalability changes in for jdk 9.
>>
>> It also includes a redo of ArrayList#removeIf to make it more efficient
>> and
>> consistent with behavior of other implementations, including ArrayDeque.
>>
>> The patches are a little tangled because they step on each other's toes.
>> File
>> CollectionTest is in "miscellaneous", but it tests both the ArrayDeque and
>> ArrayList changes.
>>
>
> Hi Martin,
>
> ArrayList.removeIf() is nice. I like the way it recovers from the
> predicate having thrown an exception. I note that it uselessly copies the
> tail of the array to itself, if the predicate throws an exception and
> nothing has been deleted yet. You could add a r != w check, or possibly
> deleted > 0 if you prefer, or maybe we don't care because this is a rare
> (we hope) error recovery case.
>
>
I had the same thoughts... I added checks for deleted > 0


> ***
>
> I have some comments on ArrayDeque:
>
> * Change in allocation/capacity policy.
>
> The removal of the power-of-two restriction, and applying a 1.5x growth
> factor (same as ArrayList) seems sensible. Does this mean that the ability
> to compute the proper array index by using x & (length-1) wasn't worth it?
> Or at least not worth the extra tail wastage?
>

There's no integer division to optimize, as with hash tables.

It was the horror of user discovering that ArrayDeque(2^n) allocated
2^(n+1) that persuaded me to go down this road.
It's also very useful to allocate exactly the requested size on
ArrayDeque(m) or be precise with trimToSize or allow growth up to (almost)
Integer.MAX_VALUE.
A circular buffer is a very simple data structure - our implementation
should be a model!


> (Potential future enhancement: allocate the array lazily, like ArrayList)
>
> Note, I haven't digested all the changes yet.
>
> * API additions
>
> I'm somewhat undecided about these.
>
> I've always felt that the ensureCapacity() and trimToSize() methods were a
> sop added to ArrayList aimed at people converting from Vector. The
> ArrayList.ensureCapacity() method seems to be used a lot, but probably not
> to great effect, since addAll() gives exact sizing anyway. The trimToSize()
> method could be useful in some cases, but should we hold out for a
> generalized one, as requested by JDK-4619094?
>

ensureCapacity is the least useful - just saving less than a factor of 2
allocation on growth.  trimToSize seems more important, since it saves
memory "permanently".


> On the other hand, ArrayDeque is modeling an array, so providing some size
> control seems mostly harmless.
>
> The replaceAll() method is a bit oddly placed here. It's a default method
> on List (which ArrayDeque doesn't, and can't implement) so it's a bit
> strange to have a special method here with the same name and semantics as
> the one on List, but with a "fork" of the specification.
>
>
yeah, it's adding a List method even though the mega-feature of List view
is deferred.


> Maybe if JDK-8143850 is implemented to give a list view of an ArrayDeque,
> the replaceAll() method could be implemented on that:
>
>     arrayDeque.asList().replaceAll(clazz::method);
>

yeah, the idea might be that we add List methods like get(i) and replaceAll
directly on ArrayDeque, but only when you call asList do you get something
that implements List ... which is ... odd.


***
>
> I'm not sure what to think of the arrangement of the tests. The "core"
> collections, that is, collections in java.util, exclusive of
> java.util.concurrent, are mostly tested by tests in jdk/test/java/util.
> This patch adds tests for ArrayList and ArrayDeque inside of
> jdk/test/java/util/concurrent/tck. There's a test group
> jdk_collections_core that's intended to test the core collections, so it'll
> miss the new tests being added here.
>
> I'm not sure what to suggest. The new tests use the JSR166TestCase
> framework, which is probably useful, and probably can't be used if the
> tests are moved elsewhere. I don't know what it would take to convert this
> to jtreg or testng. Or, the core test group could be modified to run
> selected JSR166 test cases, which doesn't seem quite right either.
> Suggestions?
>
>
There's already a certain amount of mixing, e.g. stream tests sometimes
test j.u.c. and Queue tests sometimes test all the Queue implementations.
Unlike non-test code, some redundancy and divergence of testing frameworks
and styles is fine.  I still haven't found a way of writing shared tests
for implementations of an interface that I really like.


> ***
>
> I haven't looked at the other j.u.c changes. I haven't done so
> historically, but I can if you need a reviewer.
>

Look at CollectionTest in misc .

From paul.sandoz at oracle.com  Tue Oct 18 18:41:30 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 18 Oct 2016 11:41:30 -0700
Subject: RFR 8163553 java.lang.LinkageError from test
	java/lang/ThreadGroup/Stop.java
Message-ID: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com>

Hi,

Please review:

  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8163553-vh-mh-link-errors-not-wrapped/webrev/

This is the issue that motivated a change in the behaviour of indy wrapping Errors in BootstrapMethodError, JDK-8166974. I plan to push this issue with JDK-8166974 to hs, since they are related in behaviour even though there is no direct dependency between the patches.


When invoking signature-polymorphic methods a similar but hardcoded dance occurs, with an appeal to Java code, to link the call site.

- MethodHandle.invoke/invokeExact (and the VH methods) would wrap all Errors in LinkageError. Now they are passed through, thus an Error like ThreadDeath is not wrapped.

- MethodHandle.invoke/invokeExact/invokeBasic throw Throwable, and in certain cases the Throwable is wrapped in an InternalError. In many other cases Error and RuntimeException are propagated, which i think in general is the right pattern, so i consistently applied that.

- I updated StringConcatFactory to also pass through Errors and avoid unduly wrapping StringConcatException in another instance of StringConcatException. (LambdaMetafactory and associated classes required no changes.)

Thanks,
Paul.

From peter.levart at gmail.com  Tue Oct 18 18:41:51 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 18 Oct 2016 20:41:51 +0200
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <B9A0D252-082A-48C4-A8F3-B6F4B7F9C05B@oracle.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
	<bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
	<B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>
	<0f3d14df-00d9-1f52-85e6-c64a3de55362@gmail.com>
	<B9A0D252-082A-48C4-A8F3-B6F4B7F9C05B@oracle.com>
Message-ID: <8d14ab62-7bc3-d69d-642e-6e645bf846f4@gmail.com>

Hi Mandy,


On 10/18/2016 07:36 PM, Mandy Chung wrote:
>> On Oct 18, 2016, at 3:55 AM, Peter Levart <peter.levart at gmail.com> wrote:
>>
>> They do. What do you say about this variant (compressionLevel=9):
>>
>>      http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.04/
> +1.  Ship it!  Thanks for compressing it!
>
> Mandy
>

Thanks for reviewing. Pushed.

Hope it's not too bad that I managed to enter non-ascii characters into 
commit message. I was copy-pasting from Jira issue title and didn't 
notice the difference between ' and ? . Probably an artifact of 
migrating issues from SUN's old bug-tracking system. This issue really 
had a long beard...

Regards, Peter


From tom.w.hood at gmail.com  Tue Oct 18 18:57:27 2016
From: tom.w.hood at gmail.com (Tom Hood)
Date: Tue, 18 Oct 2016 11:57:27 -0700
Subject: JDK 9 org.omg.CORBA.ORBSingletonClass loading will not use context
	class loader
Message-ID: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>

Hello,

We have a Java Webstart application that uses CORBA and requires an old
version of the Visibroker ORB (5.2.1) that will not launch with Java 9 due
to its inclusion of a change originally added to 7u55 and later backed out:
       Bug ID: JDK-8042789 org.omg.CORBA.ORBSingletonClass loading no
longer uses context class loader
<http://bugs.java.com/view_bug.do?bug_id=8042789>

What approaches should we consider for our application to support Java 9 ?

Constraints we must satisfy:

   - cannot upgrade visibroker
   - continue to support clients with java versions 1.6+

Constraints we are trying to satisfy:

   - avoid using different vendor ORB
   - avoid replacing CORBA
   - avoid modifying user's machine configuration (e.g. CLASSPATH) or jre
   installation with application-specifics (i.e. java.security, java.policy,
   etc.)

The two properties we set in the jnlp to specify the visibroker orb classes
are:
        <property name="jnlp.org.omg.CORBA.ORBClass"
value="com.inprise.vbroker.orb.ORB"/>
        <property name="jnlp.org.omg.CORBA.ORBSingletonClass"
value="com.inprise.vbroker.orb.ORBSingleton"/>

The two approaches I'm considering so far are:

   1. add vb jar to system class loader at start of application using
   URLClassLoader.addURL
   2. edit vb jar to avoid calling org.omg.CORBA.ORB.init()


*Approach #1: add vb jar to system class loader at start of application*

This works with 7u55.  The application will launch as long as I disable the
security manager (System.setSecurityManager(null)) or use a policy file
entry that explicitly grants AllPermission to the vb jar.  This is
necessary even though our jnlp file has always specified
<security><all-permissions/></security>.  To add the vb jar to the system
class loader, it appears necessary to call the protected
URLClassLoader.addURL method via reflection.

Questions I have about Approach #1:

   1. Are there additional security concerns with disabling the security
   manager at the start of main instead of granting AllPermission via policy
   files?  Our goal is for our application to continue to have the same access
   the user has to the user's machine (mostly windows).  The users do not
   have/run our application with administrator/root privileges.
   2. Is there a better way to make the vb jar visible to the system class
   loader when the application starts without calling a protected addURL
   method?


*Approach #2: edit vb jar to avoid calling org.omg.CORBA.ORB.init()*

I haven't gone down this path too far other than attempt to identify edits
that would be needed.  The source is unavailable (javap and
http://asm.ow2.org have been helpful).  Here's the list of possible edits I
have so far:

   1. Replace calls to org.omg.CORBA.ORB.init() with a call to our own
   class such as AppORB.init() that always returns the vb ORBSingleton instance
   2. Replace any object that is being downcast to the vb ORBSingleton
   (and/or its subclasses, if any) with a call to AppORB.init()
   3. Check if vb jar calls methods in the jre which then call ORB.init().
   If such calls exist, then I think I may need to replace these calls with
   something equivalent, because otherwise the default jre provided singleton
   will be used (com.sun.corba.se.impl.orb.ORBSingleton).  Maybe it's okay to
   mix the two Singleton ORB implementations in the same jvm, but not knowing
   enough about CORBA at this point it seems like a potential problem.  The
   javadoc does say it is a restricted implementation allowing use as a
   TypeCode factory only, but I'm not sure yet if it is possible to mix and
   match TypeCodes from the two ORB implementations.  Feels a little dicey to
   me.


Please let me know if you can think of any alternate approaches to consider
or potential issues/solutions to the ones outlined above.

Thank you,
-- Tom

P.S. I also posted this question at
https://community.oracle.com/message/14072662#14072662

From martinrb at google.com  Tue Oct 18 19:07:52 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 12:07:52 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
Message-ID: <CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>

On Tue, Oct 18, 2016 at 10:55 AM, Paul Sandoz <paul.sandoz at oracle.com>
wrote:

>
> Testing-wise there is always gonna be some overlap. It would be nice to
> consolidate, although arguably in principle TCK has a slightly different
> focus. Now that the tck is in the OpenJDK repo perhaps we should add that
> to the jdk_collections_core?
>

I don't think the distinction between  jdk_collections_core and
jdk_collections is useful.  Too many tests test both kinds of collections,
and there's  inheritance.  Just get rid of jdk_collections_core, and add:

test/java/util/Spliterator
test/java/util/stream/test/org/openjdk/tests/java/util/stream/CollectorsTest.java
test/java/util/stream/test/org/openjdk/tests/java/util/stream/CollectionAndMapModifyStreamTest.java

From joe.darcy at oracle.com  Tue Oct 18 19:16:38 2016
From: joe.darcy at oracle.com (joe darcy)
Date: Tue, 18 Oct 2016 12:16:38 -0700
Subject: [9] RFR of JDK-8085192: java/rmi/activation/Activatable tests
	fail intermittently due to "Port already in use"
In-Reply-To: <75410cc2-45c4-f5e7-46b0-4f254e1c3702@Oracle.com>
References: <3c0bdf63-f55c-5b93-6d74-58e26916cb3b@oracle.com>
	<f881b76f-2a76-5abd-3118-7f6fe12d9205@Oracle.com>
	<e3b084d3-de5e-700f-2f69-22f91a0f80bf@oracle.com>
	<02147682-541f-9746-b6da-2cc2800165b5@Oracle.com>
	<3E39A1E9-0764-404D-9824-85CFB75113FF@oracle.com>
	<E4E9CBA7-4EC3-4CAE-AD9C-E6190ABE7837@oracle.com>
	<02875ca6-0caa-cf19-b9ae-2018e92f0be5@oracle.com>
	<2e13f9fa-cfeb-94bb-0ed5-6319c32b3e1f@Oracle.com>
	<745be6ee-810a-4d2e-76b9-c0df79e6d6db@oracle.com>
	<75410cc2-45c4-f5e7-46b0-4f254e1c3702@Oracle.com>
Message-ID: <4faade62-a1c1-c9e0-8c92-aef1e13a26c2@oracle.com>

Hello,


On 10/12/2016 8:50 AM, Roger Riggs wrote:
> Hi Chris,
>
>
> On 10/10/2016 9:43 AM, Chris Hegarty wrote:
>> Roger,
>>
[snip]

>> Right, I was a little uneasy with this too, to being with, but
>> it has grown on me ( since it appears stable and reliable in
>> all my builds and tests ). Also the surface area of the code
>> change is very small, and the inherit channel mechanism is well
>> specified and stable.
>
> ok, lets give it a try.
>
>

I agree it is time to give another approach to these tests a try. This 
set of tests muddies our overall test results with intermittent 
failures. I suggest giving Chris's approach a try, monitoring any 
changes in the failure rate of the tests for a month or two, and then 
reassessing if the tests need further revisiting.

Thanks,

-Joe


From Alan.Bateman at oracle.com  Tue Oct 18 20:30:23 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 18 Oct 2016 21:30:23 +0100
Subject: JDK 9 org.omg.CORBA.ORBSingletonClass loading will not use
	context class loader
In-Reply-To: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>
References: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>
Message-ID: <ad81cff8-f33e-a387-2f2d-be94d3d30008@oracle.com>

On 18/10/2016 19:57, Tom Hood wrote:

> Hello,
>
> We have a Java Webstart application that uses CORBA and requires an old
> version of the Visibroker ORB (5.2.1) that will not launch with Java 9 due
> to its inclusion of a change originally added to 7u55 and later backed out:
>         Bug ID: JDK-8042789 org.omg.CORBA.ORBSingletonClass loading no
> longer uses context class loader
> <http://bugs.java.com/view_bug.do?bug_id=8042789>
>
> What approaches should we consider for our application to support Java 9 ?
>
The singleton ORB is the system-wide type code factory so having it be 
located via the TCCL is highly problematic (many reasons including 
security, memory leaks, and of course it's not going to work then there 
two or more applications/contexts in the same VM trying to do the same 
thing).

I assume your application can use the 2-arg ORB.init to create the ORB, 
in which case the implementation will be located via the TCCL (so you 
can bundle the ORB implementation with the application). From your mail 
then it sounds like the issue is that something in Visibroker is using 
the zero-arg ORB.init and so gets the JDK ORB as the singleton ORB. Does 
this lead to an interop issue or does Visibroker assuming the singleton 
ORB is its own type?

-Alan

From paul.sandoz at oracle.com  Tue Oct 18 20:52:38 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 18 Oct 2016 13:52:38 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
	<CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
Message-ID: <159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>


> On 18 Oct 2016, at 12:07, Martin Buchholz <martinrb at google.com> wrote:
> 
> 
> 
> On Tue, Oct 18, 2016 at 10:55 AM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
> 
> Testing-wise there is always gonna be some overlap. It would be nice to consolidate, although arguably in principle TCK has a slightly different focus. Now that the tck is in the OpenJDK repo perhaps we should add that to the jdk_collections_core?
> 

Ah, i see they are already implicitly included under the jdk_concurrent group.

# All collections, core and concurrent
jdk_collections = \
    :jdk_collections_core \
    :jdk_concurrent

# java.util.concurrent
# Includes concurrent collections plus other stuff
# Maintained by JSR-166 EG (Doug Lea et al)
jdk_concurrent = \
    java/util/concurrent

Hmm? not sure i have a strong opinion here regarding the removal of the core group. I think the comment should be updated to state tck tests are included.


> I don't think the distinction between  jdk_collections_core and jdk_collections is useful.  Too many tests test both kinds of collections, and there's  inheritance.  Just get rid of jdk_collections_core, and add:
> 
> test/java/util/Spliterator

Yes, that?s useful to add to the collection group.


> test/java/util/stream/test/org/openjdk/tests/java/util/stream/CollectorsTest.java

The above tests j.u.s.Collector implementations rather than collections themselves (even thought it produces collections), so i think it should not be added.


> test/java/util/stream/test/org/openjdk/tests/java/util/stream/CollectionAndMapModifyStreamTest.java

Ok.

Paul.

From martinrb at google.com  Tue Oct 18 20:58:58 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 13:58:58 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
	<CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
	<159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>
Message-ID: <CA+kOe09ftWWD7TG-i+4UGBa-M9Q=-U8Ay9kv7i_9G62JaL4GxQ@mail.gmail.com>

Latest webrev defers potential new methods:

   /* TODO: public */ private void trimToSize()

From vitalyd at gmail.com  Tue Oct 18 23:00:21 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 18 Oct 2016 19:00:21 -0400
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
Message-ID: <CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>

On Tuesday, October 18, 2016, Martin Buchholz <martinrb at google.com> wrote:

> On Tue, Oct 18, 2016 at 10:15 AM, Stuart Marks <stuart.marks at oracle.com
> <javascript:;>>
> wrote:
>
> >
> >
> > On 10/17/16 6:34 PM, Martin Buchholz wrote:
> >
> >> Most of this is for Stuart - very collection-y.
> >>
> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-
> >> jdk9-integration/
> >>
> >> This includes a rewrite of ArrayDeque to bring it to parity with
> ArrayList
> >> (except for List features).
> >> The patch includes public methods ensureCapacity, trimToSize, replaceAll
> >> as in
> >> ArrayList, but we can defer making them public to a later change (or a
> >> later
> >> release), since new public methods are always controversial.  But I'd
> >> really
> >> like to get the performance/scalability changes in for jdk 9.
> >>
> >> It also includes a redo of ArrayList#removeIf to make it more efficient
> >> and
> >> consistent with behavior of other implementations, including ArrayDeque.
> >>
> >> The patches are a little tangled because they step on each other's toes.
> >> File
> >> CollectionTest is in "miscellaneous", but it tests both the ArrayDeque
> and
> >> ArrayList changes.
> >>
> >
> > Hi Martin,
> >
> > ArrayList.removeIf() is nice. I like the way it recovers from the
> > predicate having thrown an exception. I note that it uselessly copies the
> > tail of the array to itself, if the predicate throws an exception and
> > nothing has been deleted yet. You could add a r != w check, or possibly
> > deleted > 0 if you prefer, or maybe we don't care because this is a rare
> > (we hope) error recovery case.
> >
> >
> I had the same thoughts... I added checks for deleted > 0
>
>
> > ***
> >
> > I have some comments on ArrayDeque:
> >
> > * Change in allocation/capacity policy.
> >
> > The removal of the power-of-two restriction, and applying a 1.5x growth
> > factor (same as ArrayList) seems sensible. Does this mean that the
> ability
> > to compute the proper array index by using x & (length-1) wasn't worth
> it?
> > Or at least not worth the extra tail wastage?
> >
>
> There's no integer division to optimize, as with hash tables.

But it does add some new branches, doesn't it? Potentially branches that
aren't taken prior to JIT compilation, but taken later = deopt.

Curious if you ran some benchmarks on ArrayDeque.

Also, some odd stylistic things, such as:

while (s == (capacity = (elements = this.elements).length))

Is this an attempt to help the JIT or something?


> It was the horror of user discovering that ArrayDeque(2^n) allocated
> 2^(n+1) that persuaded me to go down this road.
> It's also very useful to allocate exactly the requested size on
> ArrayDeque(m) or be precise with trimToSize or allow growth up to (almost)
> Integer.MAX_VALUE.
> A circular buffer is a very simple data structure - our implementation
> should be a model!
>
>
> > (Potential future enhancement: allocate the array lazily, like ArrayList)
> >
> > Note, I haven't digested all the changes yet.
> >
> > * API additions
> >
> > I'm somewhat undecided about these.
> >
> > I've always felt that the ensureCapacity() and trimToSize() methods were
> a
> > sop added to ArrayList aimed at people converting from Vector. The
> > ArrayList.ensureCapacity() method seems to be used a lot, but probably
> not
> > to great effect, since addAll() gives exact sizing anyway. The
> trimToSize()
> > method could be useful in some cases, but should we hold out for a
> > generalized one, as requested by JDK-4619094?
> >
>
> ensureCapacity is the least useful - just saving less than a factor of 2
> allocation on growth.  trimToSize seems more important, since it saves
> memory "permanently".
>
>
> > On the other hand, ArrayDeque is modeling an array, so providing some
> size
> > control seems mostly harmless.
> >
> > The replaceAll() method is a bit oddly placed here. It's a default method
> > on List (which ArrayDeque doesn't, and can't implement) so it's a bit
> > strange to have a special method here with the same name and semantics as
> > the one on List, but with a "fork" of the specification.
> >
> >
> yeah, it's adding a List method even though the mega-feature of List view
> is deferred.
>
>
> > Maybe if JDK-8143850 is implemented to give a list view of an ArrayDeque,
> > the replaceAll() method could be implemented on that:
> >
> >     arrayDeque.asList().replaceAll(clazz::method);
> >
>
> yeah, the idea might be that we add List methods like get(i) and replaceAll
> directly on ArrayDeque, but only when you call asList do you get something
> that implements List ... which is ... odd.
>
>
> ***
> >
> > I'm not sure what to think of the arrangement of the tests. The "core"
> > collections, that is, collections in java.util, exclusive of
> > java.util.concurrent, are mostly tested by tests in jdk/test/java/util.
> > This patch adds tests for ArrayList and ArrayDeque inside of
> > jdk/test/java/util/concurrent/tck. There's a test group
> > jdk_collections_core that's intended to test the core collections, so
> it'll
> > miss the new tests being added here.
> >
> > I'm not sure what to suggest. The new tests use the JSR166TestCase
> > framework, which is probably useful, and probably can't be used if the
> > tests are moved elsewhere. I don't know what it would take to convert
> this
> > to jtreg or testng. Or, the core test group could be modified to run
> > selected JSR166 test cases, which doesn't seem quite right either.
> > Suggestions?
> >
> >
> There's already a certain amount of mixing, e.g. stream tests sometimes
> test j.u.c. and Queue tests sometimes test all the Queue implementations.
> Unlike non-test code, some redundancy and divergence of testing frameworks
> and styles is fine.  I still haven't found a way of writing shared tests
> for implementations of an interface that I really like.
>
>
> > ***
> >
> > I haven't looked at the other j.u.c changes. I haven't done so
> > historically, but I can if you need a reviewer.
> >
>
> Look at CollectionTest in misc .
>


-- 
Sent from my phone

From martinrb at google.com  Tue Oct 18 23:18:26 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 16:18:26 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
Message-ID: <CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>

On Tue, Oct 18, 2016 at 4:00 PM, Vitaly Davidovich <vitalyd at gmail.com>
wrote:

>
>> > * Change in allocation/capacity policy.
>> >
>> > The removal of the power-of-two restriction, and applying a 1.5x growth
>> > factor (same as ArrayList) seems sensible. Does this mean that the
>> ability
>> > to compute the proper array index by using x & (length-1) wasn't worth
>> it?
>> > Or at least not worth the extra tail wastage?
>> >
>>
>> There's no integer division to optimize, as with hash tables.
>
> But it does add some new branches, doesn't it? Potentially branches that
> aren't taken prior to JIT compilation, but taken later = deopt.
>

If it's a smidgeon slower, I don't care that much; improvement in
flexibility and scalability are more important.

Of course, I do care about algorithmic improvements to e.g. removeIf


> Curious if you ran some benchmarks on ArrayDeque.
>

Not yet.  Who would like to show how slow the code is?


> Also, some odd stylistic things, such as:
>
> while (s == (capacity = (elements = this.elements).length))
>
> Is this an attempt to help the JIT or something?
>
> "jsr166 style" - makes it easy on javac and the JIT, if not for humans.

From vitalyd at gmail.com  Tue Oct 18 23:26:26 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 18 Oct 2016 19:26:26 -0400
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
	<CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
Message-ID: <CAHjP37Gqy4hTh-hso2rg9a1VrW2R+yRq9LnJCvwBxDv37qhS2g@mail.gmail.com>

On Tuesday, October 18, 2016, Martin Buchholz <martinrb at google.com> wrote:

>
>
> On Tue, Oct 18, 2016 at 4:00 PM, Vitaly Davidovich <vitalyd at gmail.com
> <javascript:_e(%7B%7D,'cvml','vitalyd at gmail.com');>> wrote:
>
>>
>>> > * Change in allocation/capacity policy.
>>> >
>>> > The removal of the power-of-two restriction, and applying a 1.5x growth
>>> > factor (same as ArrayList) seems sensible. Does this mean that the
>>> ability
>>> > to compute the proper array index by using x & (length-1) wasn't worth
>>> it?
>>> > Or at least not worth the extra tail wastage?
>>> >
>>>
>>> There's no integer division to optimize, as with hash tables.
>>
>> But it does add some new branches, doesn't it? Potentially branches that
>> aren't taken prior to JIT compilation, but taken later = deopt.
>>
>
> If it's a smidgeon slower, I don't care that much; improvement in
> flexibility and scalability are more important.
>
Well, others may :).  I'm not saying it's slower, I don't know, but you're
dismissing that possibility a bit too quickly, methinks.

>
> Of course, I do care about algorithmic improvements to e.g. removeIf
>
Sure, no debate.

>
>
>> Curious if you ran some benchmarks on ArrayDeque.
>>
>
> Not yet.  Who would like to show how slow the code is?
>
>
>> Also, some odd stylistic things, such as:
>>
>> while (s == (capacity = (elements = this.elements).length))
>>
>> Is this an attempt to help the JIT or something?
>>
>> "jsr166 style" - makes it easy on javac and the JIT, if not for humans.
>
In some of the cases here, I'd consider it a significant JIT failure if the
"jsr166 style" makes a difference (I'm not sure how this makes a difference
to javac - did you mean interpreter?).  Is this jsr style a carryover from
long ago? Maybe it needs to be reconsidered.  The code clarity isn't
terrible, but it would be unfortunate to do this in simple cases where the
compiler ought to handle it just fine - does it not?


-- 
Sent from my phone

From martinrb at google.com  Tue Oct 18 23:40:13 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 16:40:13 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CAHjP37Gqy4hTh-hso2rg9a1VrW2R+yRq9LnJCvwBxDv37qhS2g@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
	<CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
	<CAHjP37Gqy4hTh-hso2rg9a1VrW2R+yRq9LnJCvwBxDv37qhS2g@mail.gmail.com>
Message-ID: <CA+kOe0_MCB2UoW1YPaywzeVU55vMYf3zOrGhKLWxi7VTQOwP+Q@mail.gmail.com>

On Tue, Oct 18, 2016 at 4:26 PM, Vitaly Davidovich <vitalyd at gmail.com>
wrote:

> "jsr166 style" - makes it easy on javac and the JIT, if not for humans.
>>
> In some of the cases here, I'd consider it a significant JIT failure if
> the "jsr166 style" makes a difference (I'm not sure how this makes a
> difference to javac - did you mean interpreter?).  Is this jsr style a
> carryover from long ago? Maybe it needs to be reconsidered.  The code
> clarity isn't terrible, but it would be unfortunate to do this in simple
> cases where the compiler ought to handle it just fine - does it not?
>

I consider it a significant failure that

   1. JDK-6445664 <https://bugs.openjdk.java.net/browse/JDK-6445664>

Eliminate remaining performance penalty for using assert

still hasn't been fixed.

From david.holmes at oracle.com  Wed Oct 19 01:27:29 2016
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 19 Oct 2016 11:27:29 +1000
Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under
	test/lib
In-Reply-To: <0404A092-57A8-4EF6-9423-885367B57FFD@oracle.com>
References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
	<0404A092-57A8-4EF6-9423-885367B57FFD@oracle.com>
Message-ID: <8d4ab144-30ac-faf6-ddc9-a975b14ddb5f@oracle.com>

Hi Stanislav,

On 19/10/2016 1:06 AM, Stanislav Smirnov wrote:
> Hi,
>
> I'm still looking for volunteers to review

This is trivial. Reviewed. I will push for you, to jdk9/dev.

Thanks,
David

> Best regards,
> Stanislav Smirnov
>
>
>
>
>
>> On 05 Oct 2016, at 19:44, Stanislav Smirnov <stanislav.smirnov at oracle.com> wrote:
>>
>> Hi,
>>
>> Please review this fix for JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>.
>> This one is similar to another one, I have sent earlier.
>> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements.
>>
>> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>
>> Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ <http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/>
>> Best regards,
>> Stanislav Smirnov
>>
>>
>>
>>
>>
>

From vitalyd at gmail.com  Wed Oct 19 01:33:38 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 18 Oct 2016 21:33:38 -0400
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe0_MCB2UoW1YPaywzeVU55vMYf3zOrGhKLWxi7VTQOwP+Q@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
	<CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
	<CAHjP37Gqy4hTh-hso2rg9a1VrW2R+yRq9LnJCvwBxDv37qhS2g@mail.gmail.com>
	<CA+kOe0_MCB2UoW1YPaywzeVU55vMYf3zOrGhKLWxi7VTQOwP+Q@mail.gmail.com>
Message-ID: <CAHjP37FRwd4kQXWT23=qzMcm3HeK52HmBHt=w0h5_8vFss-LAw@mail.gmail.com>

On Tuesday, October 18, 2016, Martin Buchholz <martinrb at google.com> wrote:

>
>
> On Tue, Oct 18, 2016 at 4:26 PM, Vitaly Davidovich <vitalyd at gmail.com
> <javascript:_e(%7B%7D,'cvml','vitalyd at gmail.com');>> wrote:
>
>> "jsr166 style" - makes it easy on javac and the JIT, if not for humans.
>>>
>> In some of the cases here, I'd consider it a significant JIT failure if
>> the "jsr166 style" makes a difference (I'm not sure how this makes a
>> difference to javac - did you mean interpreter?).  Is this jsr style a
>> carryover from long ago? Maybe it needs to be reconsidered.  The code
>> clarity isn't terrible, but it would be unfortunate to do this in simple
>> cases where the compiler ought to handle it just fine - does it not?
>>
>
> I consider it a significant failure that
>
>    1. JDK-6445664 <https://bugs.openjdk.java.net/browse/JDK-6445664>
>
> Eliminate remaining performance penalty for using assert
>
> still hasn't been fixed.
>
Indeed - the whole inlining based on simple bytecode size is a big problem
(I've brought this up in various contexts several times on the compiler
list, but this is a known issue).  But, some of the methods using the jsr
style look to already be >35 bytes.  I've seen this style in other places
in the JDK where the method is definitely above 35.  My thinking was this
must be some manual attempt to force the compiler to put the value in a
register or stack slot, for fear that it wouldn't do it on its own because
it can't prove it's safe, rather than inlining.

Anyway, the style is odd, perhaps outdated to some degree, but not that big
of a deal in the grand scheme of things (IMO).

I mostly wanted to know of impact on ArrayDeque performance due to the code
restructuring.


-- 
Sent from my phone

From tom.w.hood at gmail.com  Wed Oct 19 02:28:06 2016
From: tom.w.hood at gmail.com (Tom Hood)
Date: Tue, 18 Oct 2016 19:28:06 -0700
Subject: JDK 9 org.omg.CORBA.ORBSingletonClass loading will not use
	context class loader
In-Reply-To: <ad81cff8-f33e-a387-2f2d-be94d3d30008@oracle.com>
References: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>
	<ad81cff8-f33e-a387-2f2d-be94d3d30008@oracle.com>
Message-ID: <CABO0Qi5ycxGf46b6tiz8Cgi2QSwnmbRcTqa_uTWQtpxmFXh8xQ@mail.gmail.com>

It's unclear if there really is an interop issue.   The application will
launch with 7u55 if I don't set the ORBSingletonClass property, although
that isn't how visibroker 5.2.1 was intended to run, so not setting it
worries me.

Our application does call ORB.init(args,props) once at startup and use that
instance throughout the code we have written.

However, visibroker calls ORB.init() all over the idl generated code and
the vb jar itself.

The javap of the vb jar does show some downcasts to its
com.inprise.vbroker.orb.ORBSingleton implementation in a few classes, but I
haven't tracked down if that code could execute in our application or if
the object it tries to downcast originated from ORB.init().  The simple
test of a basic launch and performing some activities within the
application seem to work.  Sadly we don't have any automated tests that
exercise our use of CORBA (java client to/from c++ server).

The idea of mixing ORB implementations worries me, even if only for the
singleton ORB.  I'm new to CORBA and the client side of our application and
still a newbie to java as I've been primarily working on our c++ server.

Do the reasons for the JDK change to ORB.init() apply in our use case?
There's only one application/context and ORB vendor implementation running
in the jvm in production.  Do you think approach #1 might be a reasonably
safe workaround without the risk of hidden interop issues?


On Tue, Oct 18, 2016 at 1:30 PM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> On 18/10/2016 19:57, Tom Hood wrote:
>
> Hello,
>>
>> We have a Java Webstart application that uses CORBA and requires an old
>> version of the Visibroker ORB (5.2.1) that will not launch with Java 9 due
>> to its inclusion of a change originally added to 7u55 and later backed
>> out:
>>         Bug ID: JDK-8042789 org.omg.CORBA.ORBSingletonClass loading no
>> longer uses context class loader
>> <http://bugs.java.com/view_bug.do?bug_id=8042789>
>>
>> What approaches should we consider for our application to support Java 9 ?
>>
>> The singleton ORB is the system-wide type code factory so having it be
> located via the TCCL is highly problematic (many reasons including
> security, memory leaks, and of course it's not going to work then there two
> or more applications/contexts in the same VM trying to do the same thing).
>
> I assume your application can use the 2-arg ORB.init to create the ORB, in
> which case the implementation will be located via the TCCL (so you can
> bundle the ORB implementation with the application). From your mail then it
> sounds like the issue is that something in Visibroker is using the zero-arg
> ORB.init and so gets the JDK ORB as the singleton ORB. Does this lead to an
> interop issue or does Visibroker assuming the singleton ORB is its own type?
>
> -Alan
>

From martinrb at google.com  Wed Oct 19 02:38:14 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 18 Oct 2016 19:38:14 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CAHjP37FRwd4kQXWT23=qzMcm3HeK52HmBHt=w0h5_8vFss-LAw@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
	<CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
	<CAHjP37Gqy4hTh-hso2rg9a1VrW2R+yRq9LnJCvwBxDv37qhS2g@mail.gmail.com>
	<CA+kOe0_MCB2UoW1YPaywzeVU55vMYf3zOrGhKLWxi7VTQOwP+Q@mail.gmail.com>
	<CAHjP37FRwd4kQXWT23=qzMcm3HeK52HmBHt=w0h5_8vFss-LAw@mail.gmail.com>
Message-ID: <CA+kOe0-6XtQP6-iO-xwmPhG5TTw4JUMn7YfW4-0V1rvQJR4O-g@mail.gmail.com>

On Tue, Oct 18, 2016 at 6:33 PM, Vitaly Davidovich <vitalyd at gmail.com>
wrote:

>
> Indeed - the whole inlining based on simple bytecode size is a big problem
> (I've brought this up in various contexts several times on the compiler
> list, but this is a known issue).  But, some of the methods using the jsr
> style look to already be >35 bytes.  I've seen this style in other places
> in the JDK where the method is definitely above 35.
>

- 35 is not a universal physical constant!
- maybe the same style is used in a similar method where it did cross the
35 byte threshold
- elsewhere in jsr166 we deal with concurrency, where copying fields into
locals is a good reflex for correctness

From srinivasan.raghavan at oracle.com  Wed Oct 19 04:08:37 2016
From: srinivasan.raghavan at oracle.com (Srinivasan Raghavan)
Date: Wed, 19 Oct 2016 09:38:37 +0530
Subject: RFR:JDK-8075205 java/net/URLClassLoader/closetest/CloseTest.java
	and GetResourceAsStream.java failed with existing dir
In-Reply-To: <019E6AAB-85A3-4D4E-A2A0-2DCBCF028EF3@oracle.com>
References: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>
	<019E6AAB-85A3-4D4E-A2A0-2DCBCF028EF3@oracle.com>
Message-ID: <6AE698AC-5301-46A3-92D2-4C6B64A3A2FB@oracle.com>

Thanks for the review.
> On 18-Oct-2016, at 4:16 PM, Chris Hegarty <chris.hegarty at oracle.com> wrote:
> 
> 
>> On 17 Oct 2016, at 09:51, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
>> 
>> Hi all
>> 
>> Please review the fix for the bug 
>> 
>> Bug :https://bugs.openjdk.java.net/browse/JDK-8075205
>> 
>> The tests uses classes directory for the output files. This can result in the files being left over after the test is complete which can result in instability. The tests copies the files to be compiled form test src to test classes which can result in copy of permission and result in instability because the test has delete operations. The test fails randomly mostly in copy or delete operation. I propose the test to be refactored to make the use scratch directory as its output directory and eliminate shell by using testlibrary utils.
>> 
>> fix : http://cr.openjdk.java.net/~sraghavan/8075205/webrev.00/
> 
> This looks good to me. Thanks Srinivasan.
> 
> -Chris.


From sundararajan.athijegannathan at oracle.com  Wed Oct 19 05:06:46 2016
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 19 Oct 2016 10:36:46 +0530
Subject: RFR 8071588: The spec for
	javax.script.ScriptEngineFactory.getProgram() should specify NPEs
	thrown
Message-ID: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8071588

jdk webrev: http://cr.openjdk.java.net/~sundar/8071588/jdk/webrev.00/

nashorn webrev:
http://cr.openjdk.java.net/~sundar/8071588/nashorn/webrev.00/

Thanks

-Sundar


From stanislav.smirnov at oracle.com  Wed Oct 19 06:54:10 2016
From: stanislav.smirnov at oracle.com (Stanislav Smirnov)
Date: Wed, 19 Oct 2016 09:54:10 +0300
Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under
	test/lib
In-Reply-To: <8d4ab144-30ac-faf6-ddc9-a975b14ddb5f@oracle.com>
References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com>
	<0404A092-57A8-4EF6-9423-885367B57FFD@oracle.com>
	<8d4ab144-30ac-faf6-ddc9-a975b14ddb5f@oracle.com>
Message-ID: <E91646A0-A816-4F04-B205-B96476878EF0@oracle.com>

David, thank you

Best regards,
Stanislav Smirnov





> On 19 Oct 2016, at 04:27, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Stanislav,
> 
> On 19/10/2016 1:06 AM, Stanislav Smirnov wrote:
>> Hi,
>> 
>> I'm still looking for volunteers to review
> 
> This is trivial. Reviewed. I will push for you, to jdk9/dev.
> 
> Thanks,
> David
> 
>> Best regards,
>> Stanislav Smirnov
>> 
>> 
>> 
>> 
>> 
>>> On 05 Oct 2016, at 19:44, Stanislav Smirnov <stanislav.smirnov at oracle.com> wrote:
>>> 
>>> Hi,
>>> 
>>> Please review this fix for JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>.
>>> This one is similar to another one, I have sent earlier.
>>> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements.
>>> 
>>> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 <https://bugs.openjdk.java.net/browse/JDK-8163984>
>>> Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ <http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/>
>>> Best regards,
>>> Stanislav Smirnov
>>> 
>>> 
>>> 
>>> 
>>> 
>> 


From hannes.wallnoefer at oracle.com  Wed Oct 19 07:07:40 2016
From: hannes.wallnoefer at oracle.com (=?utf-8?Q?Hannes_Walln=C3=B6fer?=)
Date: Wed, 19 Oct 2016 09:07:40 +0200
Subject: RFR 8071588: The spec for
	javax.script.ScriptEngineFactory.getProgram() should specify
	NPEs thrown
In-Reply-To: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>
References: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>
Message-ID: <8A0EF75C-ED19-46D7-8D89-A3A54AF2EF41@oracle.com>

+1

Hannes

> Am 19.10.2016 um 07:06 schrieb Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com>:
> 
> Please review.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8071588
> 
> jdk webrev: http://cr.openjdk.java.net/~sundar/8071588/jdk/webrev.00/
> 
> nashorn webrev:
> http://cr.openjdk.java.net/~sundar/8071588/nashorn/webrev.00/
> 
> Thanks
> 
> -Sundar
> 


From Alan.Bateman at oracle.com  Wed Oct 19 07:12:57 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 19 Oct 2016 08:12:57 +0100
Subject: RFR 8071588: The spec for
	javax.script.ScriptEngineFactory.getProgram() should specify NPEs
	thrown
In-Reply-To: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>
References: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>
Message-ID: <91d155ec-6018-6606-6941-85180cf477e7@oracle.com>

On 19/10/2016 06:06, Sundararajan Athijegannathan wrote:

> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8071588
>
> jdk webrev: http://cr.openjdk.java.net/~sundar/8071588/jdk/webrev.00/
>
> nashorn webrev:
> http://cr.openjdk.java.net/~sundar/8071588/nashorn/webrev.00/
>
This looks okay (except the spurious blank line after the class 
comment). It's a spec change that will need to be tracked.

-Alan

From Alan.Bateman at oracle.com  Wed Oct 19 07:48:15 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 19 Oct 2016 08:48:15 +0100
Subject: JDK 9 org.omg.CORBA.ORBSingletonClass loading will not use
	context class loader
In-Reply-To: <CABO0Qi5ycxGf46b6tiz8Cgi2QSwnmbRcTqa_uTWQtpxmFXh8xQ@mail.gmail.com>
References: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>
	<ad81cff8-f33e-a387-2f2d-be94d3d30008@oracle.com>
	<CABO0Qi5ycxGf46b6tiz8Cgi2QSwnmbRcTqa_uTWQtpxmFXh8xQ@mail.gmail.com>
Message-ID: <6a155e14-116d-edaa-46a5-b6cdbd13f218@oracle.com>

On 19/10/2016 03:28, Tom Hood wrote:

> It's unclear if there really is an interop issue.   The application 
> will launch with 7u55 if I don't set the ORBSingletonClass property, 
> although that isn't how visibroker 5.2.1 was intended to run, so not 
> setting it worries me.
>
> Our application does call ORB.init(args,props) once at startup and use 
> that instance throughout the code we have written.
>
> However, visibroker calls ORB.init() all over the idl generated code 
> and the vb jar itself.
>
> The javap of the vb jar does show some downcasts to its 
> com.inprise.vbroker.orb.ORBSingleton implementation in a few classes, 
> but I haven't tracked down if that code could execute in our 
> application or if the object it tries to downcast originated from 
> ORB.init().  The simple test of a basic launch and performing some 
> activities within the application seem to work.  Sadly we don't have 
> any automated tests that exercise our use of CORBA (java client 
> to/from c++ server).
>
> The idea of mixing ORB implementations worries me, even if only for 
> the singleton ORB.  I'm new to CORBA and the client side of our 
> application and still a newbie to java as I've been primarily working 
> on our c++ server.
>
> Do the reasons for the JDK change to ORB.init() apply in our use 
> case?  There's only one application/context and ORB vendor 
> implementation running in the jvm in production.  Do you think 
> approach #1 might be a reasonably safe workaround without the risk of 
> hidden interop issues?
>
I can't tell from this thread if you have tried with JDK 9 or not 
(you've mentioned 7u55 a few times).

I ask because in JDK 9 then the java.corba module is not resolved by 
default. If your application starts there then it suggests to me that 
Visibroker may be carrying its own copy of the org.omg.** (and RMI-IIOP) 
classes, in which case the JDK ORB (even singleton ORB) is not in the 
picture. Or maybe you've got past this and your JNLP contains:
   <j2se version="9+"" java-vm-args="--add-modules=java.corba">

which is the equivalent to the --add-modules command line option.

Another comment is that if there is code calling the no-arg ORB.init and 
blinding casting the return to com.inprise.vbroker.orb.ORBSingleton then 
it will never work in the scenario where the system-wide singleton ORB 
has been initialized by before Visibroker is used. I completely agree 
that it would be unusual to mix different ORB implementations in the 
same VM but the API has always allowed for that.

On approach #1 then one thing to be aware of is that the application 
class loader is not an instance of URLClassLoader in JDK 9. That is an 
implementation change that is known to trip up code that has been 
hacking into it.

-Alan

From sundararajan.athijegannathan at oracle.com  Wed Oct 19 07:55:22 2016
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 19 Oct 2016 13:25:22 +0530
Subject: RFR 8071588: The spec for
	javax.script.ScriptEngineFactory.getProgram()
	should specify NPEs thrown
In-Reply-To: <91d155ec-6018-6606-6941-85180cf477e7@oracle.com>
References: <12a836e8-89c6-642c-b199-061e0feb0ff4@oracle.com>
	<91d155ec-6018-6606-6941-85180cf477e7@oracle.com>
Message-ID: <580726EA.9070102@oracle.com>

Hi Alan,

Thanks for your review. Yes, spec. change is being tracked. I'll remove 
that whitespace after doc comment of getProgram method and push it.

PS. Updated webrev for the record: 
http://cr.openjdk.java.net/~sundar/8071588/jdk/webrev.01/

Thanks,
-Sundar

On 19/10/16, 12:42 PM, Alan Bateman wrote:
> On 19/10/2016 06:06, Sundararajan Athijegannathan wrote:
>
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8071588
>>
>> jdk webrev: http://cr.openjdk.java.net/~sundar/8071588/jdk/webrev.00/
>>
>> nashorn webrev:
>> http://cr.openjdk.java.net/~sundar/8071588/nashorn/webrev.00/
>>
> This looks okay (except the spurious blank line after the class 
> comment). It's a spec change that will need to be tracked.
>
> -Alan

From aleksej.efimov at oracle.com  Wed Oct 19 14:19:53 2016
From: aleksej.efimov at oracle.com (Aleks Efimov)
Date: Wed, 19 Oct 2016 17:19:53 +0300
Subject: RFR (JAXP): 8167179: Make XSL generated namespace prefixes local to
	transformation process
Message-ID: <9f5f3bcc-8614-b52e-573d-3d6388eb2e23@oracle.com>

Hi,
Please help to review the JDK9 fix for JDK-8167179 [1]:
     http://cr.openjdk.java.net/~aefimov/8167179/9/00/:
The fix changes how the xsl:element namespace prefix values are 
generated during the transformation process.

JDK9 build that includes this fix was tested with JDK XML related tests 
and with JCK-runtime/devtools suites. No failures were observed among 
these tests.

With Best Regards,
Aleksej

[1] https://bugs.openjdk.java.net/browse/JDK-8167179


From steve.drach at oracle.com  Wed Oct 19 18:34:57 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 19 Oct 2016 11:34:57 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned entry
	in base-versioned empty package
Message-ID: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>

Hi,

Please review the following changeset.  This fix allows jar tool to add a new public class in a versioned directory in a modular multi-release jar file if the class is in a concealed package. When this event takes place, a warning message is emitted saying that placing this jar on a class path will result in an incompatible public interface, the jar file does not meet the single api rule of a multi-release jar file.  As before, adding a new public class in a versioned directory of a multi-release jar file (not a modular jar file) will result in an error.

issue: https://bugs.openjdk.java.net/browse/JDK-8164805 <https://bugs.openjdk.java.net/browse/JDK-8164805>

webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/>

Thanks,
Steve



From stuart.marks at oracle.com  Wed Oct 19 19:19:18 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Wed, 19 Oct 2016 12:19:18 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
Message-ID: <4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>



On 10/18/16 11:00 AM, Martin Buchholz wrote:
> There's already a certain amount of mixing, e.g. stream tests sometimes test
> j.u.c. and Queue tests sometimes test all the Queue implementations. Unlike
> non-test code, some redundancy and divergence of testing frameworks and styles
> is fine.  I still haven't found a way of writing shared tests for
> implementations of an interface that I really like.

It's probably the case that divergence of testing frameworks is unavoidable, or 
at least it's impractical. I doubt that it's worthwhile to rewrite all the 
straight jtreg tests into TestNG, or JUnit, or whatever. But I'd draw the line 
before saying this is "fine." [1]

Maintaining the tests' organization is pretty important. Most of the collections 
tests are in jdk/test/java/util though they're spread around a bit uncomfortably 
even here. But now it's, oh, ArrayDeque and ArrayList.removeIf tests are over 
*here* instead. Having things spread around increases the likelihood of cases 
being missed or being tested redundantly.

The test groups have to be maintained as well. I know I've been bitten by 
problems (not in collections) where I *thought* I had run the right set of 
tests, but it ended up that I hadn't, resulting in me breaking the build. As it 
turns out, jdk_collections_core doesn't include any ArrayDeque tests. Hmmm. 
Well, maybe jdk_collections_core isn't useful. It would have been nice to know 
this when I added it last year. :-/ [2]

As things stand, I'm basically OK with this changeset going in. But it does seem 
to highlight some areas that need some future cleanup, particularly in the tests 
and the test group declarations.

s'marks

[1] http://knowyourmeme.com/memes/this-is-fine

[2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/7a0c06013ae6


From Roger.Riggs at Oracle.com  Wed Oct 19 19:59:23 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 19 Oct 2016 15:59:23 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
Message-ID: <71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>

The support in sun.reflect.ReflectionFactory for custom serialization, 
such as IIOP input
and output streams, is being expanded beyond the necessary constructor 
of a serializable
class to include access to the private methods readObject, writeObject, 
readResolve,
writeReplace, etc.

The IIOP implementation is updated to use a combination of 
ReflectionFactory and
Unsafe to serialize and deserialize objects and no longer rely on 
setAccessible.
Tests are included for ReflectionFactory and the affected IIOP classes.

Please review and comment,

jdk repo webrev:
http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
corba repo webrev :
http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/

Issue:
https://bugs.openjdk.java.net/browse/JDK-8164908

Thanks, Roger



From tom.w.hood at gmail.com  Wed Oct 19 21:13:42 2016
From: tom.w.hood at gmail.com (Tom Hood)
Date: Wed, 19 Oct 2016 14:13:42 -0700
Subject: JDK 9 org.omg.CORBA.ORBSingletonClass loading will not use
	context class loader
In-Reply-To: <6a155e14-116d-edaa-46a5-b6cdbd13f218@oracle.com>
References: <CABO0Qi7NO0ZXf7n4Tz2kxhhsPXqEotoh2FP1ZWP6=HMhZi8YZA@mail.gmail.com>
	<ad81cff8-f33e-a387-2f2d-be94d3d30008@oracle.com>
	<CABO0Qi5ycxGf46b6tiz8Cgi2QSwnmbRcTqa_uTWQtpxmFXh8xQ@mail.gmail.com>
	<6a155e14-116d-edaa-46a5-b6cdbd13f218@oracle.com>
Message-ID: <CABO0Qi774QHZrQmcehDvWUom9Wj_ScbDmEFZMZgZSqH8FDuSYA@mail.gmail.com>

I haven't tried JDK 9 yet.  I was assuming the 7u55 change in JDK 9 would
impact our application the same way.  I hope to try it next week as I'm
currently stuck testing on windows XP which limits me to 1.7 or earlier.
So maybe Approach #3 will be:  "It should just work"  [*I think I'm
required to add a quarter to some jar whenever I use that phrase*]    :-)

That's good  to know about the java.corba module.  The vb jar does carry a
bunch of org.omg.* classes, so it sounds like with JDK 9 everything corba
related will be taken from the vb jar (assuming I don't explicitly use
--add-modules).

The realization I'm now having is that JDK 9 will use more code from the vb
jar than earlier versions did which I think will be a good thing.  The
scary thought is that right now we must be running in production with a
Frankenstein mix of some org.omg.* classes from the vb jar and some from
the JDK which could in theory vary across JDK updates, making the whole
thing more fragile in my mind.  I believe the parent-first model is used by
the JNLPClassLoader will choose the JDK version of a class over one with
the same name from the vb jar.  So the mixing of different ORB
implementations I was so concerned about is probably already the way we're
running?  Yikes!

That's also good to know about JDK 9 application class loader not being an
instance of URLClassLoader.  I'll probably use approach #1 when jre is
older than 9 so our app will work with the select versions of 6, 7, and 8
with the ORB.init() change.

Any thoughts on question 2 of approach #1 (i.e. disabling security manager
vs. granting the application AllPermission via policy file)?  It seems like
they should be equivalent.  I'm told it is near impossible to reliably get
IT at customer facilities to make desired modifications to the jre
installation, such as a policy file.

Thanks for all your help.  Looking forward to trying JDK 9 soon.


On Wed, Oct 19, 2016 at 12:48 AM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> On 19/10/2016 03:28, Tom Hood wrote:
>
> It's unclear if there really is an interop issue.   The application will
> launch with 7u55 if I don't set the ORBSingletonClass property, although
> that isn't how visibroker 5.2.1 was intended to run, so not setting it
> worries me.
>
> Our application does call ORB.init(args,props) once at startup and use
> that instance throughout the code we have written.
>
> However, visibroker calls ORB.init() all over the idl generated code and
> the vb jar itself.
>
> The javap of the vb jar does show some downcasts to its
> com.inprise.vbroker.orb.ORBSingleton implementation in a few classes, but
> I haven't tracked down if that code could execute in our application or if
> the object it tries to downcast originated from ORB.init().  The simple
> test of a basic launch and performing some activities within the
> application seem to work.  Sadly we don't have any automated tests that
> exercise our use of CORBA (java client to/from c++ server).
>
> The idea of mixing ORB implementations worries me, even if only for the
> singleton ORB.  I'm new to CORBA and the client side of our application and
> still a newbie to java as I've been primarily working on our c++ server.
>
> Do the reasons for the JDK change to ORB.init() apply in our use case?
> There's only one application/context and ORB vendor implementation running
> in the jvm in production.  Do you think approach #1 might be a reasonably
> safe workaround without the risk of hidden interop issues?
>
> I can't tell from this thread if you have tried with JDK 9 or not (you've
> mentioned 7u55 a few times).
>
> I ask because in JDK 9 then the java.corba module is not resolved by
> default. If your application starts there then it suggests to me that
> Visibroker may be carrying its own copy of the org.omg.** (and RMI-IIOP)
> classes, in which case the JDK ORB (even singleton ORB) is not in the
> picture. Or maybe you've got past this and your JNLP contains:
>   <j2se version="9+"" java-vm-args="--add-modules=java.corba">
>
> which is the equivalent to the --add-modules command line option.
>
> Another comment is that if there is code calling the no-arg ORB.init and
> blinding casting the return to com.inprise.vbroker.orb.ORBSingleton then
> it will never work in the scenario where the system-wide singleton ORB has
> been initialized by before Visibroker is used. I completely agree that it
> would be unusual to mix different ORB implementations in the same VM but
> the API has always allowed for that.
>
> On approach #1 then one thing to be aware of is that the application class
> loader is not an instance of URLClassLoader in JDK 9. That is an
> implementation change that is known to trip up code that has been hacking
> into it.
>
> -Alan
>

From huizhe.wang at oracle.com  Wed Oct 19 21:56:24 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 19 Oct 2016 14:56:24 -0700
Subject: RFR (JAXP): 8167179: Make XSL generated namespace prefixes local
	to transformation process
In-Reply-To: <9f5f3bcc-8614-b52e-573d-3d6388eb2e23@oracle.com>
References: <9f5f3bcc-8614-b52e-573d-3d6388eb2e23@oracle.com>
Message-ID: <5807EC08.3050209@oracle.com>

Hi Aleksej,

The change looks good.

Thanks,
Joe

On 10/19/16, 7:19 AM, Aleks Efimov wrote:
> Hi,
> Please help to review the JDK9 fix for JDK-8167179 [1]:
>     http://cr.openjdk.java.net/~aefimov/8167179/9/00/:
> The fix changes how the xsl:element namespace prefix values are 
> generated during the transformation process.
>
> JDK9 build that includes this fix was tested with JDK XML related 
> tests and with JCK-runtime/devtools suites. No failures were observed 
> among these tests.
>
> With Best Regards,
> Aleksej
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8167179
>

From mandy.chung at oracle.com  Wed Oct 19 22:19:51 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 15:19:51 -0700
Subject: Review Request: JDK-8167057 jdeps to list the modules and internal
	APIs to help find @modules for tests
Message-ID: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>

Webrev at:
  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167057/webrev.00/

This patch enhances jdeps to print the dependences in the format : $MODULE[/$PACKAGE].

This is intended for analyzing the regression tests we develop and add make it easy to add the proper @modules.

Mandy

From martinrb at google.com  Wed Oct 19 22:21:08 2016
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 19 Oct 2016 15:21:08 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>
Message-ID: <CA+kOe08==314FRgC2HA15FC=oAENQ+Cz_qov=6jiRWjXwzXW2w@mail.gmail.com>

These tests are maintained outside of openjdk and have only recently been
made available within openjdk.

There may be a future where all the consumers of these tests are willing to
accept a good test framework.  Junit 5 dynamic tests look promising
http://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests
but it may be a decade before we can use it.

Probably code bases that include interfaces (e.g. Collection) should
provide infrastructure to test those interfaces for any implementation, but
openjdk does not try to do so.  CollectionTest is an experimental step in
that direction, but it looks we could do better with Junit 5 some day.  The
openjdk and jtreg projects can try to help by supporting junit 5 earlier
rather than later.

On Wed, Oct 19, 2016 at 12:19 PM, Stuart Marks <stuart.marks at oracle.com>
wrote:

>
>
> On 10/18/16 11:00 AM, Martin Buchholz wrote:
>
>> There's already a certain amount of mixing, e.g. stream tests sometimes
>> test
>> j.u.c. and Queue tests sometimes test all the Queue implementations.
>> Unlike
>> non-test code, some redundancy and divergence of testing frameworks and
>> styles
>> is fine.  I still haven't found a way of writing shared tests for
>> implementations of an interface that I really like.
>>
>
> It's probably the case that divergence of testing frameworks is
> unavoidable, or at least it's impractical. I doubt that it's worthwhile to
> rewrite all the straight jtreg tests into TestNG, or JUnit, or whatever.
> But I'd draw the line before saying this is "fine." [1]
>
> Maintaining the tests' organization is pretty important. Most of the
> collections tests are in jdk/test/java/util though they're spread around a
> bit uncomfortably even here. But now it's, oh, ArrayDeque and
> ArrayList.removeIf tests are over *here* instead. Having things spread
> around increases the likelihood of cases being missed or being tested
> redundantly.
>
> The test groups have to be maintained as well. I know I've been bitten by
> problems (not in collections) where I *thought* I had run the right set of
> tests, but it ended up that I hadn't, resulting in me breaking the build.
> As it turns out, jdk_collections_core doesn't include any ArrayDeque tests.
> Hmmm. Well, maybe jdk_collections_core isn't useful. It would have been
> nice to know this when I added it last year. :-/ [2]
>
> As things stand, I'm basically OK with this changeset going in. But it
> does seem to highlight some areas that need some future cleanup,
> particularly in the tests and the test group declarations.
>
> s'marks
>
> [1] http://knowyourmeme.com/memes/this-is-fine
>
> [2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/7a0c06013ae6
>
>

From paul.sandoz at oracle.com  Wed Oct 19 22:44:05 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Wed, 19 Oct 2016 15:44:05 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe09ftWWD7TG-i+4UGBa-M9Q=-U8Ay9kv7i_9G62JaL4GxQ@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
	<CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
	<159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>
	<CA+kOe09ftWWD7TG-i+4UGBa-M9Q=-U8Ay9kv7i_9G62JaL4GxQ@mail.gmail.com>
Message-ID: <3E410EE1-E803-43B8-91DA-8E8F2FD42D83@oracle.com>


> On 18 Oct 2016, at 13:58, Martin Buchholz <martinrb at google.com> wrote:
> 
> Latest webrev defers potential new methods:
>    /* TODO: public */ private void trimToSize()

Sorry to push back, i know it?s a hassle, but i think such features should be retained in a separate proposed patch.

Paul.

From david.dehaven at oracle.com  Wed Oct 19 22:50:50 2016
From: david.dehaven at oracle.com (David DeHaven)
Date: Wed, 19 Oct 2016 15:50:50 -0700
Subject: [9] RfR: 8165271: Fix use of reflection to gain access to private
	fields
In-Reply-To: <F45E69FA-DDE3-48BA-9EBC-BA753F9C1997@oracle.com>
References: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>
	<F45E69FA-DDE3-48BA-9EBC-BA753F9C1997@oracle.com>
Message-ID: <17F85508-FF33-487F-837A-82E65D73D6DD@oracle.com>


Please review the new patch version:
http://cr.openjdk.java.net/~ddehaven/8165271/jdk.1/

This was updated for the recent changes due to JDK-8168073. Note that the patch is against jdk9-client but I can apply and push to 9-dev. Test runs with both client and jake were successful.

-DrD-

> On Oct 13, 2016, at 5:24 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> 
>> On Oct 13, 2016, at 4:08 PM, David DeHaven <david.dehaven at oracle.com> wrote:
>> 
>> 
>> JBS Issue:
>> https://bugs.openjdk.java.net/browse/JDK-8165271
>> 
>> Webrev:
>> http://cr.openjdk.java.net/~ddehaven/8165271/jdk.0/
> 
> This change looks fine.  This hack should have been replaced long ago and happy to see them gone.
> 
> Mandy
> 


From alexandre.iline at oracle.com  Wed Oct 19 23:22:19 2016
From: alexandre.iline at oracle.com (Alexandre (Shura) Iline)
Date: Wed, 19 Oct 2016 16:22:19 -0700
Subject: Review Request: JDK-8167057 jdeps to list the modules and
	internal APIs to help find @modules for tests
In-Reply-To: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
References: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
Message-ID: <D532C374-380C-4346-ACE0-3D000B266C79@oracle.com>

Mandy,

I have a question around line 228 of src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java:
 224         new Option(false, "--list-deps", "--list-reduced-deps") {
 225             void process(JdepsTask task, String opt, String arg) {
 226                 task.options.showModulesAddExports = true;
 227                 task.options.reduced = opt.equals("--list-reduced-deps");
 228                 task.options.verbose = PACKAGE;
 229             }
 230         },

What is the expected behavior of jdeps if I use ??list-jdeps? at the same time as, say, ?-v?? 

Should there be more checks similar to these?
 483             if ((options.findJDKInternals) && (options.hasFilter() || options.showSummary)) {
 484                 showHelp();
 485                 return EXIT_CMDERR;
 486             }
 487             if (options.showSummary && options.verbose != SUMMARY) {
 488                 showHelp();
 489                 return EXIT_CMDERR;
 490             }

Looking on already existing options, with the current implementation, for ?-s? and -v? options:
?-v' ?-s? causes ?-v' to be ignored
?-s' ?-v? is not allowed.

If used with ?-jdkinternals', ?-s? is not allowed, while ?-v? is ignored.

Is this the intended behavior?

Shura


> On Oct 19, 2016, at 3:19 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Webrev at:
>  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167057/webrev.00/
> 
> This patch enhances jdeps to print the dependences in the format : $MODULE[/$PACKAGE].
> 
> This is intended for analyzing the regression tests we develop and add make it easy to add the proper @modules.
> 
> Mandy


From mandy.chung at oracle.com  Wed Oct 19 23:28:09 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 16:28:09 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
Message-ID: <CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>


> On Oct 19, 2016, at 11:34 AM, Steve Drach <steve.drach at oracle.com> wrote:
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8164805
> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ 

Issue a warning is good in the case public classes are added in a concealed package.  Some comments:

Main.java

addExtendedModuleAttributes does not seem to be the appropriate method to initialize concealedPackages.  It would be better to set concealedPackages in a separate method that will be called for each modular JAR.

Validator.java
 160                     main.error(Main.formatMsg("error.validator.concealed.public.class", entryName));

This is a warning.  You should add a new warn method to make it clear.
Similiarly,  ?error.validator.concealed.public.class? should be renamed to ?warn.validator.concealed.public.class?.  I notice there are several keys with ?error.validator? prefix that are meant for warnings.  It?d be good to  change them too.  

 247         if (idx > -1) {
 248             String pkgName = internalName.substring(0, idx).replace('/', '.');
 249             if (main.concealedPackages.contains(pkgName)) {
 250                 return true;
 251             }
 252         }

Alternative impl for the above lines:

String pkgName = idx != -1 ? internalName.substring(0, idx).replace('/', '.?)
			   : ??;
return main.concealedPackages.contains(pkgName);

jar.properties

112 error.validator.concealed.public.class=\
 113          warning - entry {0} is a public class in a concealed package, placing this \
 114          jar on the class path will result in incompatible public interfaces

line 113 needs new line ?\n?.  You may break it into 3 lines
since the entry name will take up some characters.

ConcealedPackage.java test

  60         Path destination = userdir.resolve("classes?);

I suggest to use Paths.get(?classes?) rather than ${user.dir}.  jtreg will clean up the JTwork/scratch directory after the test run.

  63         // add an empty directory
  64         Files.createDirectory(destination.resolve("p").resolve("internal"));

I suggest to take this out.  The test verifies if "jar tf mmr.jar? succeeds.

The test should test both ?cf? and ?uf? and verify that the ConcealedPackage attributes in module-info.class is updated correctly (one single module-info.class case in the root entry and two module-info.class - root and versioned).

  92     private int jar(String cmd) {
Nit: this can simply take a varargs and no need to split:
  jar(String? options)


  76     private String files(Path start) throws IOException {
Perhaps return Stream<String>.  That can replace the unnecessary concat in a String and then later split to pass to javac.

Mandy

From mandy.chung at oracle.com  Wed Oct 19 23:36:41 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 16:36:41 -0700
Subject: [9] RfR: 8165271: Fix use of reflection to gain access to private
	fields
In-Reply-To: <17F85508-FF33-487F-837A-82E65D73D6DD@oracle.com>
References: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>
	<F45E69FA-DDE3-48BA-9EBC-BA753F9C1997@oracle.com>
	<17F85508-FF33-487F-837A-82E65D73D6DD@oracle.com>
Message-ID: <B341071C-DA22-412F-93C6-E777A6BBFB02@oracle.com>


> On Oct 19, 2016, at 3:50 PM, David DeHaven <david.dehaven at oracle.com> wrote:
> 
> 
> Please review the new patch version:
> http://cr.openjdk.java.net/~ddehaven/8165271/jdk.1/
> 

Looks okay but I think the new interface should be named ?JavaNetURLClassLoaderAccess? (matching ?URLClassLoader")

> This was updated for the recent changes due to JDK-8168073. Note that the patch is against jdk9-client but I can apply and push to 9-dev. Test runs with both client and jake were successful.

Pushing to jdk9/dev would avoid any merge if any change to related area is made before the next jdk9/client integration to jdk9/dev.

Mandy

From jbluettduncan at gmail.com  Wed Oct 19 23:40:32 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Thu, 20 Oct 2016 00:40:32 +0100
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe08==314FRgC2HA15FC=oAENQ+Cz_qov=6jiRWjXwzXW2w@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>
	<CA+kOe08==314FRgC2HA15FC=oAENQ+Cz_qov=6jiRWjXwzXW2w@mail.gmail.com>
Message-ID: <CAPMys85-Ee3+n=bOaT5ae9KU6v1so-NAzAOCN4ojyp5yP=BRbg@mail.gmail.com>

Hi Martin,

By collections infrastructure, do you mean something like the collection
testers in guava-testlib?

If so, I agree that JUnit 5's dynamic tests look promising for implementing
such an infrastructure. I admit I don't have all the context here, but
would using guava-testlib in the meantime be a viable medium- or short-term
solution? Or would its dependence on JUnit 3/4 make switching impractical?
Or, perhaps, has development into CollectionTest gone so far that, for that
reason instead, it would be impractical until switch, at least until
something superior using e.g. JUnit 5's dynamic tests is made?

Kind regards,
Jonathan

On 19 October 2016 at 23:21, Martin Buchholz <martinrb at google.com> wrote:

> These tests are maintained outside of openjdk and have only recently been
> made available within openjdk.
>
> There may be a future where all the consumers of these tests are willing to
> accept a good test framework.  Junit 5 dynamic tests look promising
> http://junit.org/junit5/docs/current/user-guide/#writing-
> tests-dynamic-tests
> but it may be a decade before we can use it.
>
> Probably code bases that include interfaces (e.g. Collection) should
> provide infrastructure to test those interfaces for any implementation, but
> openjdk does not try to do so.  CollectionTest is an experimental step in
> that direction, but it looks we could do better with Junit 5 some day.  The
> openjdk and jtreg projects can try to help by supporting junit 5 earlier
> rather than later.
>
> On Wed, Oct 19, 2016 at 12:19 PM, Stuart Marks <stuart.marks at oracle.com>
> wrote:
>
> >
> >
> > On 10/18/16 11:00 AM, Martin Buchholz wrote:
> >
> >> There's already a certain amount of mixing, e.g. stream tests sometimes
> >> test
> >> j.u.c. and Queue tests sometimes test all the Queue implementations.
> >> Unlike
> >> non-test code, some redundancy and divergence of testing frameworks and
> >> styles
> >> is fine.  I still haven't found a way of writing shared tests for
> >> implementations of an interface that I really like.
> >>
> >
> > It's probably the case that divergence of testing frameworks is
> > unavoidable, or at least it's impractical. I doubt that it's worthwhile
> to
> > rewrite all the straight jtreg tests into TestNG, or JUnit, or whatever.
> > But I'd draw the line before saying this is "fine." [1]
> >
> > Maintaining the tests' organization is pretty important. Most of the
> > collections tests are in jdk/test/java/util though they're spread around
> a
> > bit uncomfortably even here. But now it's, oh, ArrayDeque and
> > ArrayList.removeIf tests are over *here* instead. Having things spread
> > around increases the likelihood of cases being missed or being tested
> > redundantly.
> >
> > The test groups have to be maintained as well. I know I've been bitten by
> > problems (not in collections) where I *thought* I had run the right set
> of
> > tests, but it ended up that I hadn't, resulting in me breaking the build.
> > As it turns out, jdk_collections_core doesn't include any ArrayDeque
> tests.
> > Hmmm. Well, maybe jdk_collections_core isn't useful. It would have been
> > nice to know this when I added it last year. :-/ [2]
> >
> > As things stand, I'm basically OK with this changeset going in. But it
> > does seem to highlight some areas that need some future cleanup,
> > particularly in the tests and the test group declarations.
> >
> > s'marks
> >
> > [1] http://knowyourmeme.com/memes/this-is-fine
> >
> > [2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/7a0c06013ae6
> >
> >
>

From steve.drach at oracle.com  Thu Oct 20 00:05:20 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 19 Oct 2016 17:05:20 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
Message-ID: <E2BFC585-C172-44B3-A902-212D58ED7D7C@oracle.com>

>> issue: https://bugs.openjdk.java.net/browse/JDK-8164805
>> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ 
> 
> Issue a warning is good in the case public classes are added in a concealed package.  Some comments:
> 
> Main.java
> 
> addExtendedModuleAttributes does not seem to be the appropriate method to initialize concealedPackages.  It would be better to set concealedPackages in a separate method that will be called for each modular JAR.

I made a simple change to existing code, I wasn?t intending to make significant changes to jar tool.  Of course as time goes on, jar tool continues to grow into a bigger hair ball.  It would definitely benefit from major cosmetic surgery.  Perhaps I don?t understand the point you are trying to make.

> 
> Validator.java
> 160                     main.error(Main.formatMsg("error.validator.concealed.public.class", entryName));
> 
> This is a warning.  You should add a new warn method to make it clear.

Ok.

> Similiarly,  ?error.validator.concealed.public.class? should be renamed to ?warn.validator.concealed.public.class?.  I notice there are several keys with ?error.validator? prefix that are meant for warnings.  It?d be good to  change them too.  
> 
> 247         if (idx > -1) {
> 248             String pkgName = internalName.substring(0, idx).replace('/', '.');
> 249             if (main.concealedPackages.contains(pkgName)) {
> 250                 return true;
> 251             }
> 252         }
> 
> Alternative impl for the above lines:
> 
> String pkgName = idx != -1 ? internalName.substring(0, idx).replace('/', '.?)
> 			   : ??;
> return main.concealedPackages.contains(pkgName);

Ok.

> 
> jar.properties
> 
> 112 error.validator.concealed.public.class=\
> 113          warning - entry {0} is a public class in a concealed package, placing this \
> 114          jar on the class path will result in incompatible public interfaces
> 
> line 113 needs new line ?\n?.  You may break it into 3 lines
> since the entry name will take up some characters.

Ok.

> 
> ConcealedPackage.java test
> 
>  60         Path destination = userdir.resolve("classes?);
> 
> I suggest to use Paths.get(?classes?) rather than ${user.dir}.

Is there a performance advantage or some other reason for doing this?  The way I do it seems reasonable.

>  jtreg will clean up the JTwork/scratch directory after the test run.

That?s what the docs say but I?ve seen problems in the past with windows machines, so I just got in the habit
of doing my own clean up.

> 
>  63         // add an empty directory
>  64         Files.createDirectory(destination.resolve("p").resolve("internal"));
> 
> I suggest to take this out.  The test verifies if "jar tf mmr.jar? succeeds.

Ok.  Just trying to make it exactly the same as the jar structure in the bug report.

> 
> The test should test both ?cf? and ?uf? and verify that the ConcealedPackage attributes in module-info.class is updated correctly (one single module-info.class case in the root entry and two module-info.class - root and versioned).

Ok.

> 
>  92     private int jar(String cmd) {
> Nit: this can simply take a varargs and no need to split:
>  jar(String? options)

I like the String because it?s more readable and I suspect the split isn?t that costly.

> 
> 
>  76     private String files(Path start) throws IOException {
> Perhaps return Stream<String>.  That can replace the unnecessary concat in a String and then later split to pass to javac.

What we probably want is stream.toArray(String[]::new), but then we?d have to copy that array into a new array that is 2 elements larger to accommodate the destination parameter.  I wonder if that?s any better.  I need to think about it.

> 
> Mandy


From steve.drach at oracle.com  Thu Oct 20 00:22:13 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 19 Oct 2016 17:22:13 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
Message-ID: <25D38911-2BA7-4F75-86E7-2160D7568800@oracle.com>

>  76     private String files(Path start) throws IOException {
> Perhaps return Stream<String>.  That can replace the unnecessary concat in a String and then later split to pass to javac.

Is this better?

    private void javac(Path source, Path destination) throws IOException {
        Stream<String> prefix = Stream.of("-d", destination.toString());
        Stream<String> suffix = Files.isDirectory(source)
                ? Files.walk(source)
                       .map(Path::toString)
                       .filter(s -> s.endsWith(".java"))
                : Stream.of(source.toString());
        String[] args = Stream.concat(prefix, suffix).toArray(String[]::new);
        JAVAC_TOOL.run(System.out, System.err, args);
    }

From mandy.chung at oracle.com  Thu Oct 20 00:25:04 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 17:25:04 -0700
Subject: Review Request: JDK-8167057 jdeps to list the modules and
	internal APIs to help find @modules for tests
In-Reply-To: <D532C374-380C-4346-ACE0-3D000B266C79@oracle.com>
References: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
	<D532C374-380C-4346-ACE0-3D000B266C79@oracle.com>
Message-ID: <A95CE1F8-AD50-469A-BB54-10EF1B6E7DFB@oracle.com>


> On Oct 19, 2016, at 4:22 PM, Alexandre (Shura) Iline <alexandre.iline at oracle.com> wrote:
> 
> Mandy,
> 
> I have a question around line 228 of src/jdk.jdeps/share/classes/com/sun/tools/jdeps/JdepsTask.java:
> 224         new Option(false, "--list-deps", "--list-reduced-deps") {
> 225             void process(JdepsTask task, String opt, String arg) {
> 226                 task.options.showModulesAddExports = true;
> 227                 task.options.reduced = opt.equals("--list-reduced-deps");
> 228                 task.options.verbose = PACKAGE;
> 229             }
> 230         },
> 
> What is the expected behavior of jdeps if I use ??list-jdeps? at the same time as, say, ?-v?? 
> 

-v (or other verbose options) specifies the granularity of analysis.  -?list-deps only does package-level analysis and so verbose option has essentially no effect.


> Should there be more checks similar to these?
> 483             if ((options.findJDKInternals) && (options.hasFilter() || options.showSummary)) {
> 484                 showHelp();
> 485                 return EXIT_CMDERR;
> 486             }
> 487             if (options.showSummary && options.verbose != SUMMARY) {
> 488                 showHelp();
> 489                 return EXIT_CMDERR;
> 490             }
> 
> Looking on already existing options, with the current implementation, for ?-s? and -v? options:
> ?-v' ?-s? causes ?-v' to be ignored

> ?-s' ?-v? is not allowed.
> 

-s and -v are conflicting options and both cases should not be allowed.  It?s a bug.

> If used with ?-jdkinternals', ?-s? is not allowed, while ?-v? is ignored.

-jdkinternals and -v and -s are conflicting options.  Can you file an issue for this bug?  I will need to take a pass on all options and should also consider subcommands.

Mandy

From mandy.chung at oracle.com  Thu Oct 20 00:32:20 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 17:32:20 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <E2BFC585-C172-44B3-A902-212D58ED7D7C@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<E2BFC585-C172-44B3-A902-212D58ED7D7C@oracle.com>
Message-ID: <4B7E318C-2907-48A7-8161-7DB24F5AFF28@oracle.com>


> On Oct 19, 2016, at 5:05 PM, Steve Drach <steve.drach at oracle.com> wrote:
> 
>>> issue: https://bugs.openjdk.java.net/browse/JDK-8164805
>>> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ 
>> 
>> Issue a warning is good in the case public classes are added in a concealed package.  Some comments:
>> 
>> Main.java
>> 
>> addExtendedModuleAttributes does not seem to be the appropriate method to initialize concealedPackages.  It would be better to set concealedPackages in a separate method that will be called for each modular JAR.
> 
> I made a simple change to existing code, I wasn?t intending to make significant changes to jar tool.  Of course as time goes on, jar tool continues to grow into a bigger hair ball.  It would definitely benefit from major cosmetic surgery.  Perhaps I don?t understand the point you are trying to make.

It made it hard for review and future maintainability.   It was not obvious to me when I reviewed the code whether this misses any case.

Refactoring it is a small change.

> 
>> 
>> ConcealedPackage.java test
>> 
>> 60         Path destination = userdir.resolve("classes?);
>> 
>> I suggest to use Paths.get(?classes?) rather than ${user.dir}.
> 
> Is there a performance advantage or some other reason for doing this?  The way I do it seems reasonable.

I just want to point out that jtreg will do the clean up if you use the scratch directory.

> 
>> jtreg will clean up the JTwork/scratch directory after the test run.
> 
> That?s what the docs say but I?ve seen problems in the past with windows machines, so I just got in the habit
> of doing my own clean up.
> 

Up to you.

>> 
>> 63         // add an empty directory
>> 64         Files.createDirectory(destination.resolve("p").resolve("internal"));
>> 
>> I suggest to take this out.  The test verifies if "jar tf mmr.jar? succeeds.
> 
> Ok.  Just trying to make it exactly the same as the jar structure in the bug report.

I updated the JBS issue to clarify that per the recent discussion.

> 
>> 
>> 92     private int jar(String cmd) {
>> Nit: this can simply take a varargs and no need to split:
>> jar(String? options)
> 
> I like the String because it?s more readable and I suspect the split isn?t that costly.
> 

I just point out that it?s a kind of silly to concat all args and then split it. 

Mandy


From mandy.chung at oracle.com  Thu Oct 20 00:33:31 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 19 Oct 2016 17:33:31 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <25D38911-2BA7-4F75-86E7-2160D7568800@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<25D38911-2BA7-4F75-86E7-2160D7568800@oracle.com>
Message-ID: <327802F6-8E93-4E9E-A173-6284A1506CFD@oracle.com>


> On Oct 19, 2016, at 5:22 PM, Steve Drach <steve.drach at oracle.com> wrote:
> 
>> 76     private String files(Path start) throws IOException {
>> Perhaps return Stream<String>.  That can replace the unnecessary concat in a String and then later split to pass to javac.
> 
> Is this better?
> 
>    private void javac(Path source, Path destination) throws IOException {
>        Stream<String> prefix = Stream.of("-d", destination.toString());
>        Stream<String> suffix = Files.isDirectory(source)
>                ? Files.walk(source)
>                       .map(Path::toString)
>                       .filter(s -> s.endsWith(".java"))
>                : Stream.of(source.toString());
>        String[] args = Stream.concat(prefix, suffix).toArray(String[]::new);
>        JAVAC_TOOL.run(System.out, System.err, args);
>    }


This is okay.

Mandy

From steve.drach at oracle.com  Thu Oct 20 00:40:09 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Wed, 19 Oct 2016 17:40:09 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <4B7E318C-2907-48A7-8161-7DB24F5AFF28@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<E2BFC585-C172-44B3-A902-212D58ED7D7C@oracle.com>
	<4B7E318C-2907-48A7-8161-7DB24F5AFF28@oracle.com>
Message-ID: <22BDE2DF-D71A-4A06-84D1-CF3870C27870@oracle.com>

>> On Oct 19, 2016, at 5:05 PM, Steve Drach <steve.drach at oracle.com> wrote:
>> 
>>>> issue: https://bugs.openjdk.java.net/browse/JDK-8164805
>>>> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ 
>>> 
>>> Issue a warning is good in the case public classes are added in a concealed package.  Some comments:
>>> 
>>> Main.java
>>> 
>>> addExtendedModuleAttributes does not seem to be the appropriate method to initialize concealedPackages.  It would be better to set concealedPackages in a separate method that will be called for each modular JAR.
>> 
>> I made a simple change to existing code, I wasn?t intending to make significant changes to jar tool.  Of course as time goes on, jar tool continues to grow into a bigger hair ball.  It would definitely benefit from major cosmetic surgery.  Perhaps I don?t understand the point you are trying to make.
> 
> It made it hard for review and future maintainability.   It was not obvious to me when I reviewed the code whether this misses any case.
> 
> Refactoring it is a small change.

Just to be clear, you?d like me to take lines 1988-1995 and put them in a separate method that gets called by addExtendedModuleAttributes?


> 
>> 
>>> 
>>> ConcealedPackage.java test
>>> 
>>> 60         Path destination = userdir.resolve("classes?);
>>> 
>>> I suggest to use Paths.get(?classes?) rather than ${user.dir}.
>> 
>> Is there a performance advantage or some other reason for doing this?  The way I do it seems reasonable.
> 
> I just want to point out that jtreg will do the clean up if you use the scratch directory.
> 
>> 
>>> jtreg will clean up the JTwork/scratch directory after the test run.
>> 
>> That?s what the docs say but I?ve seen problems in the past with windows machines, so I just got in the habit
>> of doing my own clean up.
>> 
> 
> Up to you.
> 
>>> 
>>> 63         // add an empty directory
>>> 64         Files.createDirectory(destination.resolve("p").resolve("internal"));
>>> 
>>> I suggest to take this out.  The test verifies if "jar tf mmr.jar? succeeds.
>> 
>> Ok.  Just trying to make it exactly the same as the jar structure in the bug report.
> 
> I updated the JBS issue to clarify that per the recent discussion.
> 
>> 
>>> 
>>> 92     private int jar(String cmd) {
>>> Nit: this can simply take a varargs and no need to split:
>>> jar(String? options)
>> 
>> I like the String because it?s more readable and I suspect the split isn?t that costly.
>> 
> 
> I just point out that it?s a kind of silly to concat all args and then split it. 
> 
> Mandy
> 


From martinrb at google.com  Thu Oct 20 02:33:59 2016
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 19 Oct 2016 19:33:59 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <3E410EE1-E803-43B8-91DA-8E8F2FD42D83@oracle.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
	<CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
	<159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>
	<CA+kOe09ftWWD7TG-i+4UGBa-M9Q=-U8Ay9kv7i_9G62JaL4GxQ@mail.gmail.com>
	<3E410EE1-E803-43B8-91DA-8E8F2FD42D83@oracle.com>
Message-ID: <CA+kOe09pxP5SYywp0NnLsLVj9a4QzDeA4GSqOAMFN5DWsn6v-Q@mail.gmail.com>

On Wed, Oct 19, 2016 at 3:44 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:

>
> On 18 Oct 2016, at 13:58, Martin Buchholz <martinrb at google.com> wrote:
>
> Latest webrev defers potential new methods:
>
>    /* TODO: public */ private void trimToSize()
>
> Sorry to push back, i know it?s a hassle, but i think such features should
> be retained in a separate proposed patch.
>

OK.  These methods can still be found in jsr166 CVS, but are now filtered
out during upstreaming.

From weijun.wang at oracle.com  Thu Oct 20 02:47:31 2016
From: weijun.wang at oracle.com (Wang Weijun)
Date: Thu, 20 Oct 2016 10:47:31 +0800
Subject: RFR 8168127: FilePermissionCollection merges incorrectly
Message-ID: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>

Please review the code change at

   http://cr.openjdk.java.net/~weijun/8168127/webrev.00/

Two changes:

1. npath2 is considered in equals and hashCode of FilePermission, so 2 objects with different npath2 can be added to a map and different entries.

2. special name for newPermUsingAltPath and newPermPlusAltPath results, so FilePermissionCollection::add will not merge one with the original.

Thanks
Max


From martinrb at google.com  Thu Oct 20 03:45:45 2016
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 19 Oct 2016 20:45:45 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CAPMys85-Ee3+n=bOaT5ae9KU6v1so-NAzAOCN4ojyp5yP=BRbg@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>
	<CA+kOe08==314FRgC2HA15FC=oAENQ+Cz_qov=6jiRWjXwzXW2w@mail.gmail.com>
	<CAPMys85-Ee3+n=bOaT5ae9KU6v1so-NAzAOCN4ojyp5yP=BRbg@mail.gmail.com>
Message-ID: <CA+kOe08V=0Fxj_T_f_jQ_yWyvZummqCmOAJ_7UEZBxY75vaS_w@mail.gmail.com>

On Wed, Oct 19, 2016 at 4:40 PM, Jonathan Bluett-Duncan <
jbluettduncan at gmail.com> wrote:

> Hi Martin,
>
> By collections infrastructure, do you mean something like the collection
> testers in guava-testlib?
>
> If so, I agree that JUnit 5's dynamic tests look promising for
> implementing such an infrastructure. I admit I don't have all the context
> here, but would using guava-testlib in the meantime be a viable medium- or
> short-term solution? Or would its dependence on JUnit 3/4 make switching
> impractical? Or, perhaps, has development into CollectionTest gone so far
> that, for that reason instead, it would be impractical until switch, at
> least until something superior using e.g. JUnit 5's dynamic tests is made?
>

I'm embarrassed to say I'm not familiar enough with guava's testlib.  Guava
does have generic collection oriented tests, and even runs them on jdk
classes. (Someone on the jdk quality team should be running the guava tests
using development jdks!).  I'm not familiar enough with the tools to work
on this myself, but I encourage someone who is to do that.

I see that guava testlib does have:

TestsForQueuesInJavaUtil.java
TestsForSetsInJavaUtil.java
TestsForListsInJavaUtil.java
TestsForMapsInJavaUtil.java

and that the infrastructure there is vaguely similar to what I ended up
doing.  JDK version skew is a problem, because openjdk development is
focused on jdk9, while guava is still trying to digest jdk8.

From rachna.goel at oracle.com  Thu Oct 20 08:27:32 2016
From: rachna.goel at oracle.com (Rachna Goel)
Date: Thu, 20 Oct 2016 13:57:32 +0530
Subject: RFR: JDK-8146750:java.time.Month.getDisplayName() return incorrect
	narrow names with JRE provider on locale de,de_DE,en_US.
Message-ID: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>

Hi,

Please review fix for JDK-8146750.

Bug : https://bugs.openjdk.java.net/browse/JDK-8146750

webrev : http://cr.openjdk.java.net/~rgoel/JDK-8146750/webrev.09/

Fix is to retrieve Narrow and Narrow_Standalone Month names and Day 
names one by one, as they can be duplicate.

Thanks,

Rachna

From weijun.wang at oracle.com  Thu Oct 20 08:51:10 2016
From: weijun.wang at oracle.com (Wang Weijun)
Date: Thu, 20 Oct 2016 16:51:10 +0800
Subject: RFR 8167646: Better invalid FilePermission
In-Reply-To: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
References: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
Message-ID: <F29526E9-C44F-42FC-AE11-F2246BB0EA45@oracle.com>

Please review the code change at

   http://cr.openjdk.java.net/~weijun/8167646/webrev.00/

A new flag invalid is added so invalid FilePermissions (invalid Path) do not equal or imply or is implied by anything else except for itself.

Thanks
Max


From masayoshi.okutsu at oracle.com  Thu Oct 20 09:06:01 2016
From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu)
Date: Thu, 20 Oct 2016 18:06:01 +0900
Subject: RFR: JDK-8146750:java.time.Month.getDisplayName() return
	incorrect narrow names with JRE provider on locale de,de_DE,en_US.
In-Reply-To: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>
References: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>
Message-ID: <44483b62-d6f9-1334-047a-071d7295bc1e@oracle.com>

Looks good to me.

Masayoshi


On 10/20/2016 5:27 PM, Rachna Goel wrote:
> Hi,
>
> Please review fix for JDK-8146750.
>
> Bug : https://bugs.openjdk.java.net/browse/JDK-8146750
>
> webrev : http://cr.openjdk.java.net/~rgoel/JDK-8146750/webrev.09/
>
> Fix is to retrieve Narrow and Narrow_Standalone Month names and Day 
> names one by one, as they can be duplicate.
>
> Thanks,
>
> Rachna


From Alan.Bateman at oracle.com  Thu Oct 20 12:21:01 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 20 Oct 2016 13:21:01 +0100
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
Message-ID: <1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>

On 19/10/2016 20:59, Roger Riggs wrote:

> The support in sun.reflect.ReflectionFactory for custom serialization, 
> such as IIOP input
> and output streams, is being expanded beyond the necessary constructor 
> of a serializable
> class to include access to the private methods readObject, 
> writeObject, readResolve,
> writeReplace, etc.
>
> The IIOP implementation is updated to use a combination of 
> ReflectionFactory and
> Unsafe to serialize and deserialize objects and no longer rely on 
> setAccessible.
> Tests are included for ReflectionFactory and the affected IIOP classes.
>
> Please review and comment,
>
> jdk repo webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
> corba repo webrev :
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>
>
I skimmed through the changes.

I assume findReadWriteObjectForSerialization should throw InternalError, 
rather than return null, if IllegalAccessException is thrown (as IAE is 
not possible here).

You've added @since 9 to sun.reflect.ReflectionFactory but that class is 
not new.

The javadoc for 
sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't say 
that it returns null when the Class is not Serializable.

For the MH returning methods then I assume the javadoc should say that 
it returns a direct method handle.

The synchronization in the IIOP ObjectStreamClass isn't very clear. Are 
the invoke*, read*, write* methods all invoked by the same thread that 
creates the ObjectStreamClass with the lookup method?

-Alan

From jbluettduncan at gmail.com  Thu Oct 20 13:04:41 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Thu, 20 Oct 2016 14:04:41 +0100
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe08V=0Fxj_T_f_jQ_yWyvZummqCmOAJ_7UEZBxY75vaS_w@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<4445357d-0546-adab-9d81-1c6d606b7e71@oracle.com>
	<CA+kOe08==314FRgC2HA15FC=oAENQ+Cz_qov=6jiRWjXwzXW2w@mail.gmail.com>
	<CAPMys85-Ee3+n=bOaT5ae9KU6v1so-NAzAOCN4ojyp5yP=BRbg@mail.gmail.com>
	<CA+kOe08V=0Fxj_T_f_jQ_yWyvZummqCmOAJ_7UEZBxY75vaS_w@mail.gmail.com>
Message-ID: <CAPMys86bySCPewbGsX3-H1=zdBT-m2SqLOb4Nn0URqxf=JUQ3Q@mail.gmail.com>

Ah, I see. Thanks for answering my questions Martin.

Sounds like there's no real benefit then to using guava-testlib in this
changeset instead of CollectionTest. And even if the benefit of using it
was considered to be high enough in the future, it would probably need its
own JSR, since I presume it would be applicable to a number of places in
the jdk, making it a humongous change.

Overall, it seems sensible to wait until e.g. JUnit 5 develops a worthwhile
solution to the dynamic test generation problem, at which point we could
then consider it for writing exhaustive tests for the jdk collections.

Thanks again,
Jonathan


On 20 October 2016 at 04:45, Martin Buchholz <martinrb at google.com> wrote:

>
>
> On Wed, Oct 19, 2016 at 4:40 PM, Jonathan Bluett-Duncan <
> jbluettduncan at gmail.com> wrote:
>
>> Hi Martin,
>>
>> By collections infrastructure, do you mean something like the collection
>> testers in guava-testlib?
>>
>> If so, I agree that JUnit 5's dynamic tests look promising for
>> implementing such an infrastructure. I admit I don't have all the context
>> here, but would using guava-testlib in the meantime be a viable medium- or
>> short-term solution? Or would its dependence on JUnit 3/4 make switching
>> impractical? Or, perhaps, has development into CollectionTest gone so far
>> that, for that reason instead, it would be impractical until switch, at
>> least until something superior using e.g. JUnit 5's dynamic tests is made?
>>
>
> I'm embarrassed to say I'm not familiar enough with guava's testlib.
> Guava does have generic collection oriented tests, and even runs them on
> jdk classes. (Someone on the jdk quality team should be running the guava
> tests using development jdks!).  I'm not familiar enough with the tools to
> work on this myself, but I encourage someone who is to do that.
>
> I see that guava testlib does have:
>
> TestsForQueuesInJavaUtil.java
> TestsForSetsInJavaUtil.java
> TestsForListsInJavaUtil.java
> TestsForMapsInJavaUtil.java
>
> and that the infrastructure there is vaguely similar to what I ended up
> doing.  JDK version skew is a problem, because openjdk development is
> focused on jdk9, while guava is still trying to digest jdk8.
>

From david.lloyd at redhat.com  Thu Oct 20 13:12:39 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 20 Oct 2016 08:12:39 -0500
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
Message-ID: <ebd951fa-a9bb-e797-e5be-4706df17aa7a@redhat.com>

On 10/19/2016 02:59 PM, Roger Riggs wrote:
> The support in sun.reflect.ReflectionFactory for custom serialization,
> such as IIOP input
> and output streams, is being expanded beyond the necessary constructor
> of a serializable
> class to include access to the private methods readObject, writeObject,
> readResolve,
> writeReplace, etc.
>
> The IIOP implementation is updated to use a combination of
> ReflectionFactory and
> Unsafe to serialize and deserialize objects and no longer rely on
> setAccessible.
> Tests are included for ReflectionFactory and the affected IIOP classes.
>
> Please review and comment,
>
> jdk repo webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
> corba repo webrev :
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/
>
> Issue:
> https://bugs.openjdk.java.net/browse/JDK-8164908

For the custom serialization case, it is also necessary to access the 
(usually private) readObjectNoData method (if any).

-- 
- DML

From Roger.Riggs at Oracle.com  Thu Oct 20 14:04:28 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 20 Oct 2016 10:04:28 -0400
Subject: RFR: JDK-8146750:java.time.Month.getDisplayName() return
	incorrect narrow names with JRE provider on locale de,de_DE,en_US.
In-Reply-To: <44483b62-d6f9-1334-047a-071d7295bc1e@oracle.com>
References: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>
	<44483b62-d6f9-1334-047a-071d7295bc1e@oracle.com>
Message-ID: <a7dd8146-f648-5d0b-6c46-e9d1f66f7876@Oracle.com>

+1

Looks fine

On 10/20/2016 5:06 AM, Masayoshi Okutsu wrote:
> Looks good to me.
>
> Masayoshi
>
>
> On 10/20/2016 5:27 PM, Rachna Goel wrote:
>> Hi,
>>
>> Please review fix for JDK-8146750.
>>
>> Bug : https://bugs.openjdk.java.net/browse/JDK-8146750
>>
>> webrev : http://cr.openjdk.java.net/~rgoel/JDK-8146750/webrev.09/
>>
>> Fix is to retrieve Narrow and Narrow_Standalone Month names and Day 
>> names one by one, as they can be duplicate.
>>
>> Thanks,
>>
>> Rachna
>


From david.dehaven at oracle.com  Thu Oct 20 14:45:16 2016
From: david.dehaven at oracle.com (David DeHaven)
Date: Thu, 20 Oct 2016 07:45:16 -0700
Subject: [9] RfR: 8165271: Fix use of reflection to gain access to private
	fields
In-Reply-To: <B341071C-DA22-412F-93C6-E777A6BBFB02@oracle.com>
References: <08993D7A-D8BB-4191-85F2-D17083FE64EC@oracle.com>
	<F45E69FA-DDE3-48BA-9EBC-BA753F9C1997@oracle.com>
	<17F85508-FF33-487F-837A-82E65D73D6DD@oracle.com>
	<B341071C-DA22-412F-93C6-E777A6BBFB02@oracle.com>
Message-ID: <3DBBA0B9-330B-4177-90D5-BF0F14074401@oracle.com>


>> Please review the new patch version:
>> http://cr.openjdk.java.net/~ddehaven/8165271/jdk.1/
>> 
> 
> Looks okay but I think the new interface should be named ?JavaNetURLClassLoaderAccess? (matching ?URLClassLoader")

I originally had that but changed it to look more consistent with the other JavaXXAccess interfaces... I can change it back before pushing.

-DrD-


From amy.lu at oracle.com  Thu Oct 20 15:32:42 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Thu, 20 Oct 2016 23:32:42 +0800
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
Message-ID: <4512d137-2616-45e5-4a98-05981cda8952@oracle.com>

On 10/20/16 3:59 AM, Roger Riggs wrote:
> The support in sun.reflect.ReflectionFactory for custom serialization, 
> such as IIOP input
> and output streams, is being expanded beyond the necessary constructor 
> of a serializable
> class to include access to the private methods readObject, 
> writeObject, readResolve,
> writeReplace, etc.
>
> The IIOP implementation is updated to use a combination of 
> ReflectionFactory and
> Unsafe to serialize and deserialize objects and no longer rely on 
> setAccessible.
> Tests are included for ReflectionFactory and the affected IIOP classes.
>
> Please review and comment,
>
> jdk repo webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
> corba repo webrev :
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>
>
> Issue:
> https://bugs.openjdk.java.net/browse/JDK-8164908
>
> Thanks, Roger
>
>
test/com/sun/corba/serialization/ObjectStreamTest.java

371                     startOrbd_orig();
Should test wait for several seconds after this to make sure it started?

(I'm not an official reviewer.)

Thanks,
Amy

From amy.lu at oracle.com  Thu Oct 20 15:54:05 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Thu, 20 Oct 2016 23:54:05 +0800
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <4512d137-2616-45e5-4a98-05981cda8952@oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<4512d137-2616-45e5-4a98-05981cda8952@oracle.com>
Message-ID: <05bf5a5b-bfcb-1da6-6193-4ae6aeae0e10@oracle.com>

On 10/20/16 11:32 PM, Amy Lu wrote:
> On 10/20/16 3:59 AM, Roger Riggs wrote:
>> The support in sun.reflect.ReflectionFactory for custom 
>> serialization, such as IIOP input
>> and output streams, is being expanded beyond the necessary 
>> constructor of a serializable
>> class to include access to the private methods readObject, 
>> writeObject, readResolve,
>> writeReplace, etc.
>>
>> The IIOP implementation is updated to use a combination of 
>> ReflectionFactory and
>> Unsafe to serialize and deserialize objects and no longer rely on 
>> setAccessible.
>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>
>> Please review and comment,
>>
>> jdk repo webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>> corba repo webrev :
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>
>>
>> Issue:
>> https://bugs.openjdk.java.net/browse/JDK-8164908
>>
>> Thanks, Roger
>>
>>
> test/com/sun/corba/serialization/ObjectStreamTest.java
>
> 371                     startOrbd_orig();
> Should test wait for several seconds after this to make sure it started?

413             orbdProcess.destroy();
414             if (!orbdProcess.waitFor(10, TimeUnit.SECONDS)) {

And to try best to cleanup process and release port, how about use:
             orbdProcess.destroyForcibly();
             orbdProcess.waitFor();

Thanks,
Amy
>
> (I'm not an official reviewer.)
>
> Thanks,
> Amy


From naoto.sato at oracle.com  Thu Oct 20 16:02:36 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Thu, 20 Oct 2016 09:02:36 -0700
Subject: RFR: JDK-8146750:java.time.Month.getDisplayName() return
	incorrect narrow names with JRE provider on locale de,de_DE,en_US.
In-Reply-To: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>
References: <ee3511ad-3030-e6b7-b260-b2a8cf93a969@oracle.com>
Message-ID: <b35c66b2-3a9b-be96-0ab5-d94d227914a0@oracle.com>

Looks good, Rachna.

Naoto

On 10/20/16 1:27 AM, Rachna Goel wrote:
> Hi,
>
> Please review fix for JDK-8146750.
>
> Bug : https://bugs.openjdk.java.net/browse/JDK-8146750
>
> webrev : http://cr.openjdk.java.net/~rgoel/JDK-8146750/webrev.09/
>
> Fix is to retrieve Narrow and Narrow_Standalone Month names and Day
> names one by one, as they can be duplicate.
>
> Thanks,
>
> Rachna

From paul.sandoz at oracle.com  Thu Oct 20 16:51:22 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Thu, 20 Oct 2016 09:51:22 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe09pxP5SYywp0NnLsLVj9a4QzDeA4GSqOAMFN5DWsn6v-Q@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<88A5F0D6-26E4-4956-BA4F-D963BEBB6882@oracle.com>
	<CA+kOe0-GOWegSnQ_MEiEPfqhH7pDFpGVR9JV+p3HMKODJnyWhw@mail.gmail.com>
	<159CBFB4-C090-418A-BAFA-3D0378453383@oracle.com>
	<CA+kOe09ftWWD7TG-i+4UGBa-M9Q=-U8Ay9kv7i_9G62JaL4GxQ@mail.gmail.com>
	<3E410EE1-E803-43B8-91DA-8E8F2FD42D83@oracle.com>
	<CA+kOe09pxP5SYywp0NnLsLVj9a4QzDeA4GSqOAMFN5DWsn6v-Q@mail.gmail.com>
Message-ID: <CA0DAA89-158A-41BB-A70D-90AB830632DF@oracle.com>


> On 19 Oct 2016, at 19:33, Martin Buchholz <martinrb at google.com> wrote:
> 
> 
> 
> On Wed, Oct 19, 2016 at 3:44 PM, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote:
> 
>> On 18 Oct 2016, at 13:58, Martin Buchholz <martinrb at google.com <mailto:martinrb at google.com>> wrote:
>> 
>> Latest webrev defers potential new methods:
>>    /* TODO: public */ private void trimToSize()
> 
> Sorry to push back, i know it?s a hassle, but i think such features should be retained in a separate proposed patch.
> 
> OK.  These methods can still be found in jsr166 CVS, but are now filtered out during upstreaming.


Thanks for being flexible. Looks good to me (i did not look in detail at the impl improvements on ArrayDeque, i think Stuart has that one amply covered).

Paul.


From Roger.Riggs at Oracle.com  Thu Oct 20 17:06:02 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 20 Oct 2016 13:06:02 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <ebd951fa-a9bb-e797-e5be-4706df17aa7a@redhat.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<ebd951fa-a9bb-e797-e5be-4706df17aa7a@redhat.com>
Message-ID: <54c0d24d-6e57-786a-95af-1d261addf0d7@Oracle.com>

Hi David,

Thanks for the reminder, I'll include readObjectNoData in the next update.

(fyi, readObject, readObjectNoData, and writeObject must be private 
methods.)

Roger


On 10/20/2016 9:12 AM, David M. Lloyd wrote:
> On 10/19/2016 02:59 PM, Roger Riggs wrote:
>> The support in sun.reflect.ReflectionFactory for custom serialization,
>> such as IIOP input
>> and output streams, is being expanded beyond the necessary constructor
>> of a serializable
>> class to include access to the private methods readObject, writeObject,
>> readResolve,
>> writeReplace, etc.
>>
>> The IIOP implementation is updated to use a combination of
>> ReflectionFactory and
>> Unsafe to serialize and deserialize objects and no longer rely on
>> setAccessible.
>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>
>> Please review and comment,
>>
>> jdk repo webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>> corba repo webrev :
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>
>>
>> Issue:
>> https://bugs.openjdk.java.net/browse/JDK-8164908
>
> For the custom serialization case, it is also necessary to access the 
> (usually private) readObjectNoData method (if any).
>


From Roger.Riggs at Oracle.com  Thu Oct 20 17:07:45 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 20 Oct 2016 13:07:45 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <05bf5a5b-bfcb-1da6-6193-4ae6aeae0e10@oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<4512d137-2616-45e5-4a98-05981cda8952@oracle.com>
	<05bf5a5b-bfcb-1da6-6193-4ae6aeae0e10@oracle.com>
Message-ID: <82e1cb08-52ca-55e2-ba2c-5b8afd79a371@Oracle.com>

Hi Amy,

Thanks for the suggestions.  I will fix them in the next update.
I've had some issues with port in use errors and am looking for best 
practices
in starting the servers and their cleanup.

Roger


On 10/20/2016 11:54 AM, Amy Lu wrote:
> On 10/20/16 11:32 PM, Amy Lu wrote:
>> On 10/20/16 3:59 AM, Roger Riggs wrote:
>>> The support in sun.reflect.ReflectionFactory for custom 
>>> serialization, such as IIOP input
>>> and output streams, is being expanded beyond the necessary 
>>> constructor of a serializable
>>> class to include access to the private methods readObject, 
>>> writeObject, readResolve,
>>> writeReplace, etc.
>>>
>>> The IIOP implementation is updated to use a combination of 
>>> ReflectionFactory and
>>> Unsafe to serialize and deserialize objects and no longer rely on 
>>> setAccessible.
>>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>>
>>> Please review and comment,
>>>
>>> jdk repo webrev:
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>> corba repo webrev :
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>>
>>>
>>> Issue:
>>> https://bugs.openjdk.java.net/browse/JDK-8164908
>>>
>>> Thanks, Roger
>>>
>>>
>> test/com/sun/corba/serialization/ObjectStreamTest.java
>>
>> 371                     startOrbd_orig();
>> Should test wait for several seconds after this to make sure it started?
>
> 413             orbdProcess.destroy();
> 414             if (!orbdProcess.waitFor(10, TimeUnit.SECONDS)) {
>
> And to try best to cleanup process and release port, how about use:
>             orbdProcess.destroyForcibly();
>             orbdProcess.waitFor();
>
> Thanks,
> Amy
>>
>> (I'm not an official reviewer.)
>>
>> Thanks,
>> Amy
>


From Roger.Riggs at Oracle.com  Thu Oct 20 17:57:30 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 20 Oct 2016 13:57:30 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
Message-ID: <9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>

Hi,

Thanks for the comments..

Webrev's updated in place with comments so far.  (Including David's and 
Amy's)

http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/

http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/

On 10/20/2016 8:21 AM, Alan Bateman wrote:
> On 19/10/2016 20:59, Roger Riggs wrote:
>
>> The support in sun.reflect.ReflectionFactory for custom 
>> serialization, such as IIOP input
>> and output streams, is being expanded beyond the necessary 
>> constructor of a serializable
>> class to include access to the private methods readObject, 
>> writeObject, readResolve,
>> writeReplace, etc.
>>
>> The IIOP implementation is updated to use a combination of 
>> ReflectionFactory and
>> Unsafe to serialize and deserialize objects and no longer rely on 
>> setAccessible.
>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>
>> Please review and comment,
>>
>> jdk repo webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>> corba repo webrev :
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>
>>
> I skimmed through the changes.
>
> I assume findReadWriteObjectForSerialization should throw 
> InternalError, rather than return null, if IllegalAccessException is 
> thrown (as IAE is not possible here).
fixed
>
> You've added @since 9 to sun.reflect.ReflectionFactory but that class 
> is not new.
It was new in 9, but did not previously have @since. Its internal so it 
may not be important.
Added by 8137058: Clear out all non-Critical APIs from sun.reflect

>
> The javadoc for 
> sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't 
> say that it returns null when the Class is not Serializable.
The current implementation does not check that the argument is Serializable
and there are tests for that.  I will update the documentation to not 
specify a Serializable class.
I would need to track down all the uses to understand that adding the 
check would not break something.

It also does not check that the requested constructor is for a supertype.
I think the semantics of newConstructorForSerialization should include 
the search for
the super-type's noarg constructor instead of IIOP doing that search.

>
> For the MH returning methods then I assume the javadoc should say that 
> it returns a direct method handle.
ok
>
> The synchronization in the IIOP ObjectStreamClass isn't very clear. 
> Are the invoke*, read*, write* methods all invoked by the same thread 
> that creates the ObjectStreamClass with the lookup method?
ObjectStreamClass instances are cached and re-used across all threads.
ObjectStreamClass.init() handles the synchronization of the initialization.
Any thread using the ObjectStreamClass will use the same method handle 
regardless of the
thread calling the method.

Thanks, Roger



From iris.clark at oracle.com  Fri Oct 21 03:34:49 2016
From: iris.clark at oracle.com (Iris Clark)
Date: Thu, 20 Oct 2016 20:34:49 -0700 (PDT)
Subject: RFR: 8160954: (spec) Runtime.Version regex and $PRE/$OPT issues
Message-ID: <25c9fcb4-685e-4927-83aa-2039709d0e0c@default>

Hi.

Please review changes to address the following closely related bugs:

    8160954: (spec) Runtime.Version regex and $PRE/$OPT issues
    https://bugs.openjdk.java.net/browse/JDK-8160954

    8148822: (spec) Regex in Runtime.Version and JEP 223 should match
    https://bugs.openjdk.java.net/browse/JDK-8148822

    8148877: (spec) Specify when and empty '+' is required in a version
        string
    https://bugs.openjdk.java.net/browse/JDK-8148877

webrev:

    http://cr.openjdk.java.net/~iris/verona/8160954/webrev/

Changes to the specification of the $VNUM and $OPT regexp:

For line 963, we remove the leading '^' and trailing '$'.  These characters
are not strictly required for the regex to be accurate.  They're not included
in the implementation.  The presence of the trailing '$' precludes direct
substitution of this regex into the definition of $VSTR at line 1032 (see
bug 8160954).

Also for line 963, we remove the outermost qualifier in the portion of the
regex describing elements after $MAJOR.  This qualifier is redundant and a
source of catastrophic backtracking as described in this message [0, item 1]
(see bug 8148822).  It's not in the present implementation.

For line 1050, we remove the '\' as it is unnecessary in a character class
[0, item 4] (see bug 8148822).


Changes to the spec for $PRE/$OPT:

For lines 1056-1057, we clearly specify when an empty '+' is required (see
bugs 8148877, 8160954).  This spec corresponds to the code for additional
constraints for the "empty '+'" beginning at new line 1150.


The required CCC is in progress.

Thanks,
iris

[0] http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038036.html
[1] http://download.java.net/java/jdk9/docs/api/java/lang/Runtime.Version.html

From peter.levart at gmail.com  Fri Oct 21 07:08:11 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Fri, 21 Oct 2016 09:08:11 +0200
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
	<9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
Message-ID: <e9605a23-63a9-5380-c916-c87f9d1cf5f7@gmail.com>

Hi Roger,

I don't know whether the performance of invoking 
sun.reflect.ReflactionFactory methods matters very much, but you could 
do a nit to help JIT optimize their invocation better. As both 
jdk.internal.reflect.ReflectionFactory and the 
sun.reflect.ReflectionFactory are singletons, you could use the strategy 
of sun.misc.Unsafe which references the delegate instance 
(theInternalUnsafe) off a static final field instead of instance final 
field. JIT can constant-fold such reference then.

Otherwise I think it is a good strategy to give frameworks limited 
access to selected private methods targeted for a special purpose - 
serialization in this case, instead of opening the doors for generic 
reflection (via setAccessible). It just feels a little strange that this 
is new API in an "unsupported" module which destiny is to eventually 
"fade-away" once required functionality appears in a supported way. What 
is the message?

Regards, Peter

On 10/20/2016 07:57 PM, Roger Riggs wrote:
> Hi,
>
> Thanks for the comments..
>
> Webrev's updated in place with comments so far.  (Including David's 
> and Amy's)
>
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>
>
> On 10/20/2016 8:21 AM, Alan Bateman wrote:
>> On 19/10/2016 20:59, Roger Riggs wrote:
>>
>>> The support in sun.reflect.ReflectionFactory for custom 
>>> serialization, such as IIOP input
>>> and output streams, is being expanded beyond the necessary 
>>> constructor of a serializable
>>> class to include access to the private methods readObject, 
>>> writeObject, readResolve,
>>> writeReplace, etc.
>>>
>>> The IIOP implementation is updated to use a combination of 
>>> ReflectionFactory and
>>> Unsafe to serialize and deserialize objects and no longer rely on 
>>> setAccessible.
>>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>>
>>> Please review and comment,
>>>
>>> jdk repo webrev:
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>> corba repo webrev :
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>>
>>>
>> I skimmed through the changes.
>>
>> I assume findReadWriteObjectForSerialization should throw 
>> InternalError, rather than return null, if IllegalAccessException is 
>> thrown (as IAE is not possible here).
> fixed
>>
>> You've added @since 9 to sun.reflect.ReflectionFactory but that class 
>> is not new.
> It was new in 9, but did not previously have @since. Its internal so 
> it may not be important.
> Added by 8137058: Clear out all non-Critical APIs from sun.reflect
>
>>
>> The javadoc for 
>> sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't 
>> say that it returns null when the Class is not Serializable.
> The current implementation does not check that the argument is 
> Serializable
> and there are tests for that.  I will update the documentation to not 
> specify a Serializable class.
> I would need to track down all the uses to understand that adding the 
> check would not break something.
>
> It also does not check that the requested constructor is for a supertype.
> I think the semantics of newConstructorForSerialization should include 
> the search for
> the super-type's noarg constructor instead of IIOP doing that search.
>
>>
>> For the MH returning methods then I assume the javadoc should say 
>> that it returns a direct method handle.
> ok
>>
>> The synchronization in the IIOP ObjectStreamClass isn't very clear. 
>> Are the invoke*, read*, write* methods all invoked by the same thread 
>> that creates the ObjectStreamClass with the lookup method?
> ObjectStreamClass instances are cached and re-used across all threads.
> ObjectStreamClass.init() handles the synchronization of the 
> initialization.
> Any thread using the ObjectStreamClass will use the same method handle 
> regardless of the
> thread calling the method.
>
> Thanks, Roger
>
>


From Alan.Bateman at oracle.com  Fri Oct 21 07:21:18 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 21 Oct 2016 08:21:18 +0100
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <e9605a23-63a9-5380-c916-c87f9d1cf5f7@gmail.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
	<9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
	<e9605a23-63a9-5380-c916-c87f9d1cf5f7@gmail.com>
Message-ID: <a938313b-298f-0342-4d3f-2d745ee622d2@oracle.com>

On 21/10/2016 08:08, Peter Levart wrote:

> :
>
> Otherwise I think it is a good strategy to give frameworks limited 
> access to selected private methods targeted for a special purpose - 
> serialization in this case, instead of opening the doors for generic 
> reflection (via setAccessible). It just feels a little strange that 
> this is new API in an "unsupported" module which destiny is to 
> eventually "fade-away" once required functionality appears in a 
> supported way. What is the message?
Custom serialization libraries have always had to use ReflectionFactory 
so the message and they'll have to use it more in order to work with 
serializable types in modules using strong encapsaultion. The plan is to 
update ReflectionFactory in a JDK 8 update soon, assuming the jdk8u 
maintainers approve. It might seem strange to be doing this but it's way 
too late in JDK 9 to be taking on another serialization project.

-Alan

From chris.hegarty at oracle.com  Fri Oct 21 10:36:53 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Fri, 21 Oct 2016 11:36:53 +0100
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
	<9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
Message-ID: <ebc94ac9-e143-7ded-63db-a2dd736157e2@oracle.com>

Roger,

On 20/10/16 18:57, Roger Riggs wrote:
> Hi,
>
> Thanks for the comments..
>
> Webrev's updated in place with comments so far.  (Including David's and
> Amy's)
>
> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/

This looks very good Roger. A few very minor comments:

  1) The explicit initialization of hasStaticInitializerMethod to null
     is unnecessary.

  2) hasStaticInitializerForSerialization() can have its setAccessible
     call moved to just before the invoke.

  3) Remove the @since 9 from s.m.RF. This class exists on previous
     JDK's. 8137058 moved it originally and added a new file in its
     place to help preserve the history of the file.

  4) s.m.RF, Return -> Returns

> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/

No specific comments.

-Chris.

> On 10/20/2016 8:21 AM, Alan Bateman wrote:
>> On 19/10/2016 20:59, Roger Riggs wrote:
>>
>>> The support in sun.reflect.ReflectionFactory for custom
>>> serialization, such as IIOP input
>>> and output streams, is being expanded beyond the necessary
>>> constructor of a serializable
>>> class to include access to the private methods readObject,
>>> writeObject, readResolve,
>>> writeReplace, etc.
>>>
>>> The IIOP implementation is updated to use a combination of
>>> ReflectionFactory and
>>> Unsafe to serialize and deserialize objects and no longer rely on
>>> setAccessible.
>>> Tests are included for ReflectionFactory and the affected IIOP classes.
>>>
>>> Please review and comment,
>>>
>>> jdk repo webrev:
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>> corba repo webrev :
>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/
>>>
>>>
>> I skimmed through the changes.
>>
>> I assume findReadWriteObjectForSerialization should throw
>> InternalError, rather than return null, if IllegalAccessException is
>> thrown (as IAE is not possible here).
> fixed
>>
>> You've added @since 9 to sun.reflect.ReflectionFactory but that class
>> is not new.
> It was new in 9, but did not previously have @since. Its internal so it
> may not be important.
> Added by 8137058: Clear out all non-Critical APIs from sun.reflect
>
>>
>> The javadoc for
>> sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't
>> say that it returns null when the Class is not Serializable.
> The current implementation does not check that the argument is Serializable
> and there are tests for that.  I will update the documentation to not
> specify a Serializable class.
> I would need to track down all the uses to understand that adding the
> check would not break something.
>
> It also does not check that the requested constructor is for a supertype.
> I think the semantics of newConstructorForSerialization should include
> the search for
> the super-type's noarg constructor instead of IIOP doing that search.
>
>>
>> For the MH returning methods then I assume the javadoc should say that
>> it returns a direct method handle.
> ok
>>
>> The synchronization in the IIOP ObjectStreamClass isn't very clear.
>> Are the invoke*, read*, write* methods all invoked by the same thread
>> that creates the ObjectStreamClass with the lookup method?
> ObjectStreamClass instances are cached and re-used across all threads.
> ObjectStreamClass.init() handles the synchronization of the initialization.
> Any thread using the ObjectStreamClass will use the same method handle
> regardless of the
> thread calling the method.
>
> Thanks, Roger
>
>

From Roger.Riggs at Oracle.com  Fri Oct 21 14:26:33 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 21 Oct 2016 10:26:33 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <e9605a23-63a9-5380-c916-c87f9d1cf5f7@gmail.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
	<9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
	<e9605a23-63a9-5380-c916-c87f9d1cf5f7@gmail.com>
Message-ID: <4978eca8-f149-e427-3b43-46f30fc4d4b5@Oracle.com>

Hi Peter,

Thanks for the review and suggestions.

Will update the webrev shortly.

On 10/21/2016 3:08 AM, Peter Levart wrote:
> Hi Roger,
>
> I don't know whether the performance of invoking 
> sun.reflect.ReflactionFactory methods matters very much, but you could 
> do a nit to help JIT optimize their invocation better. As both 
> jdk.internal.reflect.ReflectionFactory and the 
> sun.reflect.ReflectionFactory are singletons, you could use the 
> strategy of sun.misc.Unsafe which references the delegate instance 
> (theInternalUnsafe) off a static final field instead of instance final 
> field. JIT can constant-fold such reference then.
Will do.
>
> Otherwise I think it is a good strategy to give frameworks limited 
> access to selected private methods targeted for a special purpose - 
> serialization in this case, instead of opening the doors for generic 
> reflection (via setAccessible). It just feels a little strange that 
> this is new API in an "unsupported" module which destiny is to 
> eventually "fade-away" once required functionality appears in a 
> supported way. What is the message?
Alan answered already.

Roger

>
> Regards, Peter
>
> On 10/20/2016 07:57 PM, Roger Riggs wrote:
>> Hi,
>>
>> Thanks for the comments..
>>
>> Webrev's updated in place with comments so far.  (Including David's 
>> and Amy's)
>>
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>
>>
>> On 10/20/2016 8:21 AM, Alan Bateman wrote:
>>> On 19/10/2016 20:59, Roger Riggs wrote:
>>>
>>>> The support in sun.reflect.ReflectionFactory for custom 
>>>> serialization, such as IIOP input
>>>> and output streams, is being expanded beyond the necessary 
>>>> constructor of a serializable
>>>> class to include access to the private methods readObject, 
>>>> writeObject, readResolve,
>>>> writeReplace, etc.
>>>>
>>>> The IIOP implementation is updated to use a combination of 
>>>> ReflectionFactory and
>>>> Unsafe to serialize and deserialize objects and no longer rely on 
>>>> setAccessible.
>>>> Tests are included for ReflectionFactory and the affected IIOP 
>>>> classes.
>>>>
>>>> Please review and comment,
>>>>
>>>> jdk repo webrev:
>>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>>> corba repo webrev :
>>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>>>
>>>>
>>> I skimmed through the changes.
>>>
>>> I assume findReadWriteObjectForSerialization should throw 
>>> InternalError, rather than return null, if IllegalAccessException is 
>>> thrown (as IAE is not possible here).
>> fixed
>>>
>>> You've added @since 9 to sun.reflect.ReflectionFactory but that 
>>> class is not new.
>> It was new in 9, but did not previously have @since. Its internal so 
>> it may not be important.
>> Added by 8137058: Clear out all non-Critical APIs from sun.reflect
>>
>>>
>>> The javadoc for 
>>> sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't 
>>> say that it returns null when the Class is not Serializable.
>> The current implementation does not check that the argument is 
>> Serializable
>> and there are tests for that.  I will update the documentation to not 
>> specify a Serializable class.
>> I would need to track down all the uses to understand that adding the 
>> check would not break something.
>>
>> It also does not check that the requested constructor is for a 
>> supertype.
>> I think the semantics of newConstructorForSerialization should 
>> include the search for
>> the super-type's noarg constructor instead of IIOP doing that search.
>>
>>>
>>> For the MH returning methods then I assume the javadoc should say 
>>> that it returns a direct method handle.
>> ok
>>>
>>> The synchronization in the IIOP ObjectStreamClass isn't very clear. 
>>> Are the invoke*, read*, write* methods all invoked by the same 
>>> thread that creates the ObjectStreamClass with the lookup method?
>> ObjectStreamClass instances are cached and re-used across all threads.
>> ObjectStreamClass.init() handles the synchronization of the 
>> initialization.
>> Any thread using the ObjectStreamClass will use the same method 
>> handle regardless of the
>> thread calling the method.
>>
>> Thanks, Roger
>>
>>
>


From Roger.Riggs at Oracle.com  Fri Oct 21 14:47:58 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 21 Oct 2016 10:47:58 -0400
Subject: RFR 9: 8164908: ReflectionFactory support for IIOP and custom
	serialization
In-Reply-To: <ebc94ac9-e143-7ded-63db-a2dd736157e2@oracle.com>
References: <b67c952a-1dd8-59d7-3ee7-8d69c4a6614b@Oracle.com>
	<8e9f327f-e5bc-c968-c6ac-eb202b8dcadf@oracle.com>
	<71294ad7-2cdc-97c8-ff59-b76e5fa10cd7@Oracle.com>
	<1f14f70c-5aee-a689-d7d3-08373c92cc67@oracle.com>
	<9d0e98ce-741a-3456-72aa-e7245d5cac93@Oracle.com>
	<ebc94ac9-e143-7ded-63db-a2dd736157e2@oracle.com>
Message-ID: <9eae791e-3cbc-aa86-8dbf-f9cef8e81ca3@Oracle.com>

Hi Chris,

On 10/21/2016 6:36 AM, Chris Hegarty wrote:
> Roger,
>
> On 20/10/16 18:57, Roger Riggs wrote:
>> Hi,
>>
>> Thanks for the comments..
>>
>> Webrev's updated in place with comments so far.  (Including David's and
>> Amy's)
>>
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>
> This looks very good Roger. A few very minor comments:
>
>  1) The explicit initialization of hasStaticInitializerMethod to null
>     is unnecessary.
removed initializer
>
>  2) hasStaticInitializerForSerialization() can have its setAccessible
>     call moved to just before the invoke.
The method is cached in hasStaticInitializerMethod and does not need to be
set every time the method is called.  As is, setAccessible is only 
called once.
There's no need to repeat the security checks on every invocation.

>
>  3) Remove the @since 9 from s.m.RF. This class exists on previous
>     JDK's. 8137058 moved it originally and added a new file in its
>     place to help preserve the history of the file.
ok
>
>  4) s.m.RF, Return -> Returns
Right, fixed correctly

Thanks, Roger

>
>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>
>
> No specific comments.
>
> -Chris.
>
>> On 10/20/2016 8:21 AM, Alan Bateman wrote:
>>> On 19/10/2016 20:59, Roger Riggs wrote:
>>>
>>>> The support in sun.reflect.ReflectionFactory for custom
>>>> serialization, such as IIOP input
>>>> and output streams, is being expanded beyond the necessary
>>>> constructor of a serializable
>>>> class to include access to the private methods readObject,
>>>> writeObject, readResolve,
>>>> writeReplace, etc.
>>>>
>>>> The IIOP implementation is updated to use a combination of
>>>> ReflectionFactory and
>>>> Unsafe to serialize and deserialize objects and no longer rely on
>>>> setAccessible.
>>>> Tests are included for ReflectionFactory and the affected IIOP 
>>>> classes.
>>>>
>>>> Please review and comment,
>>>>
>>>> jdk repo webrev:
>>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-8164908/
>>>> corba repo webrev :
>>>> http://cr.openjdk.java.net/~rriggs/webrev-reflection-factory-iiop-8164908/ 
>>>>
>>>>
>>>>
>>> I skimmed through the changes.
>>>
>>> I assume findReadWriteObjectForSerialization should throw
>>> InternalError, rather than return null, if IllegalAccessException is
>>> thrown (as IAE is not possible here).
>> fixed
>>>
>>> You've added @since 9 to sun.reflect.ReflectionFactory but that class
>>> is not new.
>> It was new in 9, but did not previously have @since. Its internal so it
>> may not be important.
>> Added by 8137058: Clear out all non-Critical APIs from sun.reflect
>>
>>>
>>> The javadoc for
>>> sun.reflect.ReflectionFactory.newConstructorForSerialization doesn't
>>> say that it returns null when the Class is not Serializable.
>> The current implementation does not check that the argument is 
>> Serializable
>> and there are tests for that.  I will update the documentation to not
>> specify a Serializable class.
>> I would need to track down all the uses to understand that adding the
>> check would not break something.
>>
>> It also does not check that the requested constructor is for a 
>> supertype.
>> I think the semantics of newConstructorForSerialization should include
>> the search for
>> the super-type's noarg constructor instead of IIOP doing that search.
>>
>>>
>>> For the MH returning methods then I assume the javadoc should say that
>>> it returns a direct method handle.
>> ok
>>>
>>> The synchronization in the IIOP ObjectStreamClass isn't very clear.
>>> Are the invoke*, read*, write* methods all invoked by the same thread
>>> that creates the ObjectStreamClass with the lookup method?
>> ObjectStreamClass instances are cached and re-used across all threads.
>> ObjectStreamClass.init() handles the synchronization of the 
>> initialization.
>> Any thread using the ObjectStreamClass will use the same method handle
>> regardless of the
>> thread calling the method.
>>
>> Thanks, Roger
>>
>>


From brunoaiss at gmail.com  Fri Oct 21 16:08:18 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Fri, 21 Oct 2016 17:08:18 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
Message-ID: <22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>

Any feedback on this? I'm really interested in implementing such 
BufferedReader/BufferedStreamReader to allow speeding up my applications 
without having to think in an asynchronous way or multi-threading while 
programming with it.

That's why I'm asking this here.


On 13/10/2016 14:45, Brunoais wrote:
> Hi,
>
> I looked at BufferedReader source code for java 9 long with the source 
> code of the channels/streams used. I noticed that, like in java 7, 
> BufferedReader does not use an Async API to load data from files, 
> instead, the data loading is all done synchronously even when the OS 
> allows requesting a file to be read and getting a warning later when 
> the file is effectively read.
>
> Why Is BufferedReader not async while providing a sync API?
>


From Roger.Riggs at Oracle.com  Fri Oct 21 17:16:18 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 21 Oct 2016 13:16:18 -0400
Subject: RFR 9: 8167192 [Testbug] : java/io/Serializable/serialFilter test
	conditions wrong
Message-ID: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>

Please review a test correction to the permissions granted to the test
and the check that the global filter cannot be re-set if once set.
Removed unnecessary code supplying a OIS when testing setting the global 
filter.

Webrev:
    http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/

Thanks, Roger



From pavel.rappo at oracle.com  Fri Oct 21 17:22:55 2016
From: pavel.rappo at oracle.com (Pavel Rappo)
Date: Fri, 21 Oct 2016 18:22:55 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
Message-ID: <E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>

Off the top of my head, I would say it's not possible to change the design of an
_extensible_ type that has been out there for 20 or so years. All these I/O
streams from java.io were designed for simple synchronous use case.

It's not that their design is flawed in some way, it's that they doesn't seem to
suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel 
in your applications?

-Pavel

> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
> 
> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
> 
> That's why I'm asking this here.
> 
> 
> On 13/10/2016 14:45, Brunoais wrote:
>> Hi,
>> 
>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>> 
>> Why Is BufferedReader not async while providing a sync API?
>> 
> 


From lance.andersen at oracle.com  Fri Oct 21 17:28:49 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 21 Oct 2016 13:28:49 -0400
Subject: RFR 9: 8167192 [Testbug] : java/io/Serializable/serialFilter test
	conditions wrong
In-Reply-To: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
References: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
Message-ID: <C96EFCCF-52E7-410D-A009-E69BC41014E8@oracle.com>

Looks OK Roger

Best
Lance
> On Oct 21, 2016, at 1:16 PM, Roger Riggs <Roger.Riggs at oracle.com> wrote:
> 
> Please review a test correction to the permissions granted to the test
> and the check that the global filter cannot be re-set if once set.
> Removed unnecessary code supplying a OIS when testing setting the global filter.
> 
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/
> 
> Thanks, Roger
> 
> 

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From roger.riggs at oracle.com  Fri Oct 21 17:38:38 2016
From: roger.riggs at oracle.com (Roger Riggs)
Date: Fri, 21 Oct 2016 13:38:38 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
Message-ID: <1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>

Hi Pavel,

I think Brunoais asking for a double buffering scheme in which the 
implementation of
BufferReader fills (a second buffer) in parallel with the application 
reading from the 1st buffer
and managing the swaps and async reads transparently.
It would not change the API but would change the interactions between 
the buffered reader
and the underlying stream.  It would also increase memory requirements 
and processing
by introducing or using a separate thread and the necessary synchronization.

Though I think the formal interface semantics could be maintained, I 
have doubts
about compatibility and its unintended consequences on existing subclasses,
applications and libraries.

$.02, Roger

On 10/21/16 1:22 PM, Pavel Rappo wrote:
> Off the top of my head, I would say it's not possible to change the design of an
> _extensible_ type that has been out there for 20 or so years. All these I/O
> streams from java.io were designed for simple synchronous use case.
>
> It's not that their design is flawed in some way, it's that they doesn't seem to
> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
> in your applications?
>
> -Pavel
>
>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>
>> That's why I'm asking this here.
>>
>>
>> On 13/10/2016 14:45, Brunoais wrote:
>>> Hi,
>>>
>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>
>>> Why Is BufferedReader not async while providing a sync API?
>>>


From brunoaiss at gmail.com  Fri Oct 21 17:56:43 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Fri, 21 Oct 2016 18:56:43 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
Message-ID: <1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>

Pavel is right.

In reality, I was expecting such BufferedReader to use only a single 
buffer and have that Buffer being filled asynchronously, not in a 
different Thread.
Additionally, I don't have the intention of having a larger buffer than 
before unless stated through the API (the constructor).

In my idea, internally, it is supposed to use 
java.nio.channels.AsynchronousFileChannel or equivalent.

It does not prevent having two buffers and I do not intent to change 
BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name 
suggestion is welcome as I'm an awful namer).


On 21/10/2016 18:38, Roger Riggs wrote:
> Hi Pavel,
>
> I think Brunoais asking for a double buffering scheme in which the 
> implementation of
> BufferReader fills (a second buffer) in parallel with the application 
> reading from the 1st buffer
> and managing the swaps and async reads transparently.
> It would not change the API but would change the interactions between 
> the buffered reader
> and the underlying stream.  It would also increase memory requirements 
> and processing
> by introducing or using a separate thread and the necessary 
> synchronization.
>
> Though I think the formal interface semantics could be maintained, I 
> have doubts
> about compatibility and its unintended consequences on existing 
> subclasses,
> applications and libraries.
>
> $.02, Roger
>
> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>> Off the top of my head, I would say it's not possible to change the 
>> design of an
>> _extensible_ type that has been out there for 20 or so years. All 
>> these I/O
>> streams from java.io were designed for simple synchronous use case.
>>
>> It's not that their design is flawed in some way, it's that they 
>> doesn't seem to
>> suit your needs. Have you considered using 
>> java.nio.channels.AsynchronousFileChannel
>> in your applications?
>>
>> -Pavel
>>
>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>
>>> Any feedback on this? I'm really interested in implementing such 
>>> BufferedReader/BufferedStreamReader to allow speeding up my 
>>> applications without having to think in an asynchronous way or 
>>> multi-threading while programming with it.
>>>
>>> That's why I'm asking this here.
>>>
>>>
>>> On 13/10/2016 14:45, Brunoais wrote:
>>>> Hi,
>>>>
>>>> I looked at BufferedReader source code for java 9 long with the 
>>>> source code of the channels/streams used. I noticed that, like in 
>>>> java 7, BufferedReader does not use an Async API to load data from 
>>>> files, instead, the data loading is all done synchronously even 
>>>> when the OS allows requesting a file to be read and getting a 
>>>> warning later when the file is effectively read.
>>>>
>>>> Why Is BufferedReader not async while providing a sync API?
>>>>
>
>


From Roger.Riggs at Oracle.com  Fri Oct 21 18:18:50 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 21 Oct 2016 14:18:50 -0400
Subject: RFR 8167646: Better invalid FilePermission
In-Reply-To: <F29526E9-C44F-42FC-AE11-F2246BB0EA45@oracle.com>
References: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
	<F29526E9-C44F-42FC-AE11-F2246BB0EA45@oracle.com>
Message-ID: <29fe6a9e-a289-0fd0-6431-897279f2eea2@Oracle.com>

Hi Max,

Looks ok.
An additional reviewer would be good too.

Roger


On 10/20/2016 4:51 AM, Wang Weijun wrote:
> Please review the code change at
>
>     http://cr.openjdk.java.net/~weijun/8167646/webrev.00/
>
> A new flag invalid is added so invalid FilePermissions (invalid Path) do not equal or imply or is implied by anything else except for itself.
>
> Thanks
> Max
>


From pavel.rappo at oracle.com  Fri Oct 21 18:21:33 2016
From: pavel.rappo at oracle.com (Pavel Rappo)
Date: Fri, 21 Oct 2016 19:21:33 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
Message-ID: <F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>

Just to append to my previous email. BufferedReader wraps any Reader out there.
Not specifically FileReader. While you're talking about the case of effective
reading from a file.

I guess there's one existing possibility to provide exactly what you need (as I
understand it) under this method:

/**
 * Opens a file for reading, returning a {@code BufferedReader} to read text
 * from the file in an efficient manner...
   ...
 */
java.nio.file.Files#newBufferedReader(java.nio.file.Path)

It can return _anything_ as long as it is a BufferedReader. We can do it, but it
needs to be investigated not only for your favorite OS but for other OSes as
well. Feel free to prototype this and we can discuss it on the list later.

Thanks,
-Pavel

> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
> 
> Pavel is right.
> 
> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
> 
> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
> 
> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
> 
> 
> On 21/10/2016 18:38, Roger Riggs wrote:
>> Hi Pavel,
>> 
>> I think Brunoais asking for a double buffering scheme in which the implementation of
>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>> and managing the swaps and async reads transparently.
>> It would not change the API but would change the interactions between the buffered reader
>> and the underlying stream.  It would also increase memory requirements and processing
>> by introducing or using a separate thread and the necessary synchronization.
>> 
>> Though I think the formal interface semantics could be maintained, I have doubts
>> about compatibility and its unintended consequences on existing subclasses,
>> applications and libraries.
>> 
>> $.02, Roger
>> 
>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>> Off the top of my head, I would say it's not possible to change the design of an
>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>> streams from java.io were designed for simple synchronous use case.
>>> 
>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>> in your applications?
>>> 
>>> -Pavel
>>> 
>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>> 
>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>> 
>>>> That's why I'm asking this here.
>>>> 
>>>> 
>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>> Hi,
>>>>> 
>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>> 
>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>> 
>> 
>> 
> 


From brunoaiss at gmail.com  Fri Oct 21 18:44:45 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Fri, 21 Oct 2016 19:44:45 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
Message-ID: <e1d252d6-5619-7858-9ffa-bf3ac3b84d99@gmail.com>

OK. Will do.


On 21/10/2016 19:21, Pavel Rappo wrote:
> Just to append to my previous email. BufferedReader wraps any Reader out there.
> Not specifically FileReader. While you're talking about the case of effective
> reading from a file.
>
> I guess there's one existing possibility to provide exactly what you need (as I
> understand it) under this method:
>
> /**
>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>   * from the file in an efficient manner...
>     ...
>   */
> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>
> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
> needs to be investigated not only for your favorite OS but for other OSes as
> well. Feel free to prototype this and we can discuss it on the list later.
>
> Thanks,
> -Pavel
>
>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Pavel is right.
>>
>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>
>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>
>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>
>>
>> On 21/10/2016 18:38, Roger Riggs wrote:
>>> Hi Pavel,
>>>
>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>> and managing the swaps and async reads transparently.
>>> It would not change the API but would change the interactions between the buffered reader
>>> and the underlying stream.  It would also increase memory requirements and processing
>>> by introducing or using a separate thread and the necessary synchronization.
>>>
>>> Though I think the formal interface semantics could be maintained, I have doubts
>>> about compatibility and its unintended consequences on existing subclasses,
>>> applications and libraries.
>>>
>>> $.02, Roger
>>>
>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>> streams from java.io were designed for simple synchronous use case.
>>>>
>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>> in your applications?
>>>>
>>>> -Pavel
>>>>
>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>
>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>
>>>>> That's why I'm asking this here.
>>>>>
>>>>>
>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>
>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>
>>>
>


From david.lloyd at redhat.com  Fri Oct 21 18:58:15 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 21 Oct 2016 13:58:15 -0500
Subject: Another ProcessBuilder enhancement: execve
Message-ID: <fef89d76-31e9-9087-2b0a-1ad9efaccd68@redhat.com>

It would be useful for processes which self-update or otherwise 
self-manage to be able to exec a new process which replaces the current 
one, in the manner of POSIX execve.  This might tie in with 
ProcessBuilder, though there are a few key differences:

1. The method to execute the process (obviously) would never normally 
return, similarly to how System.exit() doesn't normally return.
2. The options for redirection would be necessarily limited to File, 
INHERIT, or DISCARD; PIPE does not make sense since the current process 
would be eliminated.

Another interesting characteristic is that this would be essentially a 
hybrid between System.exit() and ProcessBuilder.start(), from both an 
execution perspective (as mentioned above) and a security perspective, 
in that the caller would have to be authorized both to terminate the VM 
(i.e. SecurityManager.checkExit(0), or potentially a new check specific 
to this operation) and also to exec the target process (i.e. 
SecurityManager.checkExec(fileName)).

Exiting or exec'ing another process during JVM shutdown-for-exec should 
be disallowed, though the former could be tolerated in the same way that 
calling exit() during an in-progress shutdown is tolerated today (i.e. 
indefinite blocking).

My understanding is that Windows has at least one way to accomplish the 
same effect as well, but that bears more research than my quick web search.

I know the time is past for 9, but I think this idea might be good to 
explore for 10, or at least make for an interesting discussion.

Thoughts?
-- 
- DML

From martinrb at google.com  Fri Oct 21 19:07:14 2016
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 21 Oct 2016 12:07:14 -0700
Subject: Another ProcessBuilder enhancement: execve
In-Reply-To: <fef89d76-31e9-9087-2b0a-1ad9efaccd68@redhat.com>
References: <fef89d76-31e9-9087-2b0a-1ad9efaccd68@redhat.com>
Message-ID: <CA+kOe0_YQtwLGSEAJkC0ApX_ESRODhh_B26mcNehJ9p+osi-yw@mail.gmail.com>

Historically, this was thought to be not an option because Windows doesn't
have exec (or fork).
Cygwin's emulation of fork does create a windows subprocess, I believe.

The desire to have clean shutdown of the Java process does make things more
interesting as well, as you point out.

One use case is discovering you would like different JVM flags while you
are running.

On Fri, Oct 21, 2016 at 11:58 AM, David M. Lloyd <david.lloyd at redhat.com>
wrote:

> It would be useful for processes which self-update or otherwise
> self-manage to be able to exec a new process which replaces the current
> one, in the manner of POSIX execve.  This might tie in with ProcessBuilder,
> though there are a few key differences:
>
> 1. The method to execute the process (obviously) would never normally
> return, similarly to how System.exit() doesn't normally return.
> 2. The options for redirection would be necessarily limited to File,
> INHERIT, or DISCARD; PIPE does not make sense since the current process
> would be eliminated.
>
> Another interesting characteristic is that this would be essentially a
> hybrid between System.exit() and ProcessBuilder.start(), from both an
> execution perspective (as mentioned above) and a security perspective, in
> that the caller would have to be authorized both to terminate the VM (i.e.
> SecurityManager.checkExit(0), or potentially a new check specific to this
> operation) and also to exec the target process (i.e.
> SecurityManager.checkExec(fileName)).
>
> Exiting or exec'ing another process during JVM shutdown-for-exec should be
> disallowed, though the former could be tolerated in the same way that
> calling exit() during an in-progress shutdown is tolerated today (i.e.
> indefinite blocking).
>
> My understanding is that Windows has at least one way to accomplish the
> same effect as well, but that bears more research than my quick web search.
>
> I know the time is past for 9, but I think this idea might be good to
> explore for 10, or at least make for an interesting discussion.
>
> Thoughts?
> --
> - DML
>

From martinrb at google.com  Fri Oct 21 19:32:50 2016
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 21 Oct 2016 12:32:50 -0700
Subject: RFR: jsr166 jdk9 integration wave 12
In-Reply-To: <CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
References: <CA+kOe0_+MTANgEoXA9=WKf=uqg90wTLTdRn==Mm98B2YH3iUhg@mail.gmail.com>
	<8cc0d5a9-9245-e6b1-0ff2-616127dbc583@oracle.com>
	<CA+kOe082i18c0VRLjorYRJeofGb73tNJSMnXHLxBBEY3AUQTFQ@mail.gmail.com>
	<CAHjP37GMU0YEkSrfnEGr-hjKJqLo0PkNo3uhzNDHw0UUZb3f_Q@mail.gmail.com>
	<CA+kOe09TPcOuPuAVsKohJuPG=xtL_HkoiiuMqiTx+GgLt181mA@mail.gmail.com>
Message-ID: <CA+kOe08gJXT8m0FsNjj+UX5AnO_49YLoE27U8rN=edhak2aHOg@mail.gmail.com>

On Tue, Oct 18, 2016 at 4:18 PM, Martin Buchholz <martinrb at google.com>
wrote:

>
>
> On Tue, Oct 18, 2016 at 4:00 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
>
>>
>>> > * Change in allocation/capacity policy.
>>> >
>>> > The removal of the power-of-two restriction, and applying a 1.5x growth
>>> > factor (same as ArrayList) seems sensible. Does this mean that the
>>> ability
>>> > to compute the proper array index by using x & (length-1) wasn't worth
>>> it?
>>> > Or at least not worth the extra tail wastage?
>>> >
>>>
>>> There's no integer division to optimize, as with hash tables.
>>
>> But it does add some new branches, doesn't it? Potentially branches that
>> aren't taken prior to JIT compilation, but taken later = deopt.
>>
>
> If it's a smidgeon slower, I don't care that much; improvement in
> flexibility and scalability are more important.
>
> Of course, I do care about algorithmic improvements to e.g. removeIf
>
>
>> Curious if you ran some benchmarks on ArrayDeque.
>>
>
> Not yet.  Who would like to show how slow the code is?
>

I revived my ancient IteratorMicroBenchmark for the latest webrev, made it
a proper jtreg test, added ArrayDeque Jobs, and verified that the new code
is measurably faster than the old code, at least for the mundane job of
iterating.
http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ArrayList/

From nakamura at hkg.ac.jp  Sat Oct 22 04:34:48 2016
From: nakamura at hkg.ac.jp (nakamura at hkg.ac.jp)
Date: Sat, 22 Oct 2016 13:34:48 +0900
Subject: =?UTF-8?Q?Re:_RFR:_6378384_=28reflect=29_subclass_can=e2=80=99t_acc?=
	=?UTF-8?Q?ess_superclass=e2=80=99s_protected_fields_and_methods_by_reflecti?=
	=?UTF-8?Q?on?=
In-Reply-To: <8d14ab62-7bc3-d69d-642e-6e645bf846f4@gmail.com>
References: <629c397e-2813-15be-cccf-6436a7376289@gmail.com>
	<bb19b165-6a51-fa49-3ecb-029e95033714@gmail.com>
	<CE82A970-72AD-4D61-AB11-EE94CB906715@oracle.com>
	<23885332-189f-730e-bd2d-165859701d41@gmail.com>
	<A7198EFF-B5BE-41A5-96C8-BE64A9357FF6@oracle.com>
	<bb12d6a1-ab3b-3605-a3aa-fb55b41254ae@gmail.com>
	<B46A6990-63CF-45C9-A719-F82109B79E92@oracle.com>
	<0f3d14df-00d9-1f52-85e6-c64a3de55362@gmail.com>
	<B9A0D252-082A-48C4-A8F3-B6F4B7F9C05B@oracle.com>
	<8d14ab62-7bc3-d69d-642e-6e645bf846f4@gmail.com>
Message-ID: <a08693c4-5181-e969-7636-f264c6191770@hkg.ac.jp>

Dear Mr. Peter Levart:

>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/6378384_Reflection.ensureAccess/webrev.04/
>> +1.  Ship it!  Thanks for compressing it!
> Thanks for reviewing. Pushed.

Thank you very much. 6378384 was submitted by me.
Is your fix is included? By using JDK 9-ea+140-jigsaw, the subclass can 
access its superclass's protected static fields and methods by reflection.

> Hope it's not too bad that I managed to enter non-ascii characters into
> commit message. I was copy-pasting from Jira issue title and didn't
> notice the difference between ' and ? . Probably an artifact of
> migrating issues from SUN's old bug-tracking system. This issue really
> had a long beard...

Sorry. I maybe copied the text from MS-Word.

Yours sincerely,
Manabu Nakamura

From mandy.chung at oracle.com  Sat Oct 22 16:02:59 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 22 Oct 2016 09:02:59 -0700
Subject: RFR 9: 8167192 [Testbug] : java/io/Serializable/serialFilter test
	conditions wrong
In-Reply-To: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
References: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
Message-ID: <A11C9212-4065-4A8D-8DB5-55FC8369CCD9@oracle.com>


> On Oct 21, 2016, at 10:16 AM, Roger Riggs <Roger.Riggs at oracle.com> wrote:
> 
> Please review a test correction to the permissions granted to the test
> and the check that the global filter cannot be re-set if once set.
> Removed unnecessary code supplying a OIS when testing setting the global filter.
> 
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/

Should the test fail when the test does not have java.io.SerializablePermission(?serialFilter?) permission?  Has the test been passing with this typo?
 
One more typo in FilterWithSecurityManagerTest.java
  75                     + "java.security.SerializablePermission(serialFilter) ?

Mandy



From Roger.Riggs at Oracle.com  Sat Oct 22 16:43:07 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Sat, 22 Oct 2016 12:43:07 -0400
Subject: RFR 9: 8167192 [Testbug] : java/io/Serializable/serialFilter test
	conditions wrong
In-Reply-To: <A11C9212-4065-4A8D-8DB5-55FC8369CCD9@oracle.com>
References: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
	<A11C9212-4065-4A8D-8DB5-55FC8369CCD9@oracle.com>
Message-ID: <17825685-5128-1e22-64ef-910d22c39636@Oracle.com>

Hi Mandy,

Thanks for catching that typo in the error message.
The fix of the permission in the security.policy exposed bugs in the 
tests that are being addressed.

Webrev updated in place.
    http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/

Thanks, Roger


On 10/22/2016 12:02 PM, Mandy Chung wrote:
>> On Oct 21, 2016, at 10:16 AM, Roger Riggs <Roger.Riggs at oracle.com> wrote:
>>
>> Please review a test correction to the permissions granted to the test
>> and the check that the global filter cannot be re-set if once set.
>> Removed unnecessary code supplying a OIS when testing setting the global filter.
>>
>> Webrev:
>>    http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/
> Should the test fail when the test does not have java.io.SerializablePermission(?serialFilter?) permission?  Has the test been passing with this typo?
>   
> One more typo in FilterWithSecurityManagerTest.java
>    75                     + "java.security.SerializablePermission(serialFilter) ?
>
> Mandy
>
>


From mandy.chung at oracle.com  Sun Oct 23 01:20:32 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 22 Oct 2016 18:20:32 -0700
Subject: RFR 9: 8167192 [Testbug] : java/io/Serializable/serialFilter test
	conditions wrong
In-Reply-To: <17825685-5128-1e22-64ef-910d22c39636@Oracle.com>
References: <06edcb78-e83c-cec0-d28a-6279574c5c77@Oracle.com>
	<A11C9212-4065-4A8D-8DB5-55FC8369CCD9@oracle.com>
	<17825685-5128-1e22-64ef-910d22c39636@Oracle.com>
Message-ID: <71A56A7D-4A0A-4233-AD63-3FC15B33CC76@oracle.com>


> On Oct 22, 2016, at 9:43 AM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Hi Mandy,
> 
> Thanks for catching that typo in the error message.
> The fix of the permission in the security.policy exposed bugs in the tests that are being addressed.
> 
> Webrev updated in place.
>    http://cr.openjdk.java.net/~rriggs/webrev-filtertest-8167192/
> 

+1

Mandy


From brunoaiss at gmail.com  Sun Oct 23 18:14:05 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Sun, 23 Oct 2016 19:14:05 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
Message-ID: <00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>

Here's my poc/prototype:
http://pastebin.com/WRpYWDJF

I've implemented the bare minimum of the class that follows the same 
contract of BufferedReader while signaling all issues I think it may 
have or has in comments.
I also wrote some javadoc to help guiding through the class.

I could have used more fields from BufferedReader but the names were so 
minimalistic that were confusing me. I intent to change them before 
sending this to openJDK.

One of the major problems this has is long overflowing. It is major 
because it is hidden, it will be extremely rare and it takes a really 
long time to reproduce. There are different ways of dealing with it. 
 From just documenting to actually making code that works with it.

I built a simple test code for it to have some ideas about performance 
and correctness.

http://pastebin.com/eh6LFgwT

This doesn't do a through test if it is actually working correctly but I 
see no reason for it not working correctly after fixing the 2 bugs that 
test found.

I'll also leave here some conclusions about speed and resource 
consumption I found.

I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I 
noticed that, with my hardware, with the 1 530 000 000B file, I was 
getting around:

In all buffers and fake work: 10~15s speed improvement ( from 90% HDD 
speed to 100% HDD speed)
In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD 
speed to 100% HDD speed)

Changing the buffer size was giving different reading speeds but both 
were quite equal in how much they would change when changing the buffer 
size.
Finally, I could always confirm that I/O was always the slowest thing 
while this code was running.

For the ones wondering about the file size; it is both to avoid OS cache 
and to make the reading at the main use-case these objects are for 
(large streams of bytes).

@Pavel, are you open for discussion now ;)? Need anything else?

On 21/10/2016 19:21, Pavel Rappo wrote:
> Just to append to my previous email. BufferedReader wraps any Reader out there.
> Not specifically FileReader. While you're talking about the case of effective
> reading from a file.
>
> I guess there's one existing possibility to provide exactly what you need (as I
> understand it) under this method:
>
> /**
>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>   * from the file in an efficient manner...
>     ...
>   */
> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>
> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
> needs to be investigated not only for your favorite OS but for other OSes as
> well. Feel free to prototype this and we can discuss it on the list later.
>
> Thanks,
> -Pavel
>
>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Pavel is right.
>>
>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>
>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>
>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>
>>
>> On 21/10/2016 18:38, Roger Riggs wrote:
>>> Hi Pavel,
>>>
>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>> and managing the swaps and async reads transparently.
>>> It would not change the API but would change the interactions between the buffered reader
>>> and the underlying stream.  It would also increase memory requirements and processing
>>> by introducing or using a separate thread and the necessary synchronization.
>>>
>>> Though I think the formal interface semantics could be maintained, I have doubts
>>> about compatibility and its unintended consequences on existing subclasses,
>>> applications and libraries.
>>>
>>> $.02, Roger
>>>
>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>> streams from java.io were designed for simple synchronous use case.
>>>>
>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>> in your applications?
>>>>
>>>> -Pavel
>>>>
>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>
>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>
>>>>> That's why I'm asking this here.
>>>>>
>>>>>
>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>
>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>
>>>
>


From ramanand.patil at oracle.com  Mon Oct 24 07:28:08 2016
From: ramanand.patil at oracle.com (Ramanand Patil)
Date: Mon, 24 Oct 2016 00:28:08 -0700 (PDT)
Subject: RFR: 8168512: (tz) Support tzdata2016h
Message-ID: <fcc63912-310e-4e52-b065-6592a04f7dd4@default>

Hi all,

Please review the latest TZDATA integration (tzdata2016h) to JDK9.

Bug: https://bugs.openjdk.java.net/browse/JDK-8168512

Webrev: http://cr.openjdk.java.net/~rpatil/8168512/webrev.00/

 

All the TimeZone related tests are passed after integration.

 

Regards,

Ramanand.

 

From amy.lu at oracle.com  Mon Oct 24 08:08:19 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Mon, 24 Oct 2016 16:08:19 +0800
Subject: JDK 9 RFR of JDK-8168524: Remove two jdk_nio tests from ProblemList:
	BashStreams and DeleteInterference.java
Message-ID: <aaac152a-4b2a-4d1b-e05b-046a8b87202d@oracle.com>

Please review the patch to bring back two tests:

java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
     More debug info has been added to this test (JDK-8162902). It's 
hard to reproduce the failure. Let's bring back this test. Once test 
failure happen again, the debug info could help with JDK-8156511 fix.

java/nio/charset/coders/BashStreams.java 8149712 generic-all
     Mentioned issue seems does not exist anymore, suggest to bring back 
the test.

bug: https://bugs.openjdk.java.net/browse/JDK-8168524

Thanks,
Amy

--- old/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
+++ new/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
@@ -186,10 +186,6 @@
  java/nio/file/WatchService/MayFlies.java 7158947 solaris-all Solaris 11
  java/nio/file/WatchService/LotsOfEvents.java 7158947 solaris-all 
Solaris 11

-java/nio/charset/coders/BashStreams.java 8149712 generic-all
-
-java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
-
  ############################################################################

  # jdk_rmi



From pavel.rappo at oracle.com  Mon Oct 24 12:48:21 2016
From: pavel.rappo at oracle.com (Pavel Rappo)
Date: Mon, 24 Oct 2016 13:48:21 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
Message-ID: <8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>

Could you please send a new email on this list with the source attached as a
text file?

> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com> wrote:
> 
> Here's my poc/prototype:
> http://pastebin.com/WRpYWDJF
> 
> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
> I also wrote some javadoc to help guiding through the class.
> 
> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
> 
> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
> 
> I built a simple test code for it to have some ideas about performance and correctness.
> 
> http://pastebin.com/eh6LFgwT
> 
> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
> 
> I'll also leave here some conclusions about speed and resource consumption I found.
> 
> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
> 
> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
> 
> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
> 
> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
> 
> @Pavel, are you open for discussion now ;)? Need anything else?
> 
> On 21/10/2016 19:21, Pavel Rappo wrote:
>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>> Not specifically FileReader. While you're talking about the case of effective
>> reading from a file.
>> 
>> I guess there's one existing possibility to provide exactly what you need (as I
>> understand it) under this method:
>> 
>> /**
>>  * Opens a file for reading, returning a {@code BufferedReader} to read text
>>  * from the file in an efficient manner...
>>    ...
>>  */
>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>> 
>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>> needs to be investigated not only for your favorite OS but for other OSes as
>> well. Feel free to prototype this and we can discuss it on the list later.
>> 
>> Thanks,
>> -Pavel
>> 
>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>> 
>>> Pavel is right.
>>> 
>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>> 
>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>> 
>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>> 
>>> 
>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>> Hi Pavel,
>>>> 
>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>> and managing the swaps and async reads transparently.
>>>> It would not change the API but would change the interactions between the buffered reader
>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>> by introducing or using a separate thread and the necessary synchronization.
>>>> 
>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>> about compatibility and its unintended consequences on existing subclasses,
>>>> applications and libraries.
>>>> 
>>>> $.02, Roger
>>>> 
>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>> streams from java.io were designed for simple synchronous use case.
>>>>> 
>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>> in your applications?
>>>>> 
>>>>> -Pavel
>>>>> 
>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>> 
>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>> 
>>>>>> That's why I'm asking this here.
>>>>>> 
>>>>>> 
>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>> 
>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>> 
>>>> 
>> 
> 


From martinrb at google.com  Mon Oct 24 14:14:38 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 24 Oct 2016 07:14:38 -0700
Subject: RFR: 8168512: (tz) Support tzdata2016h
In-Reply-To: <fcc63912-310e-4e52-b065-6592a04f7dd4@default>
References: <fcc63912-310e-4e52-b065-6592a04f7dd4@default>
Message-ID: <CA+kOe08ZM2YXEY8cZB=Xru9FTVOghvE7dyC8QQ=4P-ki+mARsA@mail.gmail.com>

Looks good to me!

On Mon, Oct 24, 2016 at 12:28 AM, Ramanand Patil <ramanand.patil at oracle.com>
wrote:

> Hi all,
>
> Please review the latest TZDATA integration (tzdata2016h) to JDK9.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8168512
>
> Webrev: http://cr.openjdk.java.net/~rpatil/8168512/webrev.00/
>
>
>
> All the TimeZone related tests are passed after integration.
>
>
>
> Regards,
>
> Ramanand.
>
>
>

From brian.burkhalter at oracle.com  Mon Oct 24 14:50:15 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Mon, 24 Oct 2016 07:50:15 -0700
Subject: JDK 9 RFR of JDK-8168524: Remove two jdk_nio tests from
	ProblemList: BashStreams and DeleteInterference.java
In-Reply-To: <aaac152a-4b2a-4d1b-e05b-046a8b87202d@oracle.com>
References: <aaac152a-4b2a-4d1b-e05b-046a8b87202d@oracle.com>
Message-ID: <FF5D6A00-DC71-4D85-AFC7-40A3D02F91C5@oracle.com>

+1 for the DeleteInterference portion.

Brian

On Oct 24, 2016, at 1:08 AM, Amy Lu <amy.lu at oracle.com> wrote:

> Please review the patch to bring back two tests:
> 
> java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
>    More debug info has been added to this test (JDK-8162902). It's hard to reproduce the failure. Let's bring back this test. Once test failure happen again, the debug info could help with JDK-8156511 fix.
> 
> java/nio/charset/coders/BashStreams.java 8149712 generic-all
>    Mentioned issue seems does not exist anymore, suggest to bring back the test.
> 
> bug: https://bugs.openjdk.java.net/browse/JDK-8168524
> 
> Thanks,
> Amy
> 
> --- old/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
> +++ new/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
> @@ -186,10 +186,6 @@
> java/nio/file/WatchService/MayFlies.java 7158947 solaris-all Solaris 11
> java/nio/file/WatchService/LotsOfEvents.java 7158947 solaris-all Solaris 11
> 
> -java/nio/charset/coders/BashStreams.java 8149712 generic-all
> -
> -java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
> -
> ############################################################################
> 
> # jdk_rmi
> 
> 


From brunoaiss at gmail.com  Mon Oct 24 14:59:38 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Mon, 24 Oct 2016 15:59:38 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
Message-ID: <48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>

Attached and sending!


On 24/10/2016 13:48, Pavel Rappo wrote:
> Could you please send a new email on this list with the source attached as a
> text file?
>
>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Here's my poc/prototype:
>> http://pastebin.com/WRpYWDJF
>>
>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>> I also wrote some javadoc to help guiding through the class.
>>
>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>
>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>
>> I built a simple test code for it to have some ideas about performance and correctness.
>>
>> http://pastebin.com/eh6LFgwT
>>
>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>
>> I'll also leave here some conclusions about speed and resource consumption I found.
>>
>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>
>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>
>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>
>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>
>> @Pavel, are you open for discussion now ;)? Need anything else?
>>
>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>> Not specifically FileReader. While you're talking about the case of effective
>>> reading from a file.
>>>
>>> I guess there's one existing possibility to provide exactly what you need (as I
>>> understand it) under this method:
>>>
>>> /**
>>>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>   * from the file in an efficient manner...
>>>     ...
>>>   */
>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>
>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>> needs to be investigated not only for your favorite OS but for other OSes as
>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>
>>> Thanks,
>>> -Pavel
>>>
>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>>>
>>>> Pavel is right.
>>>>
>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>
>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>
>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>
>>>>
>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>> Hi Pavel,
>>>>>
>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>> and managing the swaps and async reads transparently.
>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>
>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>> applications and libraries.
>>>>>
>>>>> $.02, Roger
>>>>>
>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>
>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>> in your applications?
>>>>>>
>>>>>> -Pavel
>>>>>>
>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>>>
>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>
>>>>>>> That's why I'm asking this here.
>>>>>>>
>>>>>>>
>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>
>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>
>

-------------- next part --------------
package pocs.java;

import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;
import java.util.Objects;

public class BufferedNonBlockStream extends BufferedInputStream {

	private static final int DEFAULT_BUFFER_SIZE = 8_192;
	
	/**
     * The maximum size of array to allocate.
     * Some VMs reserve some header words in an array.
     * Attempts to allocate larger arrays may result in
     * OutOfMemoryError: Requested array size exceeds VM limit
     */
    private static int MAX_BUFFER_SIZE = Integer.MAX_VALUE - 8;

	private static final int MIN_READ_BUFFER_SIZE = 10;
	/**
     * The maximum size of the read buffer if set automatically.
     * Unless explicitly told, there's no need to starve the VM out of memory
     */
	private static final int MAX_READ_BUFFER_SIZE = 1_000_000;
	
	/**
     * The virtual index of the next position where data will be read.
     * Every operation requires executing a mathematical mod ({@code %}) operation on it against <code>buf.length</code>
     * to get the correct buffer position to work on.
     */
    protected long readPos;
    
    /**
     * The virtual index of the next position where data will be written
     * Every operation requires executing a mathematical mod ({@code %}) operation on it against <code>buf.length</code>
     * to get the correct buffer position to work on.
     * 
     * At every time, {@code readPos < writePos %} must be true
     * 
     */
    // Note to reviewers: What to do when long overflows? It will happen but does a documented warning suffice or... do I need to work with the overflows?
    protected long writePos;
	
	ReadableByteChannel inputChannel;
	ByteBuffer readBuffer;
	
	boolean isClosed;
	boolean eof;

	public BufferedNonBlockStream(InputStream inputStream) {
		super(inputStream);
	}
	
	public BufferedNonBlockStream(ReadableByteChannel inputChannel) {
		this(inputChannel, DEFAULT_BUFFER_SIZE);
	}
	
	public BufferedNonBlockStream(ReadableByteChannel inputChannel, int bufferSize) {
		this(inputChannel, bufferSize, Math.max(Math.min(bufferSize / 4, MAX_READ_BUFFER_SIZE), MIN_READ_BUFFER_SIZE));
	}
	
	public BufferedNonBlockStream(ReadableByteChannel inputChannel, int bufferSize, int readBufferSize) {
		super(null, bufferSize);
		Objects.requireNonNull(inputChannel, "The byte channel must not be null");
		this.inputChannel = inputChannel;
		
		if(readBufferSize < 1){
			throw new IllegalArgumentException("Read buffer must be at least 1 byte");
		}
		
		if(readBufferSize > bufferSize){
			throw new IllegalArgumentException("ReadBufferSize must be smaller than bufferSize");
		}
		
		// The read buffer must not be larger than the internal buffer.
		readBufferSize = Math.min(readBufferSize, MAX_BUFFER_SIZE - 1);
		this.readBuffer = ByteBuffer.allocateDirect(readBufferSize);
		isClosed = false;
	}
	
	int readSpaceAvailable(){
		// This can be an int because buf size can never be larger than the int
		return (int) (writePos - readPos);
	}
	int writeSpaceAvailable(){
		return buf.length - readSpaceAvailable();
	}
	
	void fill() throws IOException {
		fill(false);
	}
    void blockingFill(boolean forced) throws IOException {
		fill(forced);
		while(readPos == writePos){
			try {
				Thread.sleep(100);
			} catch (InterruptedException e) {
				// An interrupt may mean more data is available
			}
			fill(forced);
    	}
    }
    
    synchronized void fill(boolean forced) throws IOException {
    	isClosed = !inputChannel.isOpen();
        
        eof = inputChannel.read(readBuffer) == -1;
        
        // For the poc, markers are not coded
        
        long freeBufSpace = writeSpaceAvailable();
        
        if(	(forced && readBuffer.position() > 0) ||
    		// It's no use reading a very small quantity from the buffer
            // Please review that.
        	(freeBufSpace > readBuffer.capacity() / 2 && freeBufSpace >= readBuffer.position())
        ){
        	readBuffer.flip();
            // Ints are faster and an array can only be defined using an int
            int writeBufferFrom = (int) (writePos % buf.length);
            int canReadAmount =  Math.min((int) freeBufSpace, readBuffer.remaining());
            int canWriteUntil = (writeBufferFrom + canReadAmount) % buf.length;
            
        	// This can only be done like this because ReadBufferSize is smaller than bufferSize
            // and overflows are ignored
        	if(canWriteUntil < writeBufferFrom){
        		// Read in 2 parts
        		readBuffer.get(buf, writeBufferFrom, buf.length - writeBufferFrom);
        		readBuffer.get(buf, 0, canWriteUntil);
        	} else {
        		readBuffer.get(buf, writeBufferFrom, canWriteUntil - writeBufferFrom);
        	}

        	writePos += canReadAmount;
        	// Reset the buffer for more reading
        	readBuffer.clear();
        }
        
    }
	
	@Override
	public int read() throws IOException {
		if(inputChannel == null){
			return super.read();			
		}
		fill(false);
		if(readPos == writePos){
			// Get anything that is in the buffer
			blockingFill(true);
		}
		
		return buf[(int)(readPos++ % buf.length)] & 0xff;
	}

	@Override
	public int read(byte[] b, int off, int len) throws IOException {
		if(inputChannel == null){
			return super.read(b, off, len);
		}
		// Always fill. Even when rejecting
		fill();
		if(eof && readSpaceAvailable() == 0){
			// Buffer emptied and the stream ended
			return -1;
		}
		if ((off | len | (off + len) | (b.length - (off + len))) < 0) {
            throw new IndexOutOfBoundsException();
        } else if (len == 0) {
            return 0;
        }
		
		int bytesRead = 0;
		
		// For a ReadableChannel, assume that it has always read as much as it could
		if(readSpaceAvailable() == 0){
			fill(true);
			if(readSpaceAvailable() == 0){
				return eof ? -1 : 0;
			}
		}

		// Note for reviewing: Some optimizations that take some time were not made here.
		// E.g.
		// -> If b consumes the whole buff, copy directly from readBuffer to this buff
		// -> As long as reading buffs are large enough (requires benchmarking), 
		// 	  not do use the buf array at all (late initialization too)

        int readBufferFrom = (int) (readPos % buf.length);
        int canReadAmount =  Math.min((int) readSpaceAvailable(), len - off);
        int canReadBufferUntil = (readBufferFrom + canReadAmount) % buf.length;
        
		if(canReadBufferUntil < readBufferFrom){
    		// For reviewer: Should it always read once? Or should it read twice?
			System.arraycopy(buf, readBufferFrom, b, off, buf.length - readBufferFrom);
			int ndStartAt = off + (buf.length - readBufferFrom);
			// This if should never evaluate "true". It's here just to make sure and make breakpoints
			if(off - ndStartAt >= len){
				readPos = canReadBufferUntil;
				return off - ndStartAt; 
			}
			System.arraycopy(buf, 0, b, ndStartAt, canReadBufferUntil);
			readPos += (buf.length - readBufferFrom) + canReadBufferUntil;
			return (buf.length - readBufferFrom) + canReadBufferUntil;
    	} else {
			System.arraycopy(buf, readBufferFrom, b, off, canReadBufferUntil - readBufferFrom);
			readPos += canReadBufferUntil - readBufferFrom;
			return canReadBufferUntil - readBufferFrom;
    	}
		
		
	}

	@Override
	public long skip(long arg0) throws IOException {
		if(inputChannel == null){
			return super.skip(arg0);
		}
		
		// There's no skip in this poc
		// For the real implementation, I'd do (while I didn't skip enough):
		// I'd just "jump" the readPos accordingly (while keeping readPos <= writePos)
		// Then 
		// reposition (.position(int)) in buffer
		// OR
		// clear the buffer
		// Then
		// If SeekableChannel -> use (sord of) .position(.position()+ skipLeft)
		// else -> do not seek.
		
		
		throw new UnsupportedOperationException();
		
	}

	@Override
	public int available() throws IOException {
		if(inputChannel == null){
			return super.available();
		}
		fill();
		// For now, just return what I am sure I have
		return readSpaceAvailable();
		
	}

	@Override
	public void mark(int arg0) {
		if(inputChannel == null){
			super.mark(arg0);
		}
		
		// This is not hard to do but it requires quite some time!
		// I'll wait first if the poc passes without this method
	}

	@Override
	public void reset() throws IOException {
		if(inputChannel == null){
			super.reset();
		}
		throw new IOException("The mark was lost");
	}

	@Override
	public boolean markSupported() {
		if(inputChannel == null){
			return super.markSupported();
		}
		// false for now, at least.
		return false;
	}

	@Override
	public void close() throws IOException {
		if(inputChannel != null){
			inputChannel.close();
		}
		super.close();
	}

}
-------------- next part --------------
package pocs.java;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.channels.FileChannel;
import java.nio.file.Paths;
import java.util.zip.CRC32;

public class Tests {
	
	private static final int PROGRAM_BUFFER = 200;
	private static final int READ_BUFFER = 500_000;
	private static final boolean EXECUTE_WORK = true;

	static void buildTestFile() throws Exception{

		BufferedOutputStream outputing = new BufferedOutputStream(new FileOutputStream("TestFile"), 30000);
		
		for (int i = 0; i < 6_000_000; i++) {
			for (int j = 0; j < 255; j++) {
				outputing.write(j);
			}
		}
		
		outputing.close();
	}
	
	static void theRead(BufferedInputStream inputing) throws Exception{
		byte[] myBuf = new byte[PROGRAM_BUFFER];
		int readAmount = 0;
		byte expectingByte = 0x00;
		int strikes = 0;
		
		int readCount = 0;
		
		while((readAmount = inputing.read(myBuf)) != -1){
			readCount++;
			for (int i = 0; i < readAmount; i++) {
				if(expectingByte == -1){
					expectingByte = 0;
				}
				if(expectingByte != myBuf[i]){
					System.out.println("For byte " + (int) myBuf[i] + " expected " + (int) expectingByte );
					if(strikes++ > 50){
						return;
					}
				}
				expectingByte += 1;
			}
			if(EXECUTE_WORK){
				// doing work
				for (int i = 0; i < 30; i++) {
					new CRC32().update(myBuf);	
				}
			}
		}
	}
	
	static void normalRead() throws Exception {

		try(BufferedInputStream inputing = new BufferedInputStream(new FileInputStream("TestFile"), READ_BUFFER);){
			theRead(inputing);
		}
	}
	

	static void myRead() throws Exception {

		try(BufferedInputStream inputing = new BufferedNonBlockStream(
					FileChannel.open(new File("TestFile").toPath()), READ_BUFFER
				);
			){
			theRead(inputing);
		}
	}
	
	public static void main(String[] args) throws Exception {

//		buildTestFile();
		long start = System.currentTimeMillis();
//		normalRead();
		myRead();
		System.out.println("Time! " + (System.currentTimeMillis() - start));
		
		
	}
}


From chris.hegarty at oracle.com  Mon Oct 24 15:26:58 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 24 Oct 2016 16:26:58 +0100
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
Message-ID: <5df76ef9-15d3-d062-db7a-2aa0c34b2052@oracle.com>

On 19/10/16 19:34, Steve Drach wrote:
> Hi,
>
> Please review the following changeset.  This fix allows jar tool to add a new public class in a versioned directory in a modular multi-release jar file if the class is in a concealed package. When this event takes place, a warning message is emitted saying that placing this jar on a class path will result in an incompatible public interface, the jar file does not meet the single api rule of a multi-release jar file.  As before, adding a new public class in a versioned directory of a multi-release jar file (not a modular jar file) will result in an error.
>
> issue: https://bugs.openjdk.java.net/browse/JDK-8164805 <https://bugs.openjdk.java.net/browse/JDK-8164805>
>
> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ <http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/>

Thanks for fixing this Steve. The changes look ok to me.

-Chris.

From steve.drach at oracle.com  Mon Oct 24 17:28:55 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Mon, 24 Oct 2016 10:28:55 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
Message-ID: <C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>

There is a new webrev at http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/ <http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/> that addresses the issues raised by Mandy.  Comments inline.

>> issue: https://bugs.openjdk.java.net/browse/JDK-8164805
>> webrev: http://cr.openjdk.java.net/~sdrach/8164805/webrev.00/ 
> 
> Issue a warning is good in the case public classes are added in a concealed package.  Some comments:
> 
> Main.java
> 
> addExtendedModuleAttributes does not seem to be the appropriate method to initialize concealedPackages.  It would be better to set concealedPackages in a separate method that will be called for each modular JAR.

Done, although I still don?t see an advantage to this.

> 
> Validator.java
> 160                     main.error(Main.formatMsg("error.validator.concealed.public.class", entryName));
> 
> This is a warning.  You should add a new warn method to make it clear.
> Similiarly,  ?error.validator.concealed.public.class? should be renamed to ?warn.validator.concealed.public.class?.  I notice there are several keys with ?error.validator? prefix that are meant for warnings.  It?d be good to  change them too.  

All done.

> 
> 247         if (idx > -1) {
> 248             String pkgName = internalName.substring(0, idx).replace('/', '.');
> 249             if (main.concealedPackages.contains(pkgName)) {
> 250                 return true;
> 251             }
> 252         }
> 
> Alternative impl for the above lines:
> 
> String pkgName = idx != -1 ? internalName.substring(0, idx).replace('/', '.?)
> 			   : ??;
> return main.concealedPackages.contains(pkgName);

Done.

> 
> jar.properties
> 
> 112 error.validator.concealed.public.class=\
> 113          warning - entry {0} is a public class in a concealed package, placing this \
> 114          jar on the class path will result in incompatible public interfaces
> 
> line 113 needs new line ?\n?.  You may break it into 3 lines
> since the entry name will take up some characters.

Done.

> 
> ConcealedPackage.java test
> 
>  60         Path destination = userdir.resolve("classes?);
> 
> I suggest to use Paths.get(?classes?) rather than ${user.dir}.  jtreg will clean up the JTwork/scratch directory after the test run.

Doen, although I still clean up after tests.

> 
>  63         // add an empty directory
>  64         Files.createDirectory(destination.resolve("p").resolve("internal"));
> 
> I suggest to take this out.  The test verifies if "jar tf mmr.jar? succeeds.

Done.

> 
> The test should test both ?cf? and ?uf? and verify that the ConcealedPackage attributes in module-info.class is updated correctly (one single module-info.class case in the root entry and two module-info.class - root and versioned).

Done, but perhaps not exactly the suggested way.  The test has been considerably beefed up.

> 
>  92     private int jar(String cmd) {
> Nit: this can simply take a varargs and no need to split:
>  jar(String? options)

I left it the way it was.

> 
> 
>  76     private String files(Path start) throws IOException {
> Perhaps return Stream<String>.  That can replace the unnecessary concat in a String and then later split to pass to javac.

Done.

Also fixed a bug I found Main::toPackageName



From ecki at zusammenkunft.net  Mon Oct 24 17:40:35 2016
From: ecki at zusammenkunft.net (Bernd)
Date: Mon, 24 Oct 2016 19:40:35 +0200
Subject: XMLReader secure processing
Message-ID: <CABOR3+wWyhGq6aVWCzqi31cEHWt9wsmbjLvWhFX1r3qfHDB1Cw@mail.gmail.com>

Hello,

I am somewhat lost on how to enable or control the secure processing in the
XMLReader.

You can use XMLConstants.FEATURE_SECURE_PROCESSING and/or
XMLConstants.ACCESS_EXTERNAL_{DTD,SCHEMA} only on the SAXParserFactory, but
not XMLReader(Factory).

Is this an oversight or am I missing something?

This seems to be a work around (at least for Oracle RI):

    SAXParserFactory spf = SAXParserFactory.newInstance();
    spf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);

    SAXParser parser = spf.newSAXParser();
    System.out.println("external dtd: " +
parser.getProperty(XMLConstants.ACCESS_EXTERNAL_DTD));
    XMLReader reader = parser.getXMLReader();

In this case the protocols is "all" for FSP=false and "" for FSP=true.

XMLConstants Javadoc does not talk about XMLReader, hm.


BTW: while investigating I noticed the changed default for secure
processing is not reflected by the comment:

com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl.java
     /**
L64* State of the secure processing feature, initially <code>false</code>
     */
    private boolean fSecureProcess = true;


http://hg.openjdk.java.net/jdk9/jdk9/jaxp/file/6d980e959726/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/jaxp/SAXParserFactoryImpl.java#l64


Gruss
Bernd

From paul.sandoz at oracle.com  Mon Oct 24 18:37:14 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 24 Oct 2016 11:37:14 -0700
Subject: RFR: 8160954: (spec) Runtime.Version regex and $PRE/$OPT issues
In-Reply-To: <25c9fcb4-685e-4927-83aa-2039709d0e0c@default>
References: <25c9fcb4-685e-4927-83aa-2039709d0e0c@default>
Message-ID: <D2BF3603-DBF0-4722-9064-1629637AF1EC@oracle.com>


> On 20 Oct 2016, at 20:34, Iris Clark <iris.clark at oracle.com> wrote:
> 
> Hi.
> 
> Please review changes to address the following closely related bugs:
> 
>    8160954: (spec) Runtime.Version regex and $PRE/$OPT issues
>    https://bugs.openjdk.java.net/browse/JDK-8160954
> 
>    8148822: (spec) Regex in Runtime.Version and JEP 223 should match
>    https://bugs.openjdk.java.net/browse/JDK-8148822
> 
>    8148877: (spec) Specify when and empty '+' is required in a version
>        string
>    https://bugs.openjdk.java.net/browse/JDK-8148877
> 
> webrev:
> 
>    http://cr.openjdk.java.net/~iris/verona/8160954/webrev/
> 
> Changes to the specification of the $VNUM and $OPT regexp:
> 
> For line 963, we remove the leading '^' and trailing '$'.  These characters
> are not strictly required for the regex to be accurate.  They're not included
> in the implementation.  The presence of the trailing '$' precludes direct
> substitution of this regex into the definition of $VSTR at line 1032 (see
> bug 8160954).
> 
> Also for line 963, we remove the outermost qualifier in the portion of the
> regex describing elements after $MAJOR.  This qualifier is redundant and a
> source of catastrophic backtracking as described in this message [0, item 1]
> (see bug 8148822).  It's not in the present implementation.
> 
> For line 1050, we remove the '\' as it is unnecessary in a character class
> [0, item 4] (see bug 8148822).
> 
> 
> Changes to the spec for $PRE/$OPT:
> 
> For lines 1056-1057, we clearly specify when an empty '+' is required (see
> bugs 8148877, 8160954).  This spec corresponds to the code for additional
> constraints for the "empty '+'" beginning at new line 1150.
> 

Looks good except for:

-     * <p> A version number {@code 10-ea} matches {@code $VNUM = "10"} and
+     * <p> If the build number is unset, then {@code '+'} may only be present
+     * when {@code $OPT} is present and {@code $PRE} is not.
+     * A version number {@code 10-ea} matches {@code $VNUM = "10"} and
      * {@code $PRE = "ea"}.  The version number {@code 10+-ea} matches
      * {@code $VNUM = "10"} and {@code $OPT = "ea"}. </p>


?unset? or ?not present??

It might be better to place that new text at the end as it dives right in at the deep end. Perhaps say at the end:

  More specifically, if the build number is not present, then...

Paul.


> 
> The required CCC is in progress.
> 
> Thanks,
> iris
> 
> [0] http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038036.html
> [1] http://download.java.net/java/jdk9/docs/api/java/lang/Runtime.Version.html


From mandy.chung at oracle.com  Mon Oct 24 18:59:13 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 24 Oct 2016 11:59:13 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>
Message-ID: <6224B8CA-7D21-433F-AC85-6649A2DF4B34@oracle.com>


> On Oct 24, 2016, at 10:28 AM, Steve Drach <steve.drach at oracle.com> wrote:
> 
> There is a new webrev at http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/ 

sun/tools/jar/Main.java

Thanks for refactoring and adding the findConcealedPackages method.  What I actually meant was to move out this line:
    concealedPackages = findConcealedPackages(rd);

to probably before calling addExtendedModuleAttributes(moduleInfos) above line 342 and 1101.

2014                 .filter(p -> !p.equals("?))

For a modular JAR, there should be no unnamed package.  I think the jar tool should fail if it detects an unnamed package.  Your test does not have any unnamed package - how did you find this?

ConcealedPackage.java test

Thanks for improving the test.  It?d be good to name the @Test method with a descriptive method name e.g. 
   test1 -> testUpdateVersionedPublicClass
   test2 -> testUpdatedVersionedPublicConcealedClass

 117     @Test // updates a valid multi-release jar with a new public class in
 118           // versioned section and fails

Nit: You can consider moving the comment above @Test.

Mandy



From martinrb at google.com  Mon Oct 24 19:32:40 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 24 Oct 2016 12:32:40 -0700
Subject: Iterator.forEachRemaining and Iterator.remove
Message-ID: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>

It doesn't make a lot of sense for users to call Iterator.remove after
calling Iterator.forEachRemaining.

The default implementation has no choice but to do

        while (hasNext())
            action.accept(next());

upon which remove() removes the last element.  What should overriding
implementations do?  Emulate the default implementation's behavior or
throw IllegalStateException, or even remove the element returned by the
last actual call to next?  The spec is (perhaps intentionally) unclear on
this point.  I'm thinking we emulate the default implementation's behavior,
because common collections like ArrayList work this way, and some users may
be depending on it, perhaps unwisely.

From steve.drach at oracle.com  Mon Oct 24 20:39:58 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Mon, 24 Oct 2016 13:39:58 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <6224B8CA-7D21-433F-AC85-6649A2DF4B34@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>
	<6224B8CA-7D21-433F-AC85-6649A2DF4B34@oracle.com>
Message-ID: <5AA30A0E-4207-4ACA-AFCC-F93840809A31@oracle.com>

>> There is a new webrev at http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/ 
> 
> sun/tools/jar/Main.java
> 
> Thanks for refactoring and adding the findConcealedPackages method.  What I actually meant was to move out this line:
>    concealedPackages = findConcealedPackages(rd);
> 
> to probably before calling addExtendedModuleAttributes(moduleInfos) above line 342 and 1101.

I tried that and decided it was non-optimal because we?d have to construct the ModuleDescriptor from the modInfos twice in succession.  Let's compromise here, okay?

> 
> 2014                 .filter(p -> !p.equals("?))


> 
> For a modular JAR, there should be no unnamed package.  I think the jar tool should fail if it detects an unnamed package.  Your test does not have any unnamed package - how did you find this?

the modularJar/Basic test found a bug.  Then when I was fixing the bug in toPackageNames I noticed it could return unnamed packages (??).  And in fact there are some, a few, classes that aren?t in a package.  Jar tool shouldn?t fail with unnamed packages.  They could even exist in a modular multi-release jar when the module-info class is only in a versioned directory.  I guess it should fail if a class in an unnamed package is in a module, although I?m not even sure about that.

> 
> ConcealedPackage.java test
> 
> Thanks for improving the test.  It?d be good to name the @Test method with a descriptive method name e.g. 
>   test1 -> testUpdateVersionedPublicClass
>   test2 -> testUpdatedVersionedPublicConcealedClass

It?s difficult to come up with names that aren?t sentences.  I figured the comments would explain the test adequately.

> 
> 117     @Test // updates a valid multi-release jar with a new public class in
> 118           // versioned section and fails
> 
> Nit: You can consider moving the comment above @Test.

Ok


From huizhe.wang at oracle.com  Mon Oct 24 20:51:24 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Mon, 24 Oct 2016 13:51:24 -0700
Subject: XMLReader secure processing
In-Reply-To: <CABOR3+wWyhGq6aVWCzqi31cEHWt9wsmbjLvWhFX1r3qfHDB1Cw@mail.gmail.com>
References: <CABOR3+wWyhGq6aVWCzqi31cEHWt9wsmbjLvWhFX1r3qfHDB1Cw@mail.gmail.com>
Message-ID: <580E744C.1020201@oracle.com>



On 10/24/16, 10:40 AM, Bernd wrote:
> Hello,
>
> I am somewhat lost on how to enable or control the secure processing in the
> XMLReader.
>
> You can use XMLConstants.FEATURE_SECURE_PROCESSING and/or
> XMLConstants.ACCESS_EXTERNAL_{DTD,SCHEMA} only on the SAXParserFactory, but
> not XMLReader(Factory).
>
> Is this an oversight or am I missing something?

It's not part of the JAXP API (javax.xml.*), but was included as 
endorsed. I assume XMLReaderFactory was included in the JAXP API package 
for compatibility since it existed before JAXP API was created, and that 
was probably why it didn't have the same support in terms of features.

Note that, in JDK 9, XMLReaderFactory is deprecated. Users are 
encouraged to use SAXParserFactory instead.

>
> This seems to be a work around (at least for Oracle RI):

This is therefore not a "work-around", but the supported API.
>
>      SAXParserFactory spf = SAXParserFactory.newInstance();
>      spf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
>
>      SAXParser parser = spf.newSAXParser();
>      System.out.println("external dtd: " +
> parser.getProperty(XMLConstants.ACCESS_EXTERNAL_DTD));
>      XMLReader reader = parser.getXMLReader();
>
> In this case the protocols is "all" for FSP=false and "" for FSP=true.

ACCESS_* is "all" by default, even though FSP is true. They will be set 
to "" or not allow external access if FSP is explicitly set.
>
> XMLConstants Javadoc does not talk about XMLReader, hm.
>
>
> BTW: while investigating I noticed the changed default for secure
> processing is not reflected by the comment:
>
> com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl.java
>       /**
> L64* State of the secure processing feature, initially<code>false</code>
>       */
>      private boolean fSecureProcess = true;
>
>
> http://hg.openjdk.java.net/jdk9/jdk9/jaxp/file/6d980e959726/src/java.xml/share/classes/com/sun/org/apache/xerces/internal/jaxp/SAXParserFactoryImpl.java#l64

Thanks, will get that fixed in the next bug fix.

Best,
Joe

>
>
> Gruss
> Bernd

From Roger.Riggs at Oracle.com  Mon Oct 24 21:00:40 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 24 Oct 2016 17:00:40 -0400
Subject: RFR 9: [testbug] Disable CORBA
	com.sun.corba.serialization.ObjectStreamTest.echoObjects
Message-ID: <09a26cdf-9b3c-4b27-4a3d-27152d07cfac@Oracle.com>

Please review the disable of a problematic test bug due to address in 
use errors.
Only the ObjectStreamTest.echoObjects test needs to be disabled.
The ProblemList.txt is not used in the case.

webrev:
   http://cr.openjdk.java.net/~rriggs/webrev-echoobjects-8166814/

Issue:
  https://bugs.openjdk.java.net/browse/JDK-8168614

Thanks, Roger



From mandy.chung at oracle.com  Mon Oct 24 21:13:31 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 24 Oct 2016 14:13:31 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <5AA30A0E-4207-4ACA-AFCC-F93840809A31@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>
	<6224B8CA-7D21-433F-AC85-6649A2DF4B34@oracle.com>
	<5AA30A0E-4207-4ACA-AFCC-F93840809A31@oracle.com>
Message-ID: <3242273D-4DA4-4414-9101-F5DE0D564FE8@oracle.com>


> On Oct 24, 2016, at 1:39 PM, Steve Drach <steve.drach at oracle.com> wrote:
> 
>>> There is a new webrev at http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/ 
>> 
>> sun/tools/jar/Main.java
>> 
>> Thanks for refactoring and adding the findConcealedPackages method.  What I actually meant was to move out this line:
>>   concealedPackages = findConcealedPackages(rd);
>> 
>> to probably before calling addExtendedModuleAttributes(moduleInfos) above line 342 and 1101.
> 
> I tried that and decided it was non-optimal because we?d have to construct the ModuleDescriptor from the modInfos twice in succession.  Let's compromise here, okay?

OK.  It wasn?t clear to me that you considered that.  It?s fine to leave it for a future clean up.

> 
>> 
>> 2014                 .filter(p -> !p.equals("?))
> 
> 
>> 
>> For a modular JAR, there should be no unnamed package.  I think the jar tool should fail if it detects an unnamed package.  Your test does not have any unnamed package - how did you find this?
> 
> the modularJar/Basic test found a bug.  Then when I was fixing the bug in toPackageNames I noticed it could return unnamed packages (??).  And in fact there are some, a few, classes that aren?t in a package.  Jar tool shouldn?t fail with unnamed packages.

I meant for modular JAR.

> They could even exist in a modular multi-release jar when the module-info class is only in a versioned directory.  

module-info.class should be filtered before calling toPackageName.

> I guess it should fail if a class in an unnamed package is in a module, although I?m not even sure about that.


Mandy

From steve.drach at oracle.com  Mon Oct 24 21:35:40 2016
From: steve.drach at oracle.com (Steve Drach)
Date: Mon, 24 Oct 2016 14:35:40 -0700
Subject: RFR: 8164805: Fail to create a MR modular JAR with a versioned
	entry in base-versioned empty package
In-Reply-To: <3242273D-4DA4-4414-9101-F5DE0D564FE8@oracle.com>
References: <E1847EC9-84A6-4D3F-BED3-C4615C82459F@oracle.com>
	<CDBEA86C-ED25-4A09-B93E-07614F58FF4C@oracle.com>
	<C7E56FB8-5812-4F69-8EE4-400EF8A84009@oracle.com>
	<6224B8CA-7D21-433F-AC85-6649A2DF4B34@oracle.com>
	<5AA30A0E-4207-4ACA-AFCC-F93840809A31@oracle.com>
	<3242273D-4DA4-4414-9101-F5DE0D564FE8@oracle.com>
Message-ID: <618D152B-2CC3-4D63-9F0E-99E8F6894FEB@oracle.com>

>>>> There is a new webrev at http://cr.openjdk.java.net/~sdrach/8164805/webrev.01/ 
>>> 
>>> sun/tools/jar/Main.java
>>> 
>>> Thanks for refactoring and adding the findConcealedPackages method.  What I actually meant was to move out this line:
>>>  concealedPackages = findConcealedPackages(rd);
>>> 
>>> to probably before calling addExtendedModuleAttributes(moduleInfos) above line 342 and 1101.
>> 
>> I tried that and decided it was non-optimal because we?d have to construct the ModuleDescriptor from the modInfos twice in succession.  Let's compromise here, okay?
> 
> OK.  It wasn?t clear to me that you considered that.  It?s fine to leave it for a future clean up.
> 
>> 
>>> 
>>> 2014                 .filter(p -> !p.equals("?))
>> 
>> 
>>> 
>>> For a modular JAR, there should be no unnamed package.  I think the jar tool should fail if it detects an unnamed package.  Your test does not have any unnamed package - how did you find this?
>> 
>> the modularJar/Basic test found a bug.  Then when I was fixing the bug in toPackageNames I noticed it could return unnamed packages (??).  And in fact there are some, a few, classes that aren?t in a package.  Jar tool shouldn?t fail with unnamed packages.
> 
> I meant for modular JAR.
> 
>> They could even exist in a modular multi-release jar when the module-info class is only in a versioned directory.  
> 
> module-info.class should be filtered before calling toPackageName.

It is, although not in a stream.  My inspection of usages led me to an unused method, findPackages, that I will remove (I did so and all tests passed).

> 
>> I guess it should fail if a class in an unnamed package is in a module, although I?m not even sure about that.
> 
> 
> Mandy


From joe.darcy at oracle.com  Mon Oct 24 21:39:39 2016
From: joe.darcy at oracle.com (joe darcy)
Date: Mon, 24 Oct 2016 14:39:39 -0700
Subject: RFR 9: [testbug] Disable CORBA
	com.sun.corba.serialization.ObjectStreamTest.echoObjects
In-Reply-To: <09a26cdf-9b3c-4b27-4a3d-27152d07cfac@Oracle.com>
References: <09a26cdf-9b3c-4b27-4a3d-27152d07cfac@Oracle.com>
Message-ID: <ce8d9278-47ac-19b7-1172-77583abe59d9@oracle.com>

Looks fine Roger; thanks,

-Joe


On 10/24/2016 2:00 PM, Roger Riggs wrote:
> Please review the disable of a problematic test bug due to address in 
> use errors.
> Only the ObjectStreamTest.echoObjects test needs to be disabled.
> The ProblemList.txt is not used in the case.
>
> webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-echoobjects-8166814/
>
> Issue:
>  https://bugs.openjdk.java.net/browse/JDK-8168614
>
> Thanks, Roger
>
>


From masayoshi.okutsu at oracle.com  Tue Oct 25 00:40:42 2016
From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu)
Date: Tue, 25 Oct 2016 09:40:42 +0900
Subject: <i18n dev> RFR: 8168512: (tz) Support tzdata2016h
In-Reply-To: <CA+kOe08ZM2YXEY8cZB=Xru9FTVOghvE7dyC8QQ=4P-ki+mARsA@mail.gmail.com>
References: <fcc63912-310e-4e52-b065-6592a04f7dd4@default>
	<CA+kOe08ZM2YXEY8cZB=Xru9FTVOghvE7dyC8QQ=4P-ki+mARsA@mail.gmail.com>
Message-ID: <8573b95b-d879-cdcb-1468-3bc4ec5ef3eb@oracle.com>

+1

Masayoshi


On 10/24/2016 11:14 PM, Martin Buchholz wrote:
> Looks good to me!
>
> On Mon, Oct 24, 2016 at 12:28 AM, Ramanand Patil <ramanand.patil at oracle.com>
> wrote:
>
>> Hi all,
>>
>> Please review the latest TZDATA integration (tzdata2016h) to JDK9.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168512
>>
>> Webrev: http://cr.openjdk.java.net/~rpatil/8168512/webrev.00/
>>
>>
>>
>> All the TimeZone related tests are passed after integration.
>>
>>
>>
>> Regards,
>>
>> Ramanand.
>>
>>
>>


From kumar.x.srinivasan at oracle.com  Tue Oct 25 00:59:03 2016
From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan)
Date: Mon, 24 Oct 2016 17:59:03 -0700
Subject: RFR: 8160213: tools/pack200/Utils.java should clean up javac*.tmp
	files
Message-ID: <580EAE57.1040208@oracle.com>

Hello,

Please review fix for: https://bugs.openjdk.java.net/browse/JDK-8160213

Webrev: http://cr.openjdk.java.net/~ksrini/8160213/webrev.00/

Now the temporary javac argument file will be written to the working 
directory
of the test which will be purged upon completion.

The test for MultiRelease did not clean up after itself, added the cleanup
call, please note in the event of a failure, the files will be left 
behind for analysis.

Thanks

Kumar

From joe.darcy at oracle.com  Tue Oct 25 01:20:46 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 24 Oct 2016 18:20:46 -0700
Subject: RFR: 8160213: tools/pack200/Utils.java should clean up javac*.tmp
	files
In-Reply-To: <580EAE57.1040208@oracle.com>
References: <580EAE57.1040208@oracle.com>
Message-ID: <580EB36E.10002@oracle.com>

Looks fine Kumar; thanks,

-Joe

On 10/24/2016 5:59 PM, Kumar Srinivasan wrote:
> Hello,
>
> Please review fix for: https://bugs.openjdk.java.net/browse/JDK-8160213
>
> Webrev: http://cr.openjdk.java.net/~ksrini/8160213/webrev.00/
>
> Now the temporary javac argument file will be written to the working 
> directory
> of the test which will be purged upon completion.
>
> The test for MultiRelease did not clean up after itself, added the 
> cleanup
> call, please note in the event of a failure, the files will be left 
> behind for analysis.
>
> Thanks
>
> Kumar


From huaming.li at oracle.com  Tue Oct 25 01:22:39 2016
From: huaming.li at oracle.com (Hamlin Li)
Date: Tue, 25 Oct 2016 09:22:39 +0800
Subject: RFR of JDK 8168505: Remove the intermittent keyword from
	java/util/Arrays/ParallelPrefix.java
Message-ID: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>

bug: https://bugs.openjdk.java.net/browse/JDK-8168505

webrev: http://cr.openjdk.java.net/~mli/8168505/webrev.00/

Test ( java/util/Arrays/ParallelPrefix.java) was failing intermittently, 
related bug (JDK-8080165 
<https://bugs.openjdk.java.net/browse/JDK-8080165>) has been proved not 
reproduced. No open bug and no failure reported.
"@key intermittent" could be removed from test.

Thank you
-Hamlin


diff -r 5651fa4f1478 test/java/util/Arrays/ParallelPrefix.java
--- a/test/java/util/Arrays/ParallelPrefix.java    Sat Oct 22 17:03:17 
2016 +0300
+++ b/test/java/util/Arrays/ParallelPrefix.java    Mon Oct 24 18:15:44 
2016 -0700
@@ -26,7 +26,6 @@
   * @summary unit test for Arrays.ParallelPrefix().
   * @author Tristan Yan
   * @run testng ParallelPrefix
- * @key intermittent
   */


From amy.lu at oracle.com  Tue Oct 25 01:31:46 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Tue, 25 Oct 2016 09:31:46 +0800
Subject: RFR of JDK 8168505: Remove the intermittent keyword from
	java/util/Arrays/ParallelPrefix.java
In-Reply-To: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>
References: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>
Message-ID: <26798e27-68c3-a65c-9ef1-2bb4f7d9bbc8@oracle.com>

Looks fine.

(I'm not an official reviewer.)

Thanks,
Amy

On 10/25/16 9:22 AM, Hamlin Li wrote:
> bug: https://bugs.openjdk.java.net/browse/JDK-8168505
>
> webrev: http://cr.openjdk.java.net/~mli/8168505/webrev.00/
>
> Test ( java/util/Arrays/ParallelPrefix.java) was failing 
> intermittently, related bug (JDK-8080165 
> <https://bugs.openjdk.java.net/browse/JDK-8080165>) has been proved 
> not reproduced. No open bug and no failure reported.
> "@key intermittent" could be removed from test.
>
> Thank you
> -Hamlin
>
>
> diff -r 5651fa4f1478 test/java/util/Arrays/ParallelPrefix.java
> --- a/test/java/util/Arrays/ParallelPrefix.java    Sat Oct 22 17:03:17 
> 2016 +0300
> +++ b/test/java/util/Arrays/ParallelPrefix.java    Mon Oct 24 18:15:44 
> 2016 -0700
> @@ -26,7 +26,6 @@
>   * @summary unit test for Arrays.ParallelPrefix().
>   * @author Tristan Yan
>   * @run testng ParallelPrefix
> - * @key intermittent
>   */
>


From joe.darcy at oracle.com  Tue Oct 25 01:34:07 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Mon, 24 Oct 2016 18:34:07 -0700
Subject: RFR of JDK 8168505: Remove the intermittent keyword from
	java/util/Arrays/ParallelPrefix.java
In-Reply-To: <26798e27-68c3-a65c-9ef1-2bb4f7d9bbc8@oracle.com>
References: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>
	<26798e27-68c3-a65c-9ef1-2bb4f7d9bbc8@oracle.com>
Message-ID: <580EB68F.60103@oracle.com>

+1

(Official reviewer :-)

-Joe

On 10/24/2016 6:31 PM, Amy Lu wrote:
> Looks fine.
>
> (I'm not an official reviewer.)
>
> Thanks,
> Amy
>
> On 10/25/16 9:22 AM, Hamlin Li wrote:
>> bug: https://bugs.openjdk.java.net/browse/JDK-8168505
>>
>> webrev: http://cr.openjdk.java.net/~mli/8168505/webrev.00/
>>
>> Test ( java/util/Arrays/ParallelPrefix.java) was failing 
>> intermittently, related bug (JDK-8080165 
>> <https://bugs.openjdk.java.net/browse/JDK-8080165>) has been proved 
>> not reproduced. No open bug and no failure reported.
>> "@key intermittent" could be removed from test.
>>
>> Thank you
>> -Hamlin
>>
>>
>> diff -r 5651fa4f1478 test/java/util/Arrays/ParallelPrefix.java
>> --- a/test/java/util/Arrays/ParallelPrefix.java    Sat Oct 22 
>> 17:03:17 2016 +0300
>> +++ b/test/java/util/Arrays/ParallelPrefix.java    Mon Oct 24 
>> 18:15:44 2016 -0700
>> @@ -26,7 +26,6 @@
>>   * @summary unit test for Arrays.ParallelPrefix().
>>   * @author Tristan Yan
>>   * @run testng ParallelPrefix
>> - * @key intermittent
>>   */
>>
>


From martinrb at google.com  Tue Oct 25 01:34:32 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 24 Oct 2016 18:34:32 -0700
Subject: RFR of JDK 8168505: Remove the intermittent keyword from
	java/util/Arrays/ParallelPrefix.java
In-Reply-To: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>
References: <c6542838-42a3-462e-fb9b-724f617fa10f@oracle.com>
Message-ID: <CA+kOe0-PZbz3A47w4vrixSTqvMCB0KY4bxqhaOcOXg6q_BRhxQ@mail.gmail.com>

Reviewed!

On Mon, Oct 24, 2016 at 6:22 PM, Hamlin Li <huaming.li at oracle.com> wrote:

> bug: https://bugs.openjdk.java.net/browse/JDK-8168505
>
> webrev: http://cr.openjdk.java.net/~mli/8168505/webrev.00/
>
> Test ( java/util/Arrays/ParallelPrefix.java) was failing intermittently,
> related bug (JDK-8080165 <https://bugs.openjdk.java.net/browse/JDK-8080165>)
> has been proved not reproduced. No open bug and no failure reported.
> "@key intermittent" could be removed from test.
>
> Thank you
> -Hamlin
>
>
> diff -r 5651fa4f1478 test/java/util/Arrays/ParallelPrefix.java
> --- a/test/java/util/Arrays/ParallelPrefix.java    Sat Oct 22 17:03:17
> 2016 +0300
> +++ b/test/java/util/Arrays/ParallelPrefix.java    Mon Oct 24 18:15:44
> 2016 -0700
> @@ -26,7 +26,6 @@
>   * @summary unit test for Arrays.ParallelPrefix().
>   * @author Tristan Yan
>   * @run testng ParallelPrefix
> - * @key intermittent
>   */
>
>

From weijun.wang at oracle.com  Tue Oct 25 08:06:50 2016
From: weijun.wang at oracle.com (Wang Weijun)
Date: Tue, 25 Oct 2016 16:06:50 +0800
Subject: RFR 8167646: Better invalid FilePermission
In-Reply-To: <29fe6a9e-a289-0fd0-6431-897279f2eea2@Oracle.com>
References: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
	<F29526E9-C44F-42FC-AE11-F2246BB0EA45@oracle.com>
	<29fe6a9e-a289-0fd0-6431-897279f2eea2@Oracle.com>
Message-ID: <C70EBC9C-FD48-4864-B1B5-FD4E1D6EEC5C@oracle.com>

Thanks Roger. Can you please also take a look on another code review request at http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-October/044260.html?

--Max

> On Oct 22, 2016, at 2:18 AM, Roger Riggs <Roger.Riggs at oracle.com> wrote:
> 
> Hi Max,
> 
> Looks ok.
> An additional reviewer would be good too.
> 
> Roger
> 
> 
> On 10/20/2016 4:51 AM, Wang Weijun wrote:
>> Please review the code change at
>> 
>>    http://cr.openjdk.java.net/~weijun/8167646/webrev.00/
>> 
>> A new flag invalid is added so invalid FilePermissions (invalid Path) do not equal or imply or is implied by anything else except for itself.
>> 
>> Thanks
>> Max
>> 
> 


From claes.redestad at oracle.com  Tue Oct 25 10:51:01 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 25 Oct 2016 12:51:01 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in FileInput/-OutputStream
	improves startup
Message-ID: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>

Hi,

in https://bugs.openjdk.java.net/browse/JDK-8025619 the logic for 
closing a FileInputStream was changed from a volatile boolean + lock to 
use an AtomicBoolean with rather benign effect on startup - up until the 
point where VarHandles was integrated, which now means we're 
transitively initializing ~20 VarHandle-related classes and perform a 
non-trivial amount of initialization during VM initialization.

Avoiding AtomicBoolean improves startup and footprint for a sample of 
small applications:

Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
Bug: https://bugs.openjdk.java.net/browse/JDK-8168640

Testing: passing all File and VarHandle tests locally, running a larger 
suite of tests on a wider set of platforms with no issues so far.

Thanks!

/Claes


From shade at redhat.com  Tue Oct 25 11:09:16 2016
From: shade at redhat.com (Aleksey Shipilev)
Date: Tue, 25 Oct 2016 13:09:16 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
Message-ID: <e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>

On 10/25/2016 12:51 PM, Claes Redestad wrote:
> Avoiding AtomicBoolean improves startup and footprint for a sample of
> small applications:
> 
> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640

I would generally disagree to avoid Atomics to improve startup. In this
case, we always lock on close(), even for a short time. I wonder if
using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
instead of introducing performance bugs with synchronized-s.

Thanks,
-Aleksey



From peter.levart at gmail.com  Tue Oct 25 12:36:35 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 25 Oct 2016 14:36:35 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
Message-ID: <6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>

Hi Claes,


On 10/25/2016 01:09 PM, Aleksey Shipilev wrote:
> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>> small applications:
>>
>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
> I would generally disagree to avoid Atomics to improve startup. In this
> case, we always lock on close(), even for a short time. I wonder if
> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
> instead of introducing performance bugs with synchronized-s.
>
> Thanks,
> -Aleksey
>
>

In addition, there is no need for this:

  379                     boolean closed;
  380                     synchronized (closeLock) {
  381                         closed = this.closed;
  382                     }

A simple:

         boolean closed = this.closed;

is equivalent, since this.closed is volatile.

But something else bothers me with this code. There is a race. Here are 
the relevant parts:

  316     public void close() throws IOException {
  317         if (closed) {
  318             return;
  319         }
  320         synchronized (closeLock) {
  321             if (closed) {
  322                 return;
  323             }
  324             closed = true;
  325         }
  326
  327         FileChannel fc = channel;
  328         if (fc != null) {
  329            fc.close();
  330         }
  331
  332         fd.closeAll(new Closeable() {
  333             public void close() throws IOException {
  334                close0();
  335            }
  336         });
  337     }

and:

  372     public FileChannel getChannel() {
  373         FileChannel fc = this.channel;
  374         if (fc == null) {
  375             synchronized (this) {
  376                 fc = this.channel;
  377                 if (fc == null) {
  378                     this.channel = fc = FileChannelImpl.open(fd, 
path, true, false, this);
  379                     boolean closed;
  380                     synchronized (closeLock) {
  381                         closed = this.closed;
  382                     }
  383                     if (closed) {
  384                         try {
  385                             fc.close();
  386                         } catch (IOException ioe) {
  387                             throw new InternalError(ioe); // 
should not happen
  388                         }
  389                     }
  390                 }
  391             }
  392         }
  393         return fc;
  394     }


Suppose Thread 1 is executing close() up to line 326, Then Thread 2 
kicks-in and executes getChannel() for the 1st time on this 
FileInputStream. It opens FileChannelImpl and finds closed flag already 
set, so it closes the channel. Thread 1 then resumes from line 326 and 
finds 'channel' field already set, so it closes the channel once more.

FileChannelImpl.close() may be idempotent, but why not making sure it is 
called just once?

In order to fix this, you should read the 'channel' field in close() 
while holding closeLock and you should publish the 'channel' field in 
getChannel() while holding closeLock. Like this:

diff -r 2e7a303cd1ec 
src/java.base/share/classes/java/io/FileInputStream.java
--- a/src/java.base/share/classes/java/io/FileInputStream.java  Wed Oct 
19 11:45:43 2016 +0800
+++ b/src/java.base/share/classes/java/io/FileInputStream.java  Tue Oct 
25 14:33:05 2016 +0200
@@ -26,7 +26,6 @@
  package java.io;

  import java.nio.channels.FileChannel;
-import java.util.concurrent.atomic.AtomicBoolean;
  import sun.nio.ch.FileChannelImpl;


@@ -60,7 +59,9 @@

      private volatile FileChannel channel;

-    private final AtomicBoolean closed = new AtomicBoolean(false);
+    private final Object closeLock = new Object();
+
+    private volatile boolean closed;

      /**
       * Creates a <code>FileInputStream</code> by
@@ -313,12 +314,18 @@
       * @spec JSR-51
       */
      public void close() throws IOException {
-        if (!closed.compareAndSet(false, true)) {
-            // if compareAndSet() returns false closed was already true
+        if (closed) {
              return;
          }
+        FileChannel fc;
+        synchronized (closeLock) {
+            if (closed) {
+                return;
+            }
+            closed = true;
+            fc = channel;
+        }

-        FileChannel fc = channel;
          if (fc != null) {
             fc.close();
          }
@@ -369,8 +376,13 @@
              synchronized (this) {
                  fc = this.channel;
                  if (fc == null) {
-                    this.channel = fc = FileChannelImpl.open(fd, path, 
true, false, this);
-                    if (closed.get()) {
+                    fc = FileChannelImpl.open(fd, path, true, false, this);
+                    boolean closed;
+                    synchronized (closeLock) {
+                        closed = this.closed;
+                        this.channel = fc;
+                    }
+                    if (closed) {
                          try {
                              fc.close();
                          } catch (IOException ioe) {


Regards, Peter



From claes.redestad at oracle.com  Tue Oct 25 12:49:38 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 25 Oct 2016 14:49:38 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
Message-ID: <4eaa2746-eb3a-9400-b1b2-503f8adcf155@oracle.com>


On 2016-10-25 13:09, Aleksey Shipilev wrote:
> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>> small applications:
>>
>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
> I would generally disagree to avoid Atomics to improve startup. In this
> case, we always lock on close(), even for a short time. I wonder if
> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
> instead of introducing performance bugs with synchronized-s.

I see your point, but in this specific case - for close() in particular -
this patch more or less reverts to current 8u behavior, so risk of
any performance impact versus baseline release is slim.

Besides - apart from being quite a bit trickier to get right and
maintain - using Unsafe requires us to reflect over fields to get
the offset, which might eats into the startup gain.

(Maybe an Unsafe.objectFieldOffset that takes Class + field name
rather than Field could yield a small startup improvement?)

Thanks!

/Claes

>
> Thanks,
> -Aleksey
>
>


From claes.redestad at oracle.com  Tue Oct 25 12:57:51 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 25 Oct 2016 14:57:51 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
Message-ID: <d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>



On 2016-10-25 14:36, Peter Levart wrote:
> Hi Claes,
>
>
> On 10/25/2016 01:09 PM, Aleksey Shipilev wrote:
>> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>>> small applications:
>>>
>>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
>> I would generally disagree to avoid Atomics to improve startup. In this
>> case, we always lock on close(), even for a short time. I wonder if
>> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
>> instead of introducing performance bugs with synchronized-s.
>>
>> Thanks,
>> -Aleksey
>>
>>
>
> In addition, there is no need for this:
>
>  379                     boolean closed;
>  380                     synchronized (closeLock) {
>  381                         closed = this.closed;
>  382                     }
>
> A simple:
>
>         boolean closed = this.closed;
>
> is equivalent, since this.closed is volatile.

Good point, cumulative diff to both files:

                  fc = this.channel;
                  if (fc == null) {
                      this.channel = fc = FileChannelImpl.open(fd, path, 
false, true, this);
-                    boolean closed;
-                    synchronized (closeLock) {
-                        closed = this.closed;
-                    }
                      if (closed) {
                          try {
                              fc.close();

>
> But something else bothers me with this code. There is a race. Here 
> are the relevant parts:
>
>  316     public void close() throws IOException {
>  317         if (closed) {
>  318             return;
>  319         }
>  320         synchronized (closeLock) {
>  321             if (closed) {
>  322                 return;
>  323             }
>  324             closed = true;
>  325         }
>  326
>  327         FileChannel fc = channel;
>  328         if (fc != null) {
>  329            fc.close();
>  330         }
>  331
>  332         fd.closeAll(new Closeable() {
>  333             public void close() throws IOException {
>  334                close0();
>  335            }
>  336         });
>  337     }
>
> and:
>
>  372     public FileChannel getChannel() {
>  373         FileChannel fc = this.channel;
>  374         if (fc == null) {
>  375             synchronized (this) {
>  376                 fc = this.channel;
>  377                 if (fc == null) {
>  378                     this.channel = fc = FileChannelImpl.open(fd, 
> path, true, false, this);
>  379                     boolean closed;
>  380                     synchronized (closeLock) {
>  381                         closed = this.closed;
>  382                     }
>  383                     if (closed) {
>  384                         try {
>  385                             fc.close();
>  386                         } catch (IOException ioe) {
>  387                             throw new InternalError(ioe); // 
> should not happen
>  388                         }
>  389                     }
>  390                 }
>  391             }
>  392         }
>  393         return fc;
>  394     }
>
>
> Suppose Thread 1 is executing close() up to line 326, Then Thread 2 
> kicks-in and executes getChannel() for the 1st time on this 
> FileInputStream. It opens FileChannelImpl and finds closed flag 
> already set, so it closes the channel. Thread 1 then resumes from line 
> 326 and finds 'channel' field already set, so it closes the channel 
> once more.
>
> FileChannelImpl.close() may be idempotent, but why not making sure it 
> is called just once?

Hmm, it would seem like a pre-existing issue that was not dealt with 
neither before nor after JDK-8025619, no?

And FileChannel inherits AbstractInterruptibleChannel::close() (public 
final), which specifies behavior: "If the channel has already been 
closed then this method returns immediately." Thus I don't think the 
extra ceremony is warranted, won't you agree?

Thanks!

/Claes

From peter.levart at gmail.com  Tue Oct 25 13:21:43 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 25 Oct 2016 15:21:43 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
Message-ID: <dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>



On 10/25/2016 02:57 PM, Claes Redestad wrote:
>> FileChannelImpl.close() may be idempotent, but why not making sure it 
>> is called just once?
>
> Hmm, it would seem like a pre-existing issue that was not dealt with 
> neither before nor after JDK-8025619, no?
>
> And FileChannel inherits AbstractInterruptibleChannel::close() (public 
> final), which specifies behavior: "If the channel has already been 
> closed then this method returns immediately." Thus I don't think the 
> extra ceremony is warranted, won't you agree?
>
> Thanks!
>
> /Claes 

Ok, then what about a hint in the form of a brief comment that there is 
a race that might invoke FileChannelImpl.close() twice, but it is 
harmless as FileChannelImpl.close() is idempotent? It might be helpful 
to a future bug-hunter...

Regards, Peter


From christoph.langer at sap.com  Tue Oct 25 13:39:05 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 25 Oct 2016 13:39:05 +0000
Subject: RFR (JAXP): 8168664 [JAXP] XALAN: xsl:element generates namespace
	prefixes if namespace is given as URI only
Message-ID: <a3673dbc0f8842b085da705f96fc2b6d@DEROTE13DE08.global.corp.sap>

Hi JAXP experts,

please review a fix for a new issue regarding namespace handling in Xalan with xsl:element.

Bug: https://bugs.openjdk.java.net/browse/JDK-8168664
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8168664.0/

I'm not 100% sure if this is the right way to go or if it would break some correct behavior elsewhere. But I think the current behavior is either not correct or at least it is not required to generate new xsl namespace prefixes if the namespace comes in as URI only to an xsl:element node. The interpretative transformer from the Apache Xalan project will also produce the expected output, different to the compiled XSLTC here.

In the webrev, my changes to XslElement.java are only cosmetical/comments, the behavior does not change. In BasisLibrary.java I also took the opportunity to remove the $Id tag and the unused method "nodeList2IteratorUsingHandleFromNode".

If you think my change is good, I'll also add a test that runs the transformation which can be found in the bug...

Thanks & best regards
Christoph


From Roger.Riggs at Oracle.com  Tue Oct 25 14:12:02 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 25 Oct 2016 10:12:02 -0400
Subject: RFR 8168127: FilePermissionCollection merges incorrectly
In-Reply-To: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
References: <B721C49A-7EB3-4A36-98BF-168F03FD95CA@oracle.com>
Message-ID: <d0cbec59-d6ef-c6fc-965c-4b59caff39d6@Oracle.com>

Hi Max,

It might be worth a comment (Lines 258 and 270) that the names are 
generated to be different than the original.
Otherwise, looks fine.

Roger



On 10/19/2016 10:47 PM, Wang Weijun wrote:
> Please review the code change at
>
>     http://cr.openjdk.java.net/~weijun/8168127/webrev.00/
>
> Two changes:
>
> 1. npath2 is considered in equals and hashCode of FilePermission, so 2 objects with different npath2 can be added to a map and different entries.
>
> 2. special name for newPermUsingAltPath and newPermPlusAltPath results, so FilePermissionCollection::add will not merge one with the original.
>
> Thanks
> Max
>


From Roger.Riggs at Oracle.com  Tue Oct 25 19:51:00 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 25 Oct 2016 15:51:00 -0400
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with address
	in use
Message-ID: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>

Please review a test fix for an address-in-use problem in this CORBA 
serialization test.
The test is simplified to just communicate client to server via the orb 
in the process.

It does not need activation, a name service, or the orbd to exercise the
necessary serialization implementation.  And therefore, no contention 
for sockets, ports, etc.

Webrev:
    http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/

Thanks, Roger


From pavel.rappo at oracle.com  Tue Oct 25 19:52:58 2016
From: pavel.rappo at oracle.com (Pavel Rappo)
Date: Tue, 25 Oct 2016 20:52:58 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
Message-ID: <CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>

I've skimmed through the code and I'm not sure I can see any asynchronicity
(you were pointing at the lack of it in BufferedReader).

And the mechanics of this is very puzzling to me, to be honest:

    void blockingFill(boolean forced) throws IOException {
        fill(forced);
        while (readPos == writePos) {
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                // An interrupt may mean more data is available
            }
            fill(forced);
        }
    }

I thought you were suggesting that we should utilize the tools which OS provides
more efficiently. Instead we have something that looks very similarly to a
"busy loop" and... also who and when is supposed to interrupt Thread.sleep()?

Sorry, I'm not following. Could you please explain how this is supposed to work?

> On 24 Oct 2016, at 15:59, Brunoais <brunoaiss at gmail.com> wrote:
> 
> Attached and sending!
> 
> 
> On 24/10/2016 13:48, Pavel Rappo wrote:
>> Could you please send a new email on this list with the source attached as a
>> text file?
>> 
>>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com> wrote:
>>> 
>>> Here's my poc/prototype:
>>> http://pastebin.com/WRpYWDJF
>>> 
>>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>>> I also wrote some javadoc to help guiding through the class.
>>> 
>>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>> 
>>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>> 
>>> I built a simple test code for it to have some ideas about performance and correctness.
>>> 
>>> http://pastebin.com/eh6LFgwT
>>> 
>>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>> 
>>> I'll also leave here some conclusions about speed and resource consumption I found.
>>> 
>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>> 
>>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>> 
>>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>> 
>>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>> 
>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>> 
>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>>> Not specifically FileReader. While you're talking about the case of effective
>>>> reading from a file.
>>>> 
>>>> I guess there's one existing possibility to provide exactly what you need (as I
>>>> understand it) under this method:
>>>> 
>>>> /**
>>>>  * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>>  * from the file in an efficient manner...
>>>>    ...
>>>>  */
>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>> 
>>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>>> needs to be investigated not only for your favorite OS but for other OSes as
>>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>> 
>>>> Thanks,
>>>> -Pavel
>>>> 
>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>>>> 
>>>>> Pavel is right.
>>>>> 
>>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>> 
>>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>> 
>>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>> 
>>>>> 
>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>> Hi Pavel,
>>>>>> 
>>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>>> and managing the swaps and async reads transparently.
>>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>> 
>>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>>> applications and libraries.
>>>>>> 
>>>>>> $.02, Roger
>>>>>> 
>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>> 
>>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>>> in your applications?
>>>>>>> 
>>>>>>> -Pavel
>>>>>>> 
>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>>>> 
>>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>> 
>>>>>>>> That's why I'm asking this here.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>> 
>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>> 
>> 
> 
> <BufferedNonBlockStream.java><Tests.java>


From paul.sandoz at oracle.com  Tue Oct 25 20:31:55 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 13:31:55 -0700
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
Message-ID: <9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>

Hi Roger,

Do you need to use a double-checked locking idiom in getEchoStub?

I cannot recall the life-cycle of per test execution, but i suspect you might be able to turn the static field and method getEchoStub into a non-static field and just make the method synchronized, but you might ned to annotate the shutdown with @AfterClass instead.

Actually, perhaps even better initiate the ORB and create the stub in a @BeforeClass method and clean up in a @AfterClass method?

Paul.

> On 25 Oct 2016, at 12:51, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Please review a test fix for an address-in-use problem in this CORBA serialization test.
> The test is simplified to just communicate client to server via the orb in the process.
> 
> It does not need activation, a name service, or the orbd to exercise the
> necessary serialization implementation.  And therefore, no contention for sockets, ports, etc.
> 
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/
> 
> Thanks, Roger
> 


From paul.sandoz at oracle.com  Tue Oct 25 20:39:40 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 13:39:40 -0700
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <4eaa2746-eb3a-9400-b1b2-503f8adcf155@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<4eaa2746-eb3a-9400-b1b2-503f8adcf155@oracle.com>
Message-ID: <30BCB9BB-7FF6-421A-BCEC-962480961E62@oracle.com>


> On 25 Oct 2016, at 05:49, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> 
> On 2016-10-25 13:09, Aleksey Shipilev wrote:
>> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>>> small applications:
>>> 
>>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
>> I would generally disagree to avoid Atomics to improve startup. In this
>> case, we always lock on close(), even for a short time. I wonder if
>> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
>> instead of introducing performance bugs with synchronized-s.
> 
> I see your point, but in this specific case - for close() in particular -
> this patch more or less reverts to current 8u behavior, so risk of
> any performance impact versus baseline release is slim.
> 
> Besides - apart from being quite a bit trickier to get right and
> maintain - using Unsafe requires us to reflect over fields to get
> the offset, which might eats into the startup gain.
> 

Out of curiosity how much startup improvement are you observing, given that other VarHandle initialisation will occur at startup before execution of main?

Paul.

> (Maybe an Unsafe.objectFieldOffset that takes Class + field name
> rather than Field could yield a small startup improvement?)
> 
> Thanks!
> 
> /Claes
> 
>> 
>> Thanks,
>> -Aleksey
>> 
>> 
> 


From Roger.Riggs at Oracle.com  Tue Oct 25 20:50:25 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 25 Oct 2016 16:50:25 -0400
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
	<9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
Message-ID: <d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>

Hi,

On 10/25/2016 4:31 PM, Paul Sandoz wrote:
> Hi Roger,
>
> Do you need to use a double-checked locking idiom in getEchoStub?
Probably not, I'm  just practicing the idiom.
We are unlikely to move to concurrent test execution in testng (when it 
supports it).
>
> I cannot recall the life-cycle of per test execution, but i suspect you might be able to turn the static field and method getEchoStub into a non-static field and just make the method synchronized, but you might ned to annotate the shutdown with @AfterClass instead.
That would work too.

What's the rule for choosing synchronized vs volatile?  I've gotten 
different recommendations in different cases.
What's Occam's razor here?
>
> Actually, perhaps even better initiate the ORB and create the stub in a @BeforeClass method and clean up in a @AfterClass method?
that would work, but it is only used in one of the tests.  So just 
trying to keep the usage localized.

Thanks, Roger

>
> Paul.
>
>> On 25 Oct 2016, at 12:51, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>>
>> Please review a test fix for an address-in-use problem in this CORBA serialization test.
>> The test is simplified to just communicate client to server via the orb in the process.
>>
>> It does not need activation, a name service, or the orbd to exercise the
>> necessary serialization implementation.  And therefore, no contention for sockets, ports, etc.
>>
>> Webrev:
>>    http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/
>>
>> Thanks, Roger
>>


From brunoaiss at gmail.com  Tue Oct 25 21:16:27 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Tue, 25 Oct 2016 22:16:27 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
Message-ID: <4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>

Thank you for your time. I'll try to explain it. I hope I can clear it up.

First of it, I made a meaning mistake between asynchronous and 
non-blocking. This implementation uses a non-blocking algorithm 
internally while providing a blocking-like algorithm on the surface. It 
is single-threaded and not multi-threaded where one thread fetches data 
and blocks waiting and the other accumulates it and provides to 
whichever wants it.

Second of it, I had made a mistake of going after BufferedReader instead 
of going after BufferedInputStream. If you want me to go after 
BufferedReader it's ok but I only thought that going after 
BufferedInputStream would be more generically useful than 
BufferedReaderwhen I started the poc.

*On to my code:*

*Short answers:*

 1. The sleep(int) exists because I don't know how to wait until more
    data exists in the buffer which is part of read()'s contract.
 2. The |ByteBuffer| gives a buffer that is filled by the OS (what I
    believe Channels do) instead of getting data only by demand (what I
    believe Streams do).

*Full answers:*

The blockingFill(boolean) method is a method for a busy wait for a fill 
which is used exclusively by the read() method. All other methods use 
the version that does not sleep (fill(boolean)).
blockingFill(boolean)'s existance like that is only because the read() 
method must not return unless either:

 1. The stream ended.
 2. The next byte is ready for reading.

Additionally, statistically, that while loop will rarely evaluate to 
true as reads are in chunks so readPos will be behind writePos most of 
the time.
I have no idea if an interrupt will ever happen, to be honest. The main 
reasons why I'm using a sleep is because I didn't want a hog onto the 
CPU in a full thread usage busy wait and because I didn't find any way 
of doing a thread sleep in order to wake up later when the buffer 
managed by native code has more data.

The Non-blocking part is managed by the buffer the OS keeps filling most 
if not all the time. That buffer is the field

|ByteBuffer readBuffer|

That's the gaining part against the plain old Buffered classes.


Did that make sense to you? Feel free to ask anything else you need.


On 25/10/2016 20:52, Pavel Rappo wrote:
> I've skimmed through the code and I'm not sure I can see any asynchronicity
> (you were pointing at the lack of it in BufferedReader).
> And the mechanics of this is very puzzling to me, to be honest:
>      void blockingFill(boolean forced) throws IOException {
>          fill(forced);
>          while (readPos == writePos) {
>              try {
>                  Thread.sleep(100);
>              } catch (InterruptedException e) {
>                  // An interrupt may mean more data is available
>              }
>              fill(forced);
>          }
>      }
> I thought you were suggesting that we should utilize the tools which OS provides
> more efficiently. Instead we have something that looks very similarly to a
> "busy loop" and... also who and when is supposed to interrupt Thread.sleep()?
> Sorry, I'm not following. Could you please explain how this is supposed to work?
>> On 24 Oct 2016, at 15:59, Brunoais <brunoaiss at gmail.com> wrote:
>> Attached and sending!
>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>> Could you please send a new email on this list with the source attached as a
>>> text file?
>>>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com> wrote:
>>>> Here's my poc/prototype:
>>>> http://pastebin.com/WRpYWDJF
>>>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>>>> I also wrote some javadoc to help guiding through the class.
>>>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>>> I built a simple test code for it to have some ideas about performance and correctness.
>>>> http://pastebin.com/eh6LFgwT
>>>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>>> I'll also leave here some conclusions about speed and resource consumption I found.
>>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>>>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>>>> Not specifically FileReader. While you're talking about the case of effective
>>>>> reading from a file.
>>>>> I guess there's one existing possibility to provide exactly what you need (as I
>>>>> understand it) under this method:
>>>>> /**
>>>>>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>>>   * from the file in an efficient manner...
>>>>>     ...
>>>>>   */
>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>>>> needs to be investigated not only for your favorite OS but for other OSes as
>>>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>>> Thanks,
>>>>> -Pavel
>>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>> Pavel is right.
>>>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>>> Hi Pavel,
>>>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>>>> and managing the swaps and async reads transparently.
>>>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>>>> applications and libraries.
>>>>>>> $.02, Roger
>>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>>>> in your applications?
>>>>>>>> -Pavel
>>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>>> That's why I'm asking this here.
>>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>>> Hi,
>>>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>> <BufferedNonBlockStream.java><Tests.java>


From paul.sandoz at oracle.com  Tue Oct 25 21:18:10 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 14:18:10 -0700
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
	<9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
	<d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>
Message-ID: <72FFE4AE-EE52-4074-9878-DF3F53C02BDA@oracle.com>


> On 25 Oct 2016, at 13:50, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Hi,
> 
> On 10/25/2016 4:31 PM, Paul Sandoz wrote:
>> Hi Roger,
>> 
>> Do you need to use a double-checked locking idiom in getEchoStub?
> Probably not, I'm  just practicing the idiom.
> We are unlikely to move to concurrent test execution in testng (when it supports it).
>> 
>> I cannot recall the life-cycle of per test execution, but i suspect you might be able to turn the static field and method getEchoStub into a non-static field and just make the method synchronized, but you might ned to annotate the shutdown with @AfterClass instead.
> That would work too.
> 
> What's the rule for choosing synchronized vs volatile?  I've gotten different recommendations in different cases.
> What's Occam's razor here?

Double-checked locking requires both :-), so you can safely publish the singleton. It?s generally used when performance is a concern, otherwise i would go for the simpler solution which is more likely to be correct and easier to understand :-)


>> 
>> Actually, perhaps even better initiate the ORB and create the stub in a @BeforeClass method and clean up in a @AfterClass method?
> that would work, but it is only used in one of the tests.  So just trying to keep the usage localized.
> 

Ah, yes, i see.

Paul.


From Roger.Riggs at Oracle.com  Tue Oct 25 21:28:33 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 25 Oct 2016 17:28:33 -0400
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <72FFE4AE-EE52-4074-9878-DF3F53C02BDA@oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
	<9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
	<d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>
	<72FFE4AE-EE52-4074-9878-DF3F53C02BDA@oracle.com>
Message-ID: <3b884480-ed7e-9d45-008b-27293a0c320b@Oracle.com>

Hi Paul,

Thanks for explanation and suggestions.

I updated the webrev to use instance methods and fields and using 
synchronized.
(And a bit of long-line cleanup.)

webrev:
    http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/

Thanks, Roger


On 10/25/2016 5:18 PM, Paul Sandoz wrote:
>> On 25 Oct 2016, at 13:50, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>>
>> Hi,
>>
>> On 10/25/2016 4:31 PM, Paul Sandoz wrote:
>>> Hi Roger,
>>>
>>> Do you need to use a double-checked locking idiom in getEchoStub?
>> Probably not, I'm  just practicing the idiom.
>> We are unlikely to move to concurrent test execution in testng (when it supports it).
>>> I cannot recall the life-cycle of per test execution, but i suspect you might be able to turn the static field and method getEchoStub into a non-static field and just make the method synchronized, but you might ned to annotate the shutdown with @AfterClass instead.
>> That would work too.
>>
>> What's the rule for choosing synchronized vs volatile?  I've gotten different recommendations in different cases.
>> What's Occam's razor here?
> Double-checked locking requires both :-), so you can safely publish the singleton. It?s generally used when performance is a concern, otherwise i would go for the simpler solution which is more likely to be correct and easier to understand :-)
>
>
>>> Actually, perhaps even better initiate the ORB and create the stub in a @BeforeClass method and clean up in a @AfterClass method?
>> that would work, but it is only used in one of the tests.  So just trying to keep the usage localized.
>>
> Ah, yes, i see.
>
> Paul.
>


From paul.sandoz at oracle.com  Tue Oct 25 21:30:24 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 14:30:24 -0700
Subject: Iterator.forEachRemaining and Iterator.remove
In-Reply-To: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>
References: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>
Message-ID: <308C6C99-CCF5-4D10-9FEF-2B0F598ED299@oracle.com>

Hi Martin,

Hmm?

The intent of ArrayList/Vector/LinkedList implementations is clear, leave the iterator in the same state as a last successful next call.

I believe ArrayDeque?s iterator (in repo, unsure about your updates) has different behaviour e.g.

  ad.next();
  ad.forEachRemaining(?);
  ad.remove(); // which element is removed?


We also don?t say anything about the behaviour if remove is called by the consumer function. This is potentially more problematic since we often optimize for the bulk operation and don?t update state within the loop.

I am inclined to say the behaviour in both cases is undefined, which we could mention in the documentation.

Paul.

> On 24 Oct 2016, at 12:32, Martin Buchholz <martinrb at google.com> wrote:
> 
> It doesn't make a lot of sense for users to call Iterator.remove after calling Iterator.forEachRemaining.
> 
> The default implementation has no choice but to do
> 
>         while (hasNext())
>             action.accept(next());
> 
> upon which remove() removes the last element.  What should overriding implementations do?  Emulate the default implementation's behavior or throw IllegalStateException, or even remove the element returned by the last actual call to next?  The spec is (perhaps intentionally) unclear on this point.  I'm thinking we emulate the default implementation's behavior, because common collections like ArrayList work this way, and some users may be depending on it, perhaps unwisely.


From paul.sandoz at oracle.com  Tue Oct 25 21:35:47 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 14:35:47 -0700
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <3b884480-ed7e-9d45-008b-27293a0c320b@Oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
	<9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
	<d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>
	<72FFE4AE-EE52-4074-9878-DF3F53C02BDA@oracle.com>
	<3b884480-ed7e-9d45-008b-27293a0c320b@Oracle.com>
Message-ID: <430A8113-E183-479C-A461-9BF7D088463B@oracle.com>


> On 25 Oct 2016, at 14:28, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Hi Paul,
> 
> Thanks for explanation and suggestions.
> 
> I updated the webrev to use instance methods and fields and using synchronized.
> (And a bit of long-line cleanup.)
> 
> webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/
> 

You have a rogue @BeforeClass:
 202     @BeforeClass
 203     synchronized Echo getEchoStub() throws RemoteException {

Otherwise looks good.

Paul.

From martinrb at google.com  Tue Oct 25 22:16:25 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 25 Oct 2016 15:16:25 -0700
Subject: Iterator.forEachRemaining and Iterator.remove
In-Reply-To: <308C6C99-CCF5-4D10-9FEF-2B0F598ED299@oracle.com>
References: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>
	<308C6C99-CCF5-4D10-9FEF-2B0F598ED299@oracle.com>
Message-ID: <CA+kOe09jDgbJcaMHOKAhu5QhWn1nTDijCBPw_5Q+BaavdZcf6w@mail.gmail.com>

Actually, the ArrayList implementation updates fields only at the end of
the iteration, so if an action throws in the middle, the iterator is
semi-corrupted (in the sense that remove() will remove the "wrong" element
and next will return previously visited elements).  I think it's best to
say in the spec that after calling forEachRemaining, subsequent operations
on the Iterator are undefined (whether or not the action throws).  For
consistency, we should probably keep the historic behavior that remove()
after forEachRemaining() removes the last element, even though that is not
to be encouraged.

On Tue, Oct 25, 2016 at 2:30 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:

> Hi Martin,
>
> Hmm?
>
> The intent of ArrayList/Vector/LinkedList implementations is clear, leave
> the iterator in the same state as a last successful next call.
>
> I believe ArrayDeque?s iterator (in repo, unsure about your updates) has
> different behaviour e.g.
>
>   ad.next();
>   ad.forEachRemaining(?);
>   ad.remove(); // which element is removed?
>
>
> We also don?t say anything about the behaviour if remove is called by the
> consumer function. This is potentially more problematic since we often
> optimize for the bulk operation and don?t update state within the loop.
>
> I am inclined to say the behaviour in both cases is undefined, which we
> could mention in the documentation.
>
> Paul.
>
> > On 24 Oct 2016, at 12:32, Martin Buchholz <martinrb at google.com> wrote:
> >
> > It doesn't make a lot of sense for users to call Iterator.remove after
> calling Iterator.forEachRemaining.
> >
> > The default implementation has no choice but to do
> >
> >         while (hasNext())
> >             action.accept(next());
> >
> > upon which remove() removes the last element.  What should overriding
> implementations do?  Emulate the default implementation's behavior or throw
> IllegalStateException, or even remove the element returned by the last
> actual call to next?  The spec is (perhaps intentionally) unclear on this
> point.  I'm thinking we emulate the default implementation's behavior,
> because common collections like ArrayList work this way, and some users may
> be depending on it, perhaps unwisely.
>
>

From paul.sandoz at oracle.com  Tue Oct 25 23:13:57 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 25 Oct 2016 16:13:57 -0700
Subject: Iterator.forEachRemaining and Iterator.remove
In-Reply-To: <CA+kOe09jDgbJcaMHOKAhu5QhWn1nTDijCBPw_5Q+BaavdZcf6w@mail.gmail.com>
References: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>
	<308C6C99-CCF5-4D10-9FEF-2B0F598ED299@oracle.com>
	<CA+kOe09jDgbJcaMHOKAhu5QhWn1nTDijCBPw_5Q+BaavdZcf6w@mail.gmail.com>
Message-ID: <693FD645-CC34-4F80-AF4C-EF1EF622EFA1@oracle.com>


> On 25 Oct 2016, at 15:16, Martin Buchholz <martinrb at google.com> wrote:
> 
> Actually, the ArrayList implementation updates fields only at the end of the iteration, so if an action throws in the middle, the iterator is semi-corrupted (in the sense that remove() will remove the "wrong" element and next will return previously visited elements).

Good point.


> I think it's best to say in the spec that after calling forEachRemaining, subsequent operations on the Iterator are undefined (whether or not the action throws).


Yes (and for an action calling remove).


>   For consistency, we should probably keep the historic behavior that remove() after forEachRemaining() removes the last element, even though that is not to be encouraged.
> 

Agreed.

Paul.

From martinrb at google.com  Wed Oct 26 00:44:06 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 25 Oct 2016 17:44:06 -0700
Subject: Iterator.forEachRemaining and Iterator.remove
In-Reply-To: <693FD645-CC34-4F80-AF4C-EF1EF622EFA1@oracle.com>
References: <CA+kOe09=g_08sQ84KbtJz+hhKf1QSuosETmBVXiiU4DPEM5Vng@mail.gmail.com>
	<308C6C99-CCF5-4D10-9FEF-2B0F598ED299@oracle.com>
	<CA+kOe09jDgbJcaMHOKAhu5QhWn1nTDijCBPw_5Q+BaavdZcf6w@mail.gmail.com>
	<693FD645-CC34-4F80-AF4C-EF1EF622EFA1@oracle.com>
Message-ID: <CA+kOe0_o85tk=8Kwgjao=D_xPUhk427U48akhoTJ5sMWHHd2QQ@mail.gmail.com>

I assigned this to Paul (hope that's OK)

JDK-8168745
Iterator.forEachRemaining vs. Iterator.remove

On Tue, Oct 25, 2016 at 4:13 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:

>
> > On 25 Oct 2016, at 15:16, Martin Buchholz <martinrb at google.com> wrote:
> >
> > Actually, the ArrayList implementation updates fields only at the end of
> the iteration, so if an action throws in the middle, the iterator is
> semi-corrupted (in the sense that remove() will remove the "wrong" element
> and next will return previously visited elements).
>
> Good point.
>
>
> > I think it's best to say in the spec that after calling
> forEachRemaining, subsequent operations on the Iterator are undefined
> (whether or not the action throws).
>
>
> Yes (and for an action calling remove).
>
>
> >   For consistency, we should probably keep the historic behavior that
> remove() after forEachRemaining() removes the last element, even though
> that is not to be encouraged.
> >
>
> Agreed.
>
> Paul.
>

From srinivasan.raghavan at oracle.com  Wed Oct 26 02:50:37 2016
From: srinivasan.raghavan at oracle.com (Srinivasan Raghavan)
Date: Wed, 26 Oct 2016 08:20:37 +0530
Subject: RFR:JDK-8075205 java/net/URLClassLoader/closetest/CloseTest.java
	and GetResourceAsStream.java failed with existing dir
In-Reply-To: <6AE698AC-5301-46A3-92D2-4C6B64A3A2FB@oracle.com>
References: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>
	<019E6AAB-85A3-4D4E-A2A0-2DCBCF028EF3@oracle.com>
	<6AE698AC-5301-46A3-92D2-4C6B64A3A2FB@oracle.com>
Message-ID: <7BCAB4F4-93BF-4B5C-8494-4395292E3F7A@oracle.com>


Hi all

Please help with one more reviewer comment.

Regards
Srinivasan Raghavan
> On 19-Oct-2016, at 9:38 AM, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
> 
> Thanks for the review.
>> On 18-Oct-2016, at 4:16 PM, Chris Hegarty <chris.hegarty at oracle.com> wrote:
>> 
>> 
>>> On 17 Oct 2016, at 09:51, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
>>> 
>>> Hi all
>>> 
>>> Please review the fix for the bug 
>>> 
>>> Bug :https://bugs.openjdk.java.net/browse/JDK-8075205
>>> 
>>> The tests uses classes directory for the output files. This can result in the files being left over after the test is complete which can result in instability. The tests copies the files to be compiled form test src to test classes which can result in copy of permission and result in instability because the test has delete operations. The test fails randomly mostly in copy or delete operation. I propose the test to be refactored to make the use scratch directory as its output directory and eliminate shell by using testlibrary utils.
>>> 
>>> fix : http://cr.openjdk.java.net/~sraghavan/8075205/webrev.00/
>> 
>> This looks good to me. Thanks Srinivasan.
>> 
>> -Chris.
> 


From ecki at zusammenkunft.net  Wed Oct 26 05:14:40 2016
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Wed, 26 Oct 2016 05:14:40 +0000 (UTC)
Subject: Request/discussion: BufferedReader reading using async API
	while providing sync API
In-Reply-To: <48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
Message-ID: <57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>

?Hallo Brunoais,
In the past I die some experiments with non-blocking file channels in the hope to increase throughput in a similiar way then your buffered stream. I also used direct allocated buffers. However my results have not been that encouraging (especially If a upper layer used larger reads). I thought back in the time this was mostly die to the fact that it NOT wraps to real AsyncFIO on most platforms. But maybe I just measured it wrong, so I will have a closer look on your impl.
Generally I would recommend to make the Benchmark a bit more reliable with JMH and in order to do this to externalize the direct buffer allocation (as it ist slow if done repeatingly). This also allows you to publish some results with varrying workloads (on different machines).
I would also measure the readCount to see if short reads happen.
?BTW, I might as well try to only read till the end of the buffer in the backfilling-wraps-around case and not issue two requests, that might remove some additional latency.

Gruss
Bernd
-- 
http://bernd.eckenfels.net

		_____________________________
From: Brunoais <brunoaiss at gmail.com>
Sent: Montag, Oktober 24, 2016 6:30 PM
Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API
To: Pavel Rappo <pavel.rappo at oracle.com>
Cc:  <core-libs-dev at openjdk.java.net>


Attached and sending!


On 24/10/2016 13:48, Pavel Rappo wrote:
> Could you please send a new email on this list with the source attached as a
> text file?
>
>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Here's my poc/prototype:
>> http://pastebin.com/WRpYWDJF
>>
>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>> I also wrote some javadoc to help guiding through the class.
>>
>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>
>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>
>> I built a simple test code for it to have some ideas about performance and correctness.
>>
>> http://pastebin.com/eh6LFgwT
>>
>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>
>> I'll also leave here some conclusions about speed and resource consumption I found.
>>
>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>
>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>
>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>
>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>
>> @Pavel, are you open for discussion now ;)? Need anything else?
>>
>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>> Not specifically FileReader. While you're talking about the case of effective
>>> reading from a file.
>>>
>>> I guess there's one existing possibility to provide exactly what you need (as I
>>> understand it) under this method:
>>>
>>> /**
>>>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>   * from the file in an efficient manner...
>>>     ...
>>>   */
>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>
>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>> needs to be investigated not only for your favorite OS but for other OSes as
>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>
>>> Thanks,
>>> -Pavel
>>>
>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com> wrote:
>>>>
>>>> Pavel is right.
>>>>
>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>
>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>
>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>
>>>>
>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>> Hi Pavel,
>>>>>
>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>> and managing the swaps and async reads transparently.
>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>
>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>> applications and libraries.
>>>>>
>>>>> $.02, Roger
>>>>>
>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>
>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>> in your applications?
>>>>>>
>>>>>> -Pavel
>>>>>>
>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com> wrote:
>>>>>>>
>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>
>>>>>>> That's why I'm asking this here.
>>>>>>>
>>>>>>>
>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>
>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>
>




	

From brunoaiss at gmail.com  Wed Oct 26 06:57:49 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Wed, 26 Oct 2016 07:57:49 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
Message-ID: <ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>

Hey Bernd!

I don't know how far back you did such thing but I'm getting positive 
results with my non-JMH tests. I do have to evaluate my results against 
logic. After some reads, the OS starts caching the file which is not 
what I want. It's easy to know when that happens, though. The times fall 
from ~30s to ~5s and the HDD keeps near idle reading (just looking at 
the LED is enough to understand).

If you don't test synchronous work and you only run the reads, you will 
only get marginal results as the OS has no real time to fill the buffer.
My research shows the 2 major kernels (windows' and GNU/Linux) have 
non-blocking user-level buffer handling where I give a buffer for the OS 
to read and it keeps filling it and sending messages/signals as it 
writes chunks. Linux has an OS interrupt that only sends the signal 
after it is full, though. There's also another version of them where 
they use an internal buffer of same size as the buffer you allocate for 
the OS and then internally call memcopy() into your user-level memory 
when asked. Tests on the internet show that memcopy is as fast (for 0-1 
elements) or faster than System.arraycopy(). I have no idea if they are 
true.

All this was for me to add that, that code is tuned to copy from the 
read buffer only when it is, at least, at half capacity and the internal 
buffer has enough storage space. The process is forced only if nothing 
had been read on the previous fill() call. It is built to use JNI as 
little as possible while providing the major contract 
BufferedInputStream has.
Finally, I never, ever compact the read buffer. It requires doing a 
memcopy which is definitely not necessary.

Anyway, those tests about time I made were just to get an order of 
magnitude about speed difference. I intended to do them differently but 
JMH looks good so I'll use JMH to test now.

Short reads only happen when fill(true) is called. That happens for 
desperate get of data.

I'll look into the avoiding double reading requests. I do think it won't 
bring significant improvements if any at all. It only happens when the 
buffer is nearly empty and any byte of data is welcome "at any cost".
Besides, whomever called read at that point would also have had an 
availability() of 0 and still called read()/read(byte[]).


On 26/10/2016 06:14, Bernd Eckenfels wrote:
>  Hallo Brunoais,
>
> In the past I die some experiments with non-blocking file channels in 
> the hope to increase throughput in a similiar way then your buffered 
> stream. I also used direct allocated buffers. However my results have 
> not been that encouraging (especially If a upper layer used larger 
> reads). I thought back in the time this was mostly die to the fact 
> that it NOT wraps to real AsyncFIO on most platforms. But maybe I just 
> measured it wrong, so I will have a closer look on your impl.
>
> Generally I would recommend to make the Benchmark a bit more reliable 
> with JMH and in order to do this to externalize the direct buffer 
> allocation (as it ist slow if done repeatingly). This also allows you 
> to publish some results with varrying workloads (on different machines).
>
> I would also measure the readCount to see if short reads happen.
>
>  BTW, I might as well try to only read till the end of the buffer in 
> the backfilling-wraps-around case and not issue two requests, that 
> might remove some additional latency.
>
> Gruss
> Bernd
> -- 
> http://bernd.eckenfels.net
>
> _____________________________
> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
> Sent: Montag, Oktober 24, 2016 6:30 PM
> Subject: Re: Request/discussion: BufferedReader reading using async 
> API while providing sync API
> To: Pavel Rappo <pavel.rappo at oracle.com <mailto:pavel.rappo at oracle.com>>
> Cc: <core-libs-dev at openjdk.java.net 
> <mailto:core-libs-dev at openjdk.java.net>>
>
>
> Attached and sending!
>
>
> On 24/10/2016 13:48, Pavel Rappo wrote:
> > Could you please send a new email on this list with the source 
> attached as a
> > text file?
> >
> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
> >>
> >> Here's my poc/prototype:
> >> http://pastebin.com/WRpYWDJF
> >>
> >> I've implemented the bare minimum of the class that follows the 
> same contract of BufferedReader while signaling all issues I think it 
> may have or has in comments.
> >> I also wrote some javadoc to help guiding through the class.
> >>
> >> I could have used more fields from BufferedReader but the names 
> were so minimalistic that were confusing me. I intent to change them 
> before sending this to openJDK.
> >>
> >> One of the major problems this has is long overflowing. It is major 
> because it is hidden, it will be extremely rare and it takes a really 
> long time to reproduce. There are different ways of dealing with it. 
> From just documenting to actually making code that works with it.
> >>
> >> I built a simple test code for it to have some ideas about 
> performance and correctness.
> >>
> >> http://pastebin.com/eh6LFgwT
> >>
> >> This doesn't do a through test if it is actually working correctly 
> but I see no reason for it not working correctly after fixing the 2 
> bugs that test found.
> >>
> >> I'll also leave here some conclusions about speed and resource 
> consumption I found.
> >>
> >> I made tests with default buffer sizes, 5000B 15_000B and 500_000B. 
> I noticed that, with my hardware, with the 1 530 000 000B file, I was 
> getting around:
> >>
> >> In all buffers and fake work: 10~15s speed improvement ( from 90% 
> HDD speed to 100% HDD speed)
> >> In all buffers and no fake work: 1~2s speed improvement ( from 90% 
> HDD speed to 100% HDD speed)
> >>
> >> Changing the buffer size was giving different reading speeds but 
> both were quite equal in how much they would change when changing the 
> buffer size.
> >> Finally, I could always confirm that I/O was always the slowest 
> thing while this code was running.
> >>
> >> For the ones wondering about the file size; it is both to avoid OS 
> cache and to make the reading at the main use-case these objects are 
> for (large streams of bytes).
> >>
> >> @Pavel, are you open for discussion now ;)? Need anything else?
> >>
> >> On 21/10/2016 19:21, Pavel Rappo wrote:
> >>> Just to append to my previous email. BufferedReader wraps any 
> Reader out there.
> >>> Not specifically FileReader. While you're talking about the case 
> of effective
> >>> reading from a file.
> >>>
> >>> I guess there's one existing possibility to provide exactly what 
> you need (as I
> >>> understand it) under this method:
> >>>
> >>> /**
> >>> * Opens a file for reading, returning a {@code BufferedReader} to 
> read text
> >>> * from the file in an efficient manner...
> >>> ...
> >>> */
> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
> >>>
> >>> It can return _anything_ as long as it is a BufferedReader. We can 
> do it, but it
> >>> needs to be investigated not only for your favorite OS but for 
> other OSes as
> >>> well. Feel free to prototype this and we can discuss it on the 
> list later.
> >>>
> >>> Thanks,
> >>> -Pavel
> >>>
> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
> >>>>
> >>>> Pavel is right.
> >>>>
> >>>> In reality, I was expecting such BufferedReader to use only a 
> single buffer and have that Buffer being filled asynchronously, not in 
> a different Thread.
> >>>> Additionally, I don't have the intention of having a larger 
> buffer than before unless stated through the API (the constructor).
> >>>>
> >>>> In my idea, internally, it is supposed to use 
> java.nio.channels.AsynchronousFileChannel or equivalent.
> >>>>
> >>>> It does not prevent having two buffers and I do not intent to 
> change BufferedReader itself. I'd do an BufferedAsyncReader of sorts 
> (any name suggestion is welcome as I'm an awful namer).
> >>>>
> >>>>
> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
> >>>>> Hi Pavel,
> >>>>>
> >>>>> I think Brunoais asking for a double buffering scheme in which 
> the implementation of
> >>>>> BufferReader fills (a second buffer) in parallel with the 
> application reading from the 1st buffer
> >>>>> and managing the swaps and async reads transparently.
> >>>>> It would not change the API but would change the interactions 
> between the buffered reader
> >>>>> and the underlying stream. It would also increase memory 
> requirements and processing
> >>>>> by introducing or using a separate thread and the necessary 
> synchronization.
> >>>>>
> >>>>> Though I think the formal interface semantics could be 
> maintained, I have doubts
> >>>>> about compatibility and its unintended consequences on existing 
> subclasses,
> >>>>> applications and libraries.
> >>>>>
> >>>>> $.02, Roger
> >>>>>
> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
> >>>>>> Off the top of my head, I would say it's not possible to change 
> the design of an
> >>>>>> _extensible_ type that has been out there for 20 or so years. 
> All these I/O
> >>>>>> streams from java.io <http://java.io> were designed for simple 
> synchronous use case.
> >>>>>>
> >>>>>> It's not that their design is flawed in some way, it's that 
> they doesn't seem to
> >>>>>> suit your needs. Have you considered using 
> java.nio.channels.AsynchronousFileChannel
> >>>>>> in your applications?
> >>>>>>
> >>>>>> -Pavel
> >>>>>>
> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
> >>>>>>>
> >>>>>>> Any feedback on this? I'm really interested in implementing 
> such BufferedReader/BufferedStreamReader to allow speeding up my 
> applications without having to think in an asynchronous way or 
> multi-threading while programming with it.
> >>>>>>>
> >>>>>>> That's why I'm asking this here.
> >>>>>>>
> >>>>>>>
> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I looked at BufferedReader source code for java 9 long with 
> the source code of the channels/streams used. I noticed that, like in 
> java 7, BufferedReader does not use an Async API to load data from 
> files, instead, the data loading is all done synchronously even when 
> the OS allows requesting a file to be read and getting a warning later 
> when the file is effectively read.
> >>>>>>>>
> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
> >>>>>>>>
> >
>
>
>


From brunoaiss at gmail.com  Wed Oct 26 08:30:24 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Wed, 26 Oct 2016 09:30:24 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
Message-ID: <9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>

Hey guys. Any idea where I can find instructions on how to use JMH to:

 1. Clear OS' file reading cache.
 2. Warm up whatever it needs to (maybe reading from a Channel in memory).
 3. Create a BufferedInputStream with a FileInputStream inside, with
    configurable buffer sizes.
 4. Execute iterations to read the file fully.
     1. Allow setting the byte[] size.
     2. On each iteration, burn a set number of CPU cycles.
 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
    FileChannel.

So far I still can't find how to:

1 (clear OS' cache)
3 (the configuration part)
4 (variable number of iterations)
4.1 (the configuration)

Can someone please point me in the right direction?


On 26/10/2016 07:57, Brunoais wrote:
>
> Hey Bernd!
>
> I don't know how far back you did such thing but I'm getting positive 
> results with my non-JMH tests. I do have to evaluate my results 
> against logic. After some reads, the OS starts caching the file which 
> is not what I want. It's easy to know when that happens, though. The 
> times fall from ~30s to ~5s and the HDD keeps near idle reading (just 
> looking at the LED is enough to understand).
>
> If you don't test synchronous work and you only run the reads, you 
> will only get marginal results as the OS has no real time to fill the 
> buffer.
> My research shows the 2 major kernels (windows' and GNU/Linux) have 
> non-blocking user-level buffer handling where I give a buffer for the 
> OS to read and it keeps filling it and sending messages/signals as it 
> writes chunks. Linux has an OS interrupt that only sends the signal 
> after it is full, though. There's also another version of them where 
> they use an internal buffer of same size as the buffer you allocate 
> for the OS and then internally call memcopy() into your user-level 
> memory when asked. Tests on the internet show that memcopy is as fast 
> (for 0-1 elements) or faster than System.arraycopy(). I have no idea 
> if they are true.
>
> All this was for me to add that, that code is tuned to copy from the 
> read buffer only when it is, at least, at half capacity and the 
> internal buffer has enough storage space. The process is forced only 
> if nothing had been read on the previous fill() call. It is built to 
> use JNI as little as possible while providing the major contract 
> BufferedInputStream has.
> Finally, I never, ever compact the read buffer. It requires doing a 
> memcopy which is definitely not necessary.
>
> Anyway, those tests about time I made were just to get an order of 
> magnitude about speed difference. I intended to do them differently 
> but JMH looks good so I'll use JMH to test now.
>
> Short reads only happen when fill(true) is called. That happens for 
> desperate get of data.
>
> I'll look into the avoiding double reading requests. I do think it 
> won't bring significant improvements if any at all. It only happens 
> when the buffer is nearly empty and any byte of data is welcome "at 
> any cost".
> Besides, whomever called read at that point would also have had an 
> availability() of 0 and still called read()/read(byte[]).
>
>
> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>  Hallo Brunoais,
>>
>> In the past I die some experiments with non-blocking file channels in 
>> the hope to increase throughput in a similiar way then your buffered 
>> stream. I also used direct allocated buffers. However my results have 
>> not been that encouraging (especially If a upper layer used larger 
>> reads). I thought back in the time this was mostly die to the fact 
>> that it NOT wraps to real AsyncFIO on most platforms. But maybe I 
>> just measured it wrong, so I will have a closer look on your impl.
>>
>> Generally I would recommend to make the Benchmark a bit more reliable 
>> with JMH and in order to do this to externalize the direct buffer 
>> allocation (as it ist slow if done repeatingly). This also allows you 
>> to publish some results with varrying workloads (on different machines).
>>
>> I would also measure the readCount to see if short reads happen.
>>
>>  BTW, I might as well try to only read till the end of the buffer in 
>> the backfilling-wraps-around case and not issue two requests, that 
>> might remove some additional latency.
>>
>> Gruss
>> Bernd
>> -- 
>> http://bernd.eckenfels.net
>>
>> _____________________________
>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>> Sent: Montag, Oktober 24, 2016 6:30 PM
>> Subject: Re: Request/discussion: BufferedReader reading using async 
>> API while providing sync API
>> To: Pavel Rappo <pavel.rappo at oracle.com <mailto:pavel.rappo at oracle.com>>
>> Cc: <core-libs-dev at openjdk.java.net 
>> <mailto:core-libs-dev at openjdk.java.net>>
>>
>>
>> Attached and sending!
>>
>>
>> On 24/10/2016 13:48, Pavel Rappo wrote:
>> > Could you please send a new email on this list with the source 
>> attached as a
>> > text file?
>> >
>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>> <mailto:brunoaiss at gmail.com>> wrote:
>> >>
>> >> Here's my poc/prototype:
>> >> http://pastebin.com/WRpYWDJF
>> >>
>> >> I've implemented the bare minimum of the class that follows the 
>> same contract of BufferedReader while signaling all issues I think it 
>> may have or has in comments.
>> >> I also wrote some javadoc to help guiding through the class.
>> >>
>> >> I could have used more fields from BufferedReader but the names 
>> were so minimalistic that were confusing me. I intent to change them 
>> before sending this to openJDK.
>> >>
>> >> One of the major problems this has is long overflowing. It is 
>> major because it is hidden, it will be extremely rare and it takes a 
>> really long time to reproduce. There are different ways of dealing 
>> with it. From just documenting to actually making code that works 
>> with it.
>> >>
>> >> I built a simple test code for it to have some ideas about 
>> performance and correctness.
>> >>
>> >> http://pastebin.com/eh6LFgwT
>> >>
>> >> This doesn't do a through test if it is actually working correctly 
>> but I see no reason for it not working correctly after fixing the 2 
>> bugs that test found.
>> >>
>> >> I'll also leave here some conclusions about speed and resource 
>> consumption I found.
>> >>
>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>> 500_000B. I noticed that, with my hardware, with the 1 530 000 000B 
>> file, I was getting around:
>> >>
>> >> In all buffers and fake work: 10~15s speed improvement ( from 90% 
>> HDD speed to 100% HDD speed)
>> >> In all buffers and no fake work: 1~2s speed improvement ( from 90% 
>> HDD speed to 100% HDD speed)
>> >>
>> >> Changing the buffer size was giving different reading speeds but 
>> both were quite equal in how much they would change when changing the 
>> buffer size.
>> >> Finally, I could always confirm that I/O was always the slowest 
>> thing while this code was running.
>> >>
>> >> For the ones wondering about the file size; it is both to avoid OS 
>> cache and to make the reading at the main use-case these objects are 
>> for (large streams of bytes).
>> >>
>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>> >>
>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>> >>> Just to append to my previous email. BufferedReader wraps any 
>> Reader out there.
>> >>> Not specifically FileReader. While you're talking about the case 
>> of effective
>> >>> reading from a file.
>> >>>
>> >>> I guess there's one existing possibility to provide exactly what 
>> you need (as I
>> >>> understand it) under this method:
>> >>>
>> >>> /**
>> >>> * Opens a file for reading, returning a {@code BufferedReader} to 
>> read text
>> >>> * from the file in an efficient manner...
>> >>> ...
>> >>> */
>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>> >>>
>> >>> It can return _anything_ as long as it is a BufferedReader. We 
>> can do it, but it
>> >>> needs to be investigated not only for your favorite OS but for 
>> other OSes as
>> >>> well. Feel free to prototype this and we can discuss it on the 
>> list later.
>> >>>
>> >>> Thanks,
>> >>> -Pavel
>> >>>
>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>> <mailto:brunoaiss at gmail.com>> wrote:
>> >>>>
>> >>>> Pavel is right.
>> >>>>
>> >>>> In reality, I was expecting such BufferedReader to use only a 
>> single buffer and have that Buffer being filled asynchronously, not 
>> in a different Thread.
>> >>>> Additionally, I don't have the intention of having a larger 
>> buffer than before unless stated through the API (the constructor).
>> >>>>
>> >>>> In my idea, internally, it is supposed to use 
>> java.nio.channels.AsynchronousFileChannel or equivalent.
>> >>>>
>> >>>> It does not prevent having two buffers and I do not intent to 
>> change BufferedReader itself. I'd do an BufferedAsyncReader of sorts 
>> (any name suggestion is welcome as I'm an awful namer).
>> >>>>
>> >>>>
>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>> >>>>> Hi Pavel,
>> >>>>>
>> >>>>> I think Brunoais asking for a double buffering scheme in which 
>> the implementation of
>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>> application reading from the 1st buffer
>> >>>>> and managing the swaps and async reads transparently.
>> >>>>> It would not change the API but would change the interactions 
>> between the buffered reader
>> >>>>> and the underlying stream. It would also increase memory 
>> requirements and processing
>> >>>>> by introducing or using a separate thread and the necessary 
>> synchronization.
>> >>>>>
>> >>>>> Though I think the formal interface semantics could be 
>> maintained, I have doubts
>> >>>>> about compatibility and its unintended consequences on existing 
>> subclasses,
>> >>>>> applications and libraries.
>> >>>>>
>> >>>>> $.02, Roger
>> >>>>>
>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>> >>>>>> Off the top of my head, I would say it's not possible to 
>> change the design of an
>> >>>>>> _extensible_ type that has been out there for 20 or so years. 
>> All these I/O
>> >>>>>> streams from java.io <http://java.io> were designed for simple 
>> synchronous use case.
>> >>>>>>
>> >>>>>> It's not that their design is flawed in some way, it's that 
>> they doesn't seem to
>> >>>>>> suit your needs. Have you considered using 
>> java.nio.channels.AsynchronousFileChannel
>> >>>>>> in your applications?
>> >>>>>>
>> >>>>>> -Pavel
>> >>>>>>
>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>> <mailto:brunoaiss at gmail.com>> wrote:
>> >>>>>>>
>> >>>>>>> Any feedback on this? I'm really interested in implementing 
>> such BufferedReader/BufferedStreamReader to allow speeding up my 
>> applications without having to think in an asynchronous way or 
>> multi-threading while programming with it.
>> >>>>>>>
>> >>>>>>> That's why I'm asking this here.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>> >>>>>>>> Hi,
>> >>>>>>>>
>> >>>>>>>> I looked at BufferedReader source code for java 9 long with 
>> the source code of the channels/streams used. I noticed that, like in 
>> java 7, BufferedReader does not use an Async API to load data from 
>> files, instead, the data loading is all done synchronously even when 
>> the OS allows requesting a file to be read and getting a warning 
>> later when the file is effectively read.
>> >>>>>>>>
>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>> >>>>>>>>
>> >
>>
>>
>>
>


From pavel.rappo at oracle.com  Wed Oct 26 10:57:11 2016
From: pavel.rappo at oracle.com (Pavel Rappo)
Date: Wed, 26 Oct 2016 11:57:11 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
Message-ID: <A495710C-9F61-4045-AFF9-88512510F008@oracle.com>

I believe I see where you coming from. Please correct me if I'm wrong. 

Your implementation is based on the premise that a call to ReadableByteChannel.read()
_initiates_ the operation and returns immediately. The OS then continues to fill
the buffer while there's a free space in the buffer and the channel hasn't encountered EOF.

Is that right?

> On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com> wrote:
> 
> Thank you for your time. I'll try to explain it. I hope I can clear it up.
> First of it, I made a meaning mistake between asynchronous and non-blocking. This implementation uses a non-blocking algorithm internally while providing a blocking-like algorithm on the surface. It is single-threaded and not multi-threaded where one thread fetches data and blocks waiting and the other accumulates it and provides to whichever wants it.
> 
> Second of it, I had made a mistake of going after BufferedReader instead of going after BufferedInputStream. If you want me to go after BufferedReader it's ok but I only thought that going after BufferedInputStream would be more generically useful than BufferedReaderwhen I started the poc.
> 
> On to my code: 
> Short answers: 
> 	? The sleep(int) exists because I don't know how to wait until more data exists in the buffer which is part of read()'s contract.
> 	? The ByteBuffer gives a buffer that is filled by the OS (what I believe Channels do) instead of getting data only         by demand (what I believe Streams do).
> Full answers: 
> The blockingFill(boolean) method is a method for a busy wait for a fill which is used exclusively by the read() method. All other methods use the version that does not sleep (fill(boolean)).
> blockingFill(boolean)'s existance like that is only because the read() method must not return unless either:
> 
> 	? The stream ended.
> 	? The next byte is ready for reading.
> Additionally, statistically, that while loop will rarely evaluate to true as reads are in chunks so readPos will be behind writePos most of the time.
> I have no idea if an interrupt will ever happen, to be honest. The main reasons why I'm using a sleep is because I didn't want a hog onto the CPU in a full thread usage busy wait and because I didn't find any way of doing a thread sleep in order to wake up later when the buffer managed by native code has more data.
> The Non-blocking part is managed by the buffer the OS keeps filling most if not all the time. That buffer is the field
> 
> ByteBuffer readBuffer 
> That's the gaining part against the plain old Buffered classes.
> 
> 
> Did that make sense to you? Feel free to ask anything else you need.
> 
> On 25/10/2016 20:52, Pavel Rappo wrote:
>> I've skimmed through the code and I'm not sure I can see any asynchronicity
>> (you were pointing at the lack of it in BufferedReader).
>> And the mechanics of this is very puzzling to me, to be honest:
>>     void blockingFill(boolean forced) throws IOException {
>>         fill(forced);
>>         while (readPos == writePos) {
>>             try {
>>                 Thread.sleep(100);
>>             } catch (InterruptedException e) {
>>                 // An interrupt may mean more data is available
>>             }
>>             fill(forced);
>>         }
>>     }
>> I thought you were suggesting that we should utilize the tools which OS provides
>> more efficiently. Instead we have something that looks very similarly to a
>> "busy loop" and... also who and when is supposed to interrupt Thread.sleep()?
>> Sorry, I'm not following. Could you please explain how this is supposed to work?
>> 
>>> On 24 Oct 2016, at 15:59, Brunoais <brunoaiss at gmail.com>
>>>  wrote:
>>> Attached and sending!
>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>> 
>>>> Could you please send a new email on this list with the source attached as a
>>>> text file?
>>>> 
>>>>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com>
>>>>>  wrote:
>>>>> Here's my poc/prototype:
>>>>> 
>>>>> http://pastebin.com/WRpYWDJF
>>>>> 
>>>>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>>>>> I also wrote some javadoc to help guiding through the class.
>>>>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>>>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>>>> I built a simple test code for it to have some ideas about performance and correctness.
>>>>> 
>>>>> http://pastebin.com/eh6LFgwT
>>>>> 
>>>>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>>>> I'll also leave here some conclusions about speed and resource consumption I found.
>>>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>>>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>>>>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>>>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>> 
>>>>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>>>>> Not specifically FileReader. While you're talking about the case of effective
>>>>>> reading from a file.
>>>>>> I guess there's one existing possibility to provide exactly what you need (as I
>>>>>> understand it) under this method:
>>>>>> /**
>>>>>>  * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>>>>  * from the file in an efficient manner...
>>>>>>    ...
>>>>>>  */
>>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>>>>> needs to be investigated not only for your favorite OS but for other OSes as
>>>>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>>>> Thanks,
>>>>>> -Pavel
>>>>>> 
>>>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com>
>>>>>>>  wrote:
>>>>>>> Pavel is right.
>>>>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>>> 
>>>>>>>> Hi Pavel,
>>>>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>>>>> and managing the swaps and async reads transparently.
>>>>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>>>>> applications and libraries.
>>>>>>>> $.02, Roger
>>>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>>> 
>>>>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>>>>> in your applications?
>>>>>>>>> -Pavel
>>>>>>>>> 
>>>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com>
>>>>>>>>>>  wrote:
>>>>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>>>> That's why I'm asking this here.
>>>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>>>> 
>>> <BufferedNonBlockStream.java><Tests.java>
>>> 
> 


From claes.redestad at oracle.com  Wed Oct 26 12:11:52 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 26 Oct 2016 14:11:52 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
Message-ID: <1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>

Hi,

does this commentary suffice?

http://cr.openjdk.java.net/~redestad/8168640/webrev.02/

To answer Paul's question about the exact startup gain:

20 named classes observable with -Xlog:class+load (11
additional anonymous show up in heap dumps), for example:

   java.lang.invoke.MethodType$ConcurrentWeakInternSet
   java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
   java.lang.invoke.MethodTypeForm
   java.lang.invoke.VarHandle$1
   java.lang.invoke.VarHandle$AccessDescriptor
   java.lang.invoke.VarHandle$AccessMode
   java.lang.invoke.VarHandle$AccessType
   java.lang.invoke.VarHandleGuards
   java.lang.invoke.VarHandleInts$FieldInstanceReadOnly
   java.lang.invoke.VarHandleInts$FieldInstanceReadWrite
   java.lang.invoke.VarHandles
   java.lang.invoke.VarHandle$TypesAndInvokers

Not initializing this eagerly drops retained heap on a minimal
Hello World by ~17Kb, and a 3-10ms startup improvement
on similar programs (naturally varies a lot between systems
due to timing of when/if JIT compilations happen during early
execution).

Thanks!

/Claes

On 2016-10-25 15:21, Peter Levart wrote:
>
>
>
> On 10/25/2016 02:57 PM, Claes Redestad wrote:
>>> FileChannelImpl.close() may be idempotent, but why not making sure 
>>> it is called just once?
>>
>> Hmm, it would seem like a pre-existing issue that was not dealt with 
>> neither before nor after JDK-8025619, no?
>>
>> And FileChannel inherits AbstractInterruptibleChannel::close() 
>> (public final), which specifies behavior: "If the channel has already 
>> been closed then this method returns immediately." Thus I don't think 
>> the extra ceremony is warranted, won't you agree?
>>
>> Thanks!
>>
>> /Claes 
>
> Ok, then what about a hint in the form of a brief comment that there 
> is a race that might invoke FileChannelImpl.close() twice, but it is 
> harmless as FileChannelImpl.close() is idempotent? It might be helpful 
> to a future bug-hunter...
>
> Regards, Peter
>


From brunoaiss at gmail.com  Wed Oct 26 12:42:06 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Wed, 26 Oct 2016 13:42:06 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
Message-ID: <10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>

It is actually based on the premise that:

 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS
    buffer size to fill in as the same size as ByteBuffer.
 2. The consecutive calls to ReadableByteChannel.read(ByteBuffer) orders
    the JVM to order the OS to execute memcpy() to copy from its memory
    to the shared memory created at ByteBuffer instantiation (in java 8)
    using Unsafe and then for the JVM to update the ByteBuffer fields.
 3. The call will not block waiting for I/O and it won't take longer
    than the JNI interface if no new data exists. However, it will block
    waiting for the OS to execute memcpy() to the shared memory.

Is my premise wrong?

If I read correctly, if I don't use a DirectBuffer, there would be even 
another intermediate buffer to copy data to before giving it to the 
"user" which would be useless.


On 26/10/2016 11:57, Pavel Rappo wrote:
> I believe I see where you coming from. Please correct me if I'm wrong.
>
> Your implementation is based on the premise that a call to ReadableByteChannel.read()
> _initiates_ the operation and returns immediately. The OS then continues to fill
> the buffer while there's a free space in the buffer and the channel hasn't encountered EOF.
>
> Is that right?
>
>> On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com> wrote:
>>
>> Thank you for your time. I'll try to explain it. I hope I can clear it up.
>> First of it, I made a meaning mistake between asynchronous and non-blocking. This implementation uses a non-blocking algorithm internally while providing a blocking-like algorithm on the surface. It is single-threaded and not multi-threaded where one thread fetches data and blocks waiting and the other accumulates it and provides to whichever wants it.
>>
>> Second of it, I had made a mistake of going after BufferedReader instead of going after BufferedInputStream. If you want me to go after BufferedReader it's ok but I only thought that going after BufferedInputStream would be more generically useful than BufferedReaderwhen I started the poc.
>>
>> On to my code:
>> Short answers:
>> 	? The sleep(int) exists because I don't know how to wait until more data exists in the buffer which is part of read()'s contract.
>> 	? The ByteBuffer gives a buffer that is filled by the OS (what I believe Channels do) instead of getting data only         by demand (what I believe Streams do).
>> Full answers:
>> The blockingFill(boolean) method is a method for a busy wait for a fill which is used exclusively by the read() method. All other methods use the version that does not sleep (fill(boolean)).
>> blockingFill(boolean)'s existance like that is only because the read() method must not return unless either:
>>
>> 	? The stream ended.
>> 	? The next byte is ready for reading.
>> Additionally, statistically, that while loop will rarely evaluate to true as reads are in chunks so readPos will be behind writePos most of the time.
>> I have no idea if an interrupt will ever happen, to be honest. The main reasons why I'm using a sleep is because I didn't want a hog onto the CPU in a full thread usage busy wait and because I didn't find any way of doing a thread sleep in order to wake up later when the buffer managed by native code has more data.
>> The Non-blocking part is managed by the buffer the OS keeps filling most if not all the time. That buffer is the field
>>
>> ByteBuffer readBuffer
>> That's the gaining part against the plain old Buffered classes.
>>
>>
>> Did that make sense to you? Feel free to ask anything else you need.
>>
>> On 25/10/2016 20:52, Pavel Rappo wrote:
>>> I've skimmed through the code and I'm not sure I can see any asynchronicity
>>> (you were pointing at the lack of it in BufferedReader).
>>> And the mechanics of this is very puzzling to me, to be honest:
>>>      void blockingFill(boolean forced) throws IOException {
>>>          fill(forced);
>>>          while (readPos == writePos) {
>>>              try {
>>>                  Thread.sleep(100);
>>>              } catch (InterruptedException e) {
>>>                  // An interrupt may mean more data is available
>>>              }
>>>              fill(forced);
>>>          }
>>>      }
>>> I thought you were suggesting that we should utilize the tools which OS provides
>>> more efficiently. Instead we have something that looks very similarly to a
>>> "busy loop" and... also who and when is supposed to interrupt Thread.sleep()?
>>> Sorry, I'm not following. Could you please explain how this is supposed to work?
>>>
>>>> On 24 Oct 2016, at 15:59, Brunoais <brunoaiss at gmail.com>
>>>>   wrote:
>>>> Attached and sending!
>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>
>>>>> Could you please send a new email on this list with the source attached as a
>>>>> text file?
>>>>>
>>>>>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com>
>>>>>>   wrote:
>>>>>> Here's my poc/prototype:
>>>>>>
>>>>>> http://pastebin.com/WRpYWDJF
>>>>>>
>>>>>> I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments.
>>>>>> I also wrote some javadoc to help guiding through the class.
>>>>>> I could have used more fields from BufferedReader but the names were so minimalistic that were confusing me. I intent to change them before sending this to openJDK.
>>>>>> One of the major problems this has is long overflowing. It is major because it is hidden, it will be extremely rare and it takes a really long time to reproduce. There are different ways of dealing with it. From just documenting to actually making code that works with it.
>>>>>> I built a simple test code for it to have some ideas about performance and correctness.
>>>>>>
>>>>>> http://pastebin.com/eh6LFgwT
>>>>>>
>>>>>> This doesn't do a through test if it is actually working correctly but I see no reason for it not working correctly after fixing the 2 bugs that test found.
>>>>>> I'll also leave here some conclusions about speed and resource consumption I found.
>>>>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I noticed that, with my hardware, with the 1 530 000 000B file, I was getting around:
>>>>>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>>>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD speed to 100% HDD speed)
>>>>>> Changing the buffer size was giving different reading speeds but both were quite equal in how much they would change when changing the buffer size.
>>>>>> Finally, I could always confirm that I/O was always the slowest thing while this code was running.
>>>>>> For the ones wondering about the file size; it is both to avoid OS cache and to make the reading at the main use-case these objects are for (large streams of bytes).
>>>>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>>>
>>>>>>> Just to append to my previous email. BufferedReader wraps any Reader out there.
>>>>>>> Not specifically FileReader. While you're talking about the case of effective
>>>>>>> reading from a file.
>>>>>>> I guess there's one existing possibility to provide exactly what you need (as I
>>>>>>> understand it) under this method:
>>>>>>> /**
>>>>>>>   * Opens a file for reading, returning a {@code BufferedReader} to read text
>>>>>>>   * from the file in an efficient manner...
>>>>>>>     ...
>>>>>>>   */
>>>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>>> It can return _anything_ as long as it is a BufferedReader. We can do it, but it
>>>>>>> needs to be investigated not only for your favorite OS but for other OSes as
>>>>>>> well. Feel free to prototype this and we can discuss it on the list later.
>>>>>>> Thanks,
>>>>>>> -Pavel
>>>>>>>
>>>>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com>
>>>>>>>>   wrote:
>>>>>>>> Pavel is right.
>>>>>>>> In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread.
>>>>>>>> Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor).
>>>>>>>> In my idea, internally, it is supposed to use java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>>>> It does not prevent having two buffers and I do not intent to change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name suggestion is welcome as I'm an awful namer).
>>>>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>>>>
>>>>>>>>> Hi Pavel,
>>>>>>>>> I think Brunoais asking for a double buffering scheme in which the implementation of
>>>>>>>>> BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer
>>>>>>>>> and managing the swaps and async reads transparently.
>>>>>>>>> It would not change the API but would change the interactions between the buffered reader
>>>>>>>>> and the underlying stream.  It would also increase memory requirements and processing
>>>>>>>>> by introducing or using a separate thread and the necessary synchronization.
>>>>>>>>> Though I think the formal interface semantics could be maintained, I have doubts
>>>>>>>>> about compatibility and its unintended consequences on existing subclasses,
>>>>>>>>> applications and libraries.
>>>>>>>>> $.02, Roger
>>>>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>>>>
>>>>>>>>>> Off the top of my head, I would say it's not possible to change the design of an
>>>>>>>>>> _extensible_ type that has been out there for 20 or so years. All these I/O
>>>>>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>>>>> It's not that their design is flawed in some way, it's that they doesn't seem to
>>>>>>>>>> suit your needs. Have you considered using java.nio.channels.AsynchronousFileChannel
>>>>>>>>>> in your applications?
>>>>>>>>>> -Pavel
>>>>>>>>>>
>>>>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com>
>>>>>>>>>>>   wrote:
>>>>>>>>>>> Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it.
>>>>>>>>>>> That's why I'm asking this here.
>>>>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requesting a file to be read and getting a warning later when the file is effectively read.
>>>>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>>>>>
>>>> <BufferedNonBlockStream.java><Tests.java>
>>>>
>


From daniel.fuchs at oracle.com  Wed Oct 26 13:58:28 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Wed, 26 Oct 2016 14:58:28 +0100
Subject: RFR: 8163162: The separation between system loggers and application
	loggers should take the extension loader in consideration.
Message-ID: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>

Hi,

Please find below a small patch for

8163162: The separation between system loggers and application
          loggers should take the extension loader in consideration.
https://bugs.openjdk.java.net/browse/JDK-8163162

With the deprivileging of some JDK modules, classes loaded
by the Platform class loader should get the same kind of
loggers than classes loaded by the Boot class loader (null loader).

http://cr.openjdk.java.net/~dfuchs/webrev_8163162/webrev.00/

best regards,

-- daniel

From Roger.Riggs at Oracle.com  Wed Oct 26 14:24:03 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 26 Oct 2016 10:24:03 -0400
Subject: RFR 9: [testbug] 8168613 : CORBA ObjectStreamTest fails with
	address in use
In-Reply-To: <430A8113-E183-479C-A461-9BF7D088463B@oracle.com>
References: <0f179652-a2dd-aae8-24e6-34a7212526cf@Oracle.com>
	<9C0D0AFB-5221-4C0D-BA43-94D9D4D7F849@oracle.com>
	<d9118bd1-43be-e248-e200-4786f470c2a3@Oracle.com>
	<72FFE4AE-EE52-4074-9878-DF3F53C02BDA@oracle.com>
	<3b884480-ed7e-9d45-008b-27293a0c320b@Oracle.com>
	<430A8113-E183-479C-A461-9BF7D088463B@oracle.com>
Message-ID: <74354073-4d13-8883-7803-4a5491ae3ab4@Oracle.com>

Thanks,  Fixed and pushed


On 10/25/2016 5:35 PM, Paul Sandoz wrote:
>> On 25 Oct 2016, at 14:28, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>>
>> Hi Paul,
>>
>> Thanks for explanation and suggestions.
>>
>> I updated the webrev to use instance methods and fields and using synchronized.
>> (And a bit of long-line cleanup.)
>>
>> webrev:
>>    http://cr.openjdk.java.net/~rriggs/webrev-orb-addrinuse-8168613/
>>
> You have a rogue @BeforeClass:
>   202     @BeforeClass
>   203     synchronized Echo getEchoStub() throws RemoteException {
>
> Otherwise looks good.
>
> Paul.


From peter.levart at gmail.com  Wed Oct 26 15:00:43 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 26 Oct 2016 17:00:43 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
	<1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
Message-ID: <f1461a6b-5dd9-2479-c500-bf0c5a50aaca@gmail.com>



On 10/26/2016 02:11 PM, Claes Redestad wrote:
> does this commentary suffice?
>
> http://cr.openjdk.java.net/~redestad/8168640/webrev.02/

Yes, that's good. Thanks.

Regards, Peter


From Roger.Riggs at Oracle.com  Wed Oct 26 15:06:17 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 26 Oct 2016 11:06:17 -0400
Subject: RFR: 8163162: The separation between system loggers and
	application loggers should take the extension loader in consideration.
In-Reply-To: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
References: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
Message-ID: <593e8adf-de3c-252b-20ad-b54863b5db91@Oracle.com>

Hi Daniel,

It is common now to use a lambda for the target of doPriv; its a bit 
lighter weight
as it becomes a method reference instead of an additional inner class.

Looks fine with or without the lambda.

Roger


On 10/26/2016 9:58 AM, Daniel Fuchs wrote:
> Hi,
>
> Please find below a small patch for
>
> 8163162: The separation between system loggers and application
>          loggers should take the extension loader in consideration.
> https://bugs.openjdk.java.net/browse/JDK-8163162
>
> With the deprivileging of some JDK modules, classes loaded
> by the Platform class loader should get the same kind of
> loggers than classes loaded by the Boot class loader (null loader).
>
> http://cr.openjdk.java.net/~dfuchs/webrev_8163162/webrev.00/
>
> best regards,
>
> -- daniel


From Alan.Bateman at oracle.com  Wed Oct 26 15:10:26 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 26 Oct 2016 16:10:26 +0100
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
	<1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
Message-ID: <df7f0eac-83f2-b917-a538-0f7f26c58250@oracle.com>

On 26/10/2016 13:11, Claes Redestad wrote:

> Hi,
>
> does this commentary suffice?
>
> http://cr.openjdk.java.net/~redestad/8168640/webrev.02/
>
> To answer Paul's question about the exact startup gain:
>
> 20 named classes observable with -Xlog:class+load (11
> additional anonymous show up in heap dumps), for example:
>
>   java.lang.invoke.MethodType$ConcurrentWeakInternSet
>   java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
>   java.lang.invoke.MethodTypeForm
>   java.lang.invoke.VarHandle$1
>   java.lang.invoke.VarHandle$AccessDescriptor
>   java.lang.invoke.VarHandle$AccessMode
>   java.lang.invoke.VarHandle$AccessType
>   java.lang.invoke.VarHandleGuards
>   java.lang.invoke.VarHandleInts$FieldInstanceReadOnly
>   java.lang.invoke.VarHandleInts$FieldInstanceReadWrite
>   java.lang.invoke.VarHandles
>   java.lang.invoke.VarHandle$TypesAndInvokers
>
> Not initializing this eagerly drops retained heap on a minimal
> Hello World by ~17Kb, and a 3-10ms startup improvement
> on similar programs (naturally varies a lot between systems
> due to timing of when/if JIT compilations happen during early
> execution).
I assume these classes will be loaded by anything non-trivial but having 
them but keeping them out of hello world/short running tools is good.

The updated webrev looks okay to me.

-Alan





From claes.redestad at oracle.com  Wed Oct 26 15:59:47 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 26 Oct 2016 17:59:47 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <df7f0eac-83f2-b917-a538-0f7f26c58250@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
	<1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
	<df7f0eac-83f2-b917-a538-0f7f26c58250@oracle.com>
Message-ID: <25202e23-0c4b-6ad3-cf5f-d4ffdffa8db1@oracle.com>

On 2016-10-26 17:10, Alan Bateman wrote:
> On 26/10/2016 13:11, Claes Redestad wrote:
>
>> Hi,
>>
>> does this commentary suffice?
>>
>> http://cr.openjdk.java.net/~redestad/8168640/webrev.02/
>>
>> To answer Paul's question about the exact startup gain:
>>
>> 20 named classes observable with -Xlog:class+load (11
>> additional anonymous show up in heap dumps), for example:
>>
>>   java.lang.invoke.MethodType$ConcurrentWeakInternSet
>>   java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
>>   java.lang.invoke.MethodTypeForm
>>   java.lang.invoke.VarHandle$1
>>   java.lang.invoke.VarHandle$AccessDescriptor
>>   java.lang.invoke.VarHandle$AccessMode
>>   java.lang.invoke.VarHandle$AccessType
>>   java.lang.invoke.VarHandleGuards
>>   java.lang.invoke.VarHandleInts$FieldInstanceReadOnly
>>   java.lang.invoke.VarHandleInts$FieldInstanceReadWrite
>>   java.lang.invoke.VarHandles
>>   java.lang.invoke.VarHandle$TypesAndInvokers
>>
>> Not initializing this eagerly drops retained heap on a minimal
>> Hello World by ~17Kb, and a 3-10ms startup improvement
>> on similar programs (naturally varies a lot between systems
>> due to timing of when/if JIT compilations happen during early
>> execution).
> I assume these classes will be loaded by anything non-trivial but 
> having them but keeping them out of hello world/short running tools is 
> good.

Right, making things lazier helps small programs, additionally there's 
potential long-term value in reducing the dependency graph to system 
initialization.

>
> The updated webrev looks okay to me.

Thanks for reviewing!

/Claes

>
> -Alan
>
>
>
>


From daniel.fuchs at oracle.com  Wed Oct 26 16:15:48 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Wed, 26 Oct 2016 17:15:48 +0100
Subject: RFR: 8163162: The separation between system loggers and
	application loggers should take the extension loader in consideration.
In-Reply-To: <593e8adf-de3c-252b-20ad-b54863b5db91@Oracle.com>
References: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
	<593e8adf-de3c-252b-20ad-b54863b5db91@Oracle.com>
Message-ID: <7e8ac56e-a7f6-e171-dd9d-edf1a8c4f946@oracle.com>

Hi Roger,

On 26/10/16 16:06, Roger Riggs wrote:
> Hi Daniel,
>
> It is common now to use a lambda for the target of doPriv; its a bit
> lighter weight
> as it becomes a method reference instead of an additional inner class.

Yes - I've been bitten before with using lambda
in logging code - especially in those parts that
can be invoked early during platform class
initialization - so I tend to avoid using them
in places that are in the code path triggered
before the full initialization of the logging
system.

> Looks fine with or without the lambda.

Thanks!

-- daniel

>
> Roger
>
>
> On 10/26/2016 9:58 AM, Daniel Fuchs wrote:
>> Hi,
>>
>> Please find below a small patch for
>>
>> 8163162: The separation between system loggers and application
>>          loggers should take the extension loader in consideration.
>> https://bugs.openjdk.java.net/browse/JDK-8163162
>>
>> With the deprivileging of some JDK modules, classes loaded
>> by the Platform class loader should get the same kind of
>> loggers than classes loaded by the Boot class loader (null loader).
>>
>> http://cr.openjdk.java.net/~dfuchs/webrev_8163162/webrev.00/
>>
>> best regards,
>>
>> -- daniel
>


From peter.levart at gmail.com  Wed Oct 26 17:24:04 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 26 Oct 2016 19:24:04 +0200
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
Message-ID: <229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>

Hi Brunoais,

I'll try to tell what I know from my JMH practice:

On 10/26/2016 10:30 AM, Brunoais wrote:
> Hey guys. Any idea where I can find instructions on how to use JMH to:
>
> 1. Clear OS' file reading cache.

You can create a public void method and make it called by JMH before each:
- trial (a set of iterations)
- iteration (a set of test method invocations)
- invocation

...simply by annotating it by @Setup( [ Level.Trial | Level.Iteration | 
Level.Invocation ] ).

So create a method that spawns a script that clears the cache.

> 2. Warm up whatever it needs to (maybe reading from a Channel in memory).

JMH already warms-up the code and VM simply be executing "warmup" 
iterations before starting real measured iterations. You can control the 
number of warm-up iterations and real measured iterations by annotating 
either the class or the method(s) with:

@Warmup(iterations = ...)
@Measurement(iterations = ...)

If you want to warm-up resources by code that is not equal to code in 
test method(s) then maybe @Setup methods on different levels could be 
used for that.

>
> 3. Create a BufferedInputStream with a FileInputStream inside, with
>    configurable buffer sizes.

You can annotate a field of int, long or String type of a class 
annotated with @State annotation (can be the benchmark class itself) 
with @Param annotation, enumerating values this field will get before 
executing the @Setup(Level.Trial) method(s). So you enumerate the buffer 
sizes in @Param annotation and instantiate the BufferedInputStream using 
the value in @Setup method. Viola.

> 4. Execute iterations to read the file fully.

Then perhaps you could use only one invocation per iteration and 
measured it using @BenchmarkMode(Mode.SingleShotTime), constructing the 
loop by yourself.

> 1. Allow setting the byte[] size.

Use @Parameter on a field to hold the byte[] size and create the byte[] 
in @Setup method...

> 2. On each iteration, burn a set number of CPU cycles.

BlackHole.consumeCPU(tokens)

> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>    FileChannel.

If you wrap them all into a common API (by delegation), you can use 
@Parameter String implType, with @Setup method to instantiate the 
appropriate implementation. Then just invoke the common API in the test 
method.

>
> So far I still can't find how to:
>
> 1 (clear OS' cache)
> 3 (the configuration part)
> 4 (variable number of iterations)
> 4.1 (the configuration)
>
> Can someone please point me in the right direction?

I can create an example test if you like and you can then extend it...

Regards, Peter

>
>
> On 26/10/2016 07:57, Brunoais wrote:
>>
>> Hey Bernd!
>>
>> I don't know how far back you did such thing but I'm getting positive 
>> results with my non-JMH tests. I do have to evaluate my results 
>> against logic. After some reads, the OS starts caching the file which 
>> is not what I want. It's easy to know when that happens, though. The 
>> times fall from ~30s to ~5s and the HDD keeps near idle reading (just 
>> looking at the LED is enough to understand).
>>
>> If you don't test synchronous work and you only run the reads, you 
>> will only get marginal results as the OS has no real time to fill the 
>> buffer.
>> My research shows the 2 major kernels (windows' and GNU/Linux) have 
>> non-blocking user-level buffer handling where I give a buffer for the 
>> OS to read and it keeps filling it and sending messages/signals as it 
>> writes chunks. Linux has an OS interrupt that only sends the signal 
>> after it is full, though. There's also another version of them where 
>> they use an internal buffer of same size as the buffer you allocate 
>> for the OS and then internally call memcopy() into your user-level 
>> memory when asked. Tests on the internet show that memcopy is as fast 
>> (for 0-1 elements) or faster than System.arraycopy(). I have no idea 
>> if they are true.
>>
>> All this was for me to add that, that code is tuned to copy from the 
>> read buffer only when it is, at least, at half capacity and the 
>> internal buffer has enough storage space. The process is forced only 
>> if nothing had been read on the previous fill() call. It is built to 
>> use JNI as little as possible while providing the major contract 
>> BufferedInputStream has.
>> Finally, I never, ever compact the read buffer. It requires doing a 
>> memcopy which is definitely not necessary.
>>
>> Anyway, those tests about time I made were just to get an order of 
>> magnitude about speed difference. I intended to do them differently 
>> but JMH looks good so I'll use JMH to test now.
>>
>> Short reads only happen when fill(true) is called. That happens for 
>> desperate get of data.
>>
>> I'll look into the avoiding double reading requests. I do think it 
>> won't bring significant improvements if any at all. It only happens 
>> when the buffer is nearly empty and any byte of data is welcome "at 
>> any cost".
>> Besides, whomever called read at that point would also have had an 
>> availability() of 0 and still called read()/read(byte[]).
>>
>>
>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>  Hallo Brunoais,
>>>
>>> In the past I die some experiments with non-blocking file channels 
>>> in the hope to increase throughput in a similiar way then your 
>>> buffered stream. I also used direct allocated buffers. However my 
>>> results have not been that encouraging (especially If a upper layer 
>>> used larger reads). I thought back in the time this was mostly die 
>>> to the fact that it NOT wraps to real AsyncFIO on most platforms. 
>>> But maybe I just measured it wrong, so I will have a closer look on 
>>> your impl.
>>>
>>> Generally I would recommend to make the Benchmark a bit more 
>>> reliable with JMH and in order to do this to externalize the direct 
>>> buffer allocation (as it ist slow if done repeatingly). This also 
>>> allows you to publish some results with varrying workloads (on 
>>> different machines).
>>>
>>> I would also measure the readCount to see if short reads happen.
>>>
>>>  BTW, I might as well try to only read till the end of the buffer in 
>>> the backfilling-wraps-around case and not issue two requests, that 
>>> might remove some additional latency.
>>>
>>> Gruss
>>> Bernd
>>> -- 
>>> http://bernd.eckenfels.net
>>>
>>> _____________________________
>>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>> Subject: Re: Request/discussion: BufferedReader reading using async 
>>> API while providing sync API
>>> To: Pavel Rappo <pavel.rappo at oracle.com 
>>> <mailto:pavel.rappo at oracle.com>>
>>> Cc: <core-libs-dev at openjdk.java.net 
>>> <mailto:core-libs-dev at openjdk.java.net>>
>>>
>>>
>>> Attached and sending!
>>>
>>>
>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>> > Could you please send a new email on this list with the source 
>>> attached as a
>>> > text file?
>>> >
>>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>>> <mailto:brunoaiss at gmail.com>> wrote:
>>> >>
>>> >> Here's my poc/prototype:
>>> >> http://pastebin.com/WRpYWDJF
>>> >>
>>> >> I've implemented the bare minimum of the class that follows the 
>>> same contract of BufferedReader while signaling all issues I think 
>>> it may have or has in comments.
>>> >> I also wrote some javadoc to help guiding through the class.
>>> >>
>>> >> I could have used more fields from BufferedReader but the names 
>>> were so minimalistic that were confusing me. I intent to change them 
>>> before sending this to openJDK.
>>> >>
>>> >> One of the major problems this has is long overflowing. It is 
>>> major because it is hidden, it will be extremely rare and it takes a 
>>> really long time to reproduce. There are different ways of dealing 
>>> with it. From just documenting to actually making code that works 
>>> with it.
>>> >>
>>> >> I built a simple test code for it to have some ideas about 
>>> performance and correctness.
>>> >>
>>> >> http://pastebin.com/eh6LFgwT
>>> >>
>>> >> This doesn't do a through test if it is actually working 
>>> correctly but I see no reason for it not working correctly after 
>>> fixing the 2 bugs that test found.
>>> >>
>>> >> I'll also leave here some conclusions about speed and resource 
>>> consumption I found.
>>> >>
>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 000B 
>>> file, I was getting around:
>>> >>
>>> >> In all buffers and fake work: 10~15s speed improvement ( from 90% 
>>> HDD speed to 100% HDD speed)
>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>> 90% HDD speed to 100% HDD speed)
>>> >>
>>> >> Changing the buffer size was giving different reading speeds but 
>>> both were quite equal in how much they would change when changing 
>>> the buffer size.
>>> >> Finally, I could always confirm that I/O was always the slowest 
>>> thing while this code was running.
>>> >>
>>> >> For the ones wondering about the file size; it is both to avoid 
>>> OS cache and to make the reading at the main use-case these objects 
>>> are for (large streams of bytes).
>>> >>
>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>> >>
>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>> Reader out there.
>>> >>> Not specifically FileReader. While you're talking about the case 
>>> of effective
>>> >>> reading from a file.
>>> >>>
>>> >>> I guess there's one existing possibility to provide exactly what 
>>> you need (as I
>>> >>> understand it) under this method:
>>> >>>
>>> >>> /**
>>> >>> * Opens a file for reading, returning a {@code BufferedReader} 
>>> to read text
>>> >>> * from the file in an efficient manner...
>>> >>> ...
>>> >>> */
>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>> >>>
>>> >>> It can return _anything_ as long as it is a BufferedReader. We 
>>> can do it, but it
>>> >>> needs to be investigated not only for your favorite OS but for 
>>> other OSes as
>>> >>> well. Feel free to prototype this and we can discuss it on the 
>>> list later.
>>> >>>
>>> >>> Thanks,
>>> >>> -Pavel
>>> >>>
>>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>>> <mailto:brunoaiss at gmail.com>> wrote:
>>> >>>>
>>> >>>> Pavel is right.
>>> >>>>
>>> >>>> In reality, I was expecting such BufferedReader to use only a 
>>> single buffer and have that Buffer being filled asynchronously, not 
>>> in a different Thread.
>>> >>>> Additionally, I don't have the intention of having a larger 
>>> buffer than before unless stated through the API (the constructor).
>>> >>>>
>>> >>>> In my idea, internally, it is supposed to use 
>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>> >>>>
>>> >>>> It does not prevent having two buffers and I do not intent to 
>>> change BufferedReader itself. I'd do an BufferedAsyncReader of sorts 
>>> (any name suggestion is welcome as I'm an awful namer).
>>> >>>>
>>> >>>>
>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>> >>>>> Hi Pavel,
>>> >>>>>
>>> >>>>> I think Brunoais asking for a double buffering scheme in which 
>>> the implementation of
>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>> application reading from the 1st buffer
>>> >>>>> and managing the swaps and async reads transparently.
>>> >>>>> It would not change the API but would change the interactions 
>>> between the buffered reader
>>> >>>>> and the underlying stream. It would also increase memory 
>>> requirements and processing
>>> >>>>> by introducing or using a separate thread and the necessary 
>>> synchronization.
>>> >>>>>
>>> >>>>> Though I think the formal interface semantics could be 
>>> maintained, I have doubts
>>> >>>>> about compatibility and its unintended consequences on 
>>> existing subclasses,
>>> >>>>> applications and libraries.
>>> >>>>>
>>> >>>>> $.02, Roger
>>> >>>>>
>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>> change the design of an
>>> >>>>>> _extensible_ type that has been out there for 20 or so years. 
>>> All these I/O
>>> >>>>>> streams from java.io <http://java.io> were designed for 
>>> simple synchronous use case.
>>> >>>>>>
>>> >>>>>> It's not that their design is flawed in some way, it's that 
>>> they doesn't seem to
>>> >>>>>> suit your needs. Have you considered using 
>>> java.nio.channels.AsynchronousFileChannel
>>> >>>>>> in your applications?
>>> >>>>>>
>>> >>>>>> -Pavel
>>> >>>>>>
>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>>> <mailto:brunoaiss at gmail.com>> wrote:
>>> >>>>>>>
>>> >>>>>>> Any feedback on this? I'm really interested in implementing 
>>> such BufferedReader/BufferedStreamReader to allow speeding up my 
>>> applications without having to think in an asynchronous way or 
>>> multi-threading while programming with it.
>>> >>>>>>>
>>> >>>>>>> That's why I'm asking this here.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>> >>>>>>>> Hi,
>>> >>>>>>>>
>>> >>>>>>>> I looked at BufferedReader source code for java 9 long with 
>>> the source code of the channels/streams used. I noticed that, like 
>>> in java 7, BufferedReader does not use an Async API to load data 
>>> from files, instead, the data loading is all done synchronously even 
>>> when the OS allows requesting a file to be read and getting a 
>>> warning later when the file is effectively read.
>>> >>>>>>>>
>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>> >>>>>>>>
>>> >
>>>
>>>
>>>
>>
>


From huizhe.wang at oracle.com  Wed Oct 26 18:29:18 2016
From: huizhe.wang at oracle.com (Joe Wang)
Date: Wed, 26 Oct 2016 11:29:18 -0700
Subject: RFR (JAXP) 8069098 : StAX produces the wrong event stream
Message-ID: <5810F5FE.4040908@oracle.com>

Hi,

Please review an update to the Javadoc of the StAX API from the JSR 173 
specification [1], and fix to the implementation to comply with the 
specification that the reader's initial event shall be START_DOCUMENT.

JBS: https://bugs.openjdk.java.net/browse/JDK-8069098
webrev: http://cr.openjdk.java.net/~joehw/jdk9/8069098/webrev/


[1] https://www.jcp.org/en/jsr/detail?id=173

Thanks,
Joe



From paul.sandoz at oracle.com  Wed Oct 26 19:38:15 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Wed, 26 Oct 2016 12:38:15 -0700
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
	<1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
Message-ID: <859EA247-ABCA-450C-8E45-5D5BAC4D4995@oracle.com>


> On 26 Oct 2016, at 05:11, Claes Redestad <claes.redestad at oracle.com> wrote:
> 
> Hi,
> 
> does this commentary suffice?
> 
> http://cr.openjdk.java.net/~redestad/8168640/webrev.02/
> 

+1


> To answer Paul's question about the exact startup gain:
> 
> 20 named classes observable with -Xlog:class+load (11
> additional anonymous show up in heap dumps), for example:
> 
>  java.lang.invoke.MethodType$ConcurrentWeakInternSet
>  java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
>  java.lang.invoke.MethodTypeForm
>  java.lang.invoke.VarHandle$1
>  java.lang.invoke.VarHandle$AccessDescriptor
>  java.lang.invoke.VarHandle$AccessMode
>  java.lang.invoke.VarHandle$AccessType
>  java.lang.invoke.VarHandleGuards
>  java.lang.invoke.VarHandleInts$FieldInstanceReadOnly
>  java.lang.invoke.VarHandleInts$FieldInstanceReadWrite
>  java.lang.invoke.VarHandles
>  java.lang.invoke.VarHandle$TypesAndInvokers
> 
> Not initializing this eagerly drops retained heap on a minimal
> Hello World by ~17Kb, and a 3-10ms startup improvement
> on similar programs (naturally varies a lot between systems
> due to timing of when/if JIT compilations happen during early
> execution).
> 

Thanks.

Do you observe that MethodHandleStatics.<clinit> can take a large proportion of the VH initialisation time?

Paul.

From brunoaiss at gmail.com  Wed Oct 26 19:46:30 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Wed, 26 Oct 2016 20:46:30 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
Message-ID: <65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>

Thank you.

Only one thing left. How can I "burn" the OS' file read cache?
I only know how to do that by allocating a very large amount of memory 
based on the information I see in the resource manager (windows) or 
system monitor (linux) of the cached I/O and run the program. In this 
case, I have no idea how much memory each one's computer has so I cannot 
use the same method. How would you do such program excerpt?

As for the rest of the pointers: thank you I'll start building the 
benchmark code based on that information.


On 26/10/2016 18:24, Peter Levart wrote:
> Hi Brunoais,
>
> I'll try to tell what I know from my JMH practice:
>
> On 10/26/2016 10:30 AM, Brunoais wrote:
>> Hey guys. Any idea where I can find instructions on how to use JMH to:
>>
>> 1. Clear OS' file reading cache.
>
> You can create a public void method and make it called by JMH before each:
> - trial (a set of iterations)
> - iteration (a set of test method invocations)
> - invocation
>
> ...simply by annotating it by @Setup( [ Level.Trial | Level.Iteration 
> | Level.Invocation ] ).
>
> So create a method that spawns a script that clears the cache.
>
>> 2. Warm up whatever it needs to (maybe reading from a Channel in memory).
>
> JMH already warms-up the code and VM simply be executing "warmup" 
> iterations before starting real measured iterations. You can control 
> the number of warm-up iterations and real measured iterations by 
> annotating either the class or the method(s) with:
>
> @Warmup(iterations = ...)
> @Measurement(iterations = ...)
>
> If you want to warm-up resources by code that is not equal to code in 
> test method(s) then maybe @Setup methods on different levels could be 
> used for that.
>
>>
>> 3. Create a BufferedInputStream with a FileInputStream inside, with
>>    configurable buffer sizes.
>
> You can annotate a field of int, long or String type of a class 
> annotated with @State annotation (can be the benchmark class itself) 
> with @Param annotation, enumerating values this field will get before 
> executing the @Setup(Level.Trial) method(s). So you enumerate the 
> buffer sizes in @Param annotation and instantiate the 
> BufferedInputStream using the value in @Setup method. Viola.
>
>> 4. Execute iterations to read the file fully.
>
> Then perhaps you could use only one invocation per iteration and 
> measured it using @BenchmarkMode(Mode.SingleShotTime), constructing 
> the loop by yourself.
>
>> 1. Allow setting the byte[] size.
>
> Use @Parameter on a field to hold the byte[] size and create the 
> byte[] in @Setup method...
>
>> 2. On each iteration, burn a set number of CPU cycles.
>
> BlackHole.consumeCPU(tokens)
>
>> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>>    FileChannel.
>
> If you wrap them all into a common API (by delegation), you can use 
> @Parameter String implType, with @Setup method to instantiate the 
> appropriate implementation. Then just invoke the common API in the 
> test method.
>
>>
>> So far I still can't find how to:
>>
>> 1 (clear OS' cache)
>> 3 (the configuration part)
>> 4 (variable number of iterations)
>> 4.1 (the configuration)
>>
>> Can someone please point me in the right direction?
>
> I can create an example test if you like and you can then extend it...
>
> Regards, Peter
>
>>
>>
>> On 26/10/2016 07:57, Brunoais wrote:
>>>
>>> Hey Bernd!
>>>
>>> I don't know how far back you did such thing but I'm getting 
>>> positive results with my non-JMH tests. I do have to evaluate my 
>>> results against logic. After some reads, the OS starts caching the 
>>> file which is not what I want. It's easy to know when that happens, 
>>> though. The times fall from ~30s to ~5s and the HDD keeps near idle 
>>> reading (just looking at the LED is enough to understand).
>>>
>>> If you don't test synchronous work and you only run the reads, you 
>>> will only get marginal results as the OS has no real time to fill 
>>> the buffer.
>>> My research shows the 2 major kernels (windows' and GNU/Linux) have 
>>> non-blocking user-level buffer handling where I give a buffer for 
>>> the OS to read and it keeps filling it and sending messages/signals 
>>> as it writes chunks. Linux has an OS interrupt that only sends the 
>>> signal after it is full, though. There's also another version of 
>>> them where they use an internal buffer of same size as the buffer 
>>> you allocate for the OS and then internally call memcopy() into your 
>>> user-level memory when asked. Tests on the internet show that 
>>> memcopy is as fast (for 0-1 elements) or faster than 
>>> System.arraycopy(). I have no idea if they are true.
>>>
>>> All this was for me to add that, that code is tuned to copy from the 
>>> read buffer only when it is, at least, at half capacity and the 
>>> internal buffer has enough storage space. The process is forced only 
>>> if nothing had been read on the previous fill() call. It is built to 
>>> use JNI as little as possible while providing the major contract 
>>> BufferedInputStream has.
>>> Finally, I never, ever compact the read buffer. It requires doing a 
>>> memcopy which is definitely not necessary.
>>>
>>> Anyway, those tests about time I made were just to get an order of 
>>> magnitude about speed difference. I intended to do them differently 
>>> but JMH looks good so I'll use JMH to test now.
>>>
>>> Short reads only happen when fill(true) is called. That happens for 
>>> desperate get of data.
>>>
>>> I'll look into the avoiding double reading requests. I do think it 
>>> won't bring significant improvements if any at all. It only happens 
>>> when the buffer is nearly empty and any byte of data is welcome "at 
>>> any cost".
>>> Besides, whomever called read at that point would also have had an 
>>> availability() of 0 and still called read()/read(byte[]).
>>>
>>>
>>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>>  Hallo Brunoais,
>>>>
>>>> In the past I die some experiments with non-blocking file channels 
>>>> in the hope to increase throughput in a similiar way then your 
>>>> buffered stream. I also used direct allocated buffers. However my 
>>>> results have not been that encouraging (especially If a upper layer 
>>>> used larger reads). I thought back in the time this was mostly die 
>>>> to the fact that it NOT wraps to real AsyncFIO on most platforms. 
>>>> But maybe I just measured it wrong, so I will have a closer look on 
>>>> your impl.
>>>>
>>>> Generally I would recommend to make the Benchmark a bit more 
>>>> reliable with JMH and in order to do this to externalize the direct 
>>>> buffer allocation (as it ist slow if done repeatingly). This also 
>>>> allows you to publish some results with varrying workloads (on 
>>>> different machines).
>>>>
>>>> I would also measure the readCount to see if short reads happen.
>>>>
>>>>  BTW, I might as well try to only read till the end of the buffer 
>>>> in the backfilling-wraps-around case and not issue two requests, 
>>>> that might remove some additional latency.
>>>>
>>>> Gruss
>>>> Bernd
>>>> -- 
>>>> http://bernd.eckenfels.net
>>>>
>>>> _____________________________
>>>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>>> Subject: Re: Request/discussion: BufferedReader reading using async 
>>>> API while providing sync API
>>>> To: Pavel Rappo <pavel.rappo at oracle.com 
>>>> <mailto:pavel.rappo at oracle.com>>
>>>> Cc: <core-libs-dev at openjdk.java.net 
>>>> <mailto:core-libs-dev at openjdk.java.net>>
>>>>
>>>>
>>>> Attached and sending!
>>>>
>>>>
>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>> > Could you please send a new email on this list with the source 
>>>> attached as a
>>>> > text file?
>>>> >
>>>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>> >>
>>>> >> Here's my poc/prototype:
>>>> >> http://pastebin.com/WRpYWDJF
>>>> >>
>>>> >> I've implemented the bare minimum of the class that follows the 
>>>> same contract of BufferedReader while signaling all issues I think 
>>>> it may have or has in comments.
>>>> >> I also wrote some javadoc to help guiding through the class.
>>>> >>
>>>> >> I could have used more fields from BufferedReader but the names 
>>>> were so minimalistic that were confusing me. I intent to change 
>>>> them before sending this to openJDK.
>>>> >>
>>>> >> One of the major problems this has is long overflowing. It is 
>>>> major because it is hidden, it will be extremely rare and it takes 
>>>> a really long time to reproduce. There are different ways of 
>>>> dealing with it. From just documenting to actually making code that 
>>>> works with it.
>>>> >>
>>>> >> I built a simple test code for it to have some ideas about 
>>>> performance and correctness.
>>>> >>
>>>> >> http://pastebin.com/eh6LFgwT
>>>> >>
>>>> >> This doesn't do a through test if it is actually working 
>>>> correctly but I see no reason for it not working correctly after 
>>>> fixing the 2 bugs that test found.
>>>> >>
>>>> >> I'll also leave here some conclusions about speed and resource 
>>>> consumption I found.
>>>> >>
>>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 000B 
>>>> file, I was getting around:
>>>> >>
>>>> >> In all buffers and fake work: 10~15s speed improvement ( from 
>>>> 90% HDD speed to 100% HDD speed)
>>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>>> 90% HDD speed to 100% HDD speed)
>>>> >>
>>>> >> Changing the buffer size was giving different reading speeds but 
>>>> both were quite equal in how much they would change when changing 
>>>> the buffer size.
>>>> >> Finally, I could always confirm that I/O was always the slowest 
>>>> thing while this code was running.
>>>> >>
>>>> >> For the ones wondering about the file size; it is both to avoid 
>>>> OS cache and to make the reading at the main use-case these objects 
>>>> are for (large streams of bytes).
>>>> >>
>>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>>> >>
>>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>>> Reader out there.
>>>> >>> Not specifically FileReader. While you're talking about the 
>>>> case of effective
>>>> >>> reading from a file.
>>>> >>>
>>>> >>> I guess there's one existing possibility to provide exactly 
>>>> what you need (as I
>>>> >>> understand it) under this method:
>>>> >>>
>>>> >>> /**
>>>> >>> * Opens a file for reading, returning a {@code BufferedReader} 
>>>> to read text
>>>> >>> * from the file in an efficient manner...
>>>> >>> ...
>>>> >>> */
>>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>> >>>
>>>> >>> It can return _anything_ as long as it is a BufferedReader. We 
>>>> can do it, but it
>>>> >>> needs to be investigated not only for your favorite OS but for 
>>>> other OSes as
>>>> >>> well. Feel free to prototype this and we can discuss it on the 
>>>> list later.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> -Pavel
>>>> >>>
>>>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>> >>>>
>>>> >>>> Pavel is right.
>>>> >>>>
>>>> >>>> In reality, I was expecting such BufferedReader to use only a 
>>>> single buffer and have that Buffer being filled asynchronously, not 
>>>> in a different Thread.
>>>> >>>> Additionally, I don't have the intention of having a larger 
>>>> buffer than before unless stated through the API (the constructor).
>>>> >>>>
>>>> >>>> In my idea, internally, it is supposed to use 
>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>> >>>>
>>>> >>>> It does not prevent having two buffers and I do not intent to 
>>>> change BufferedReader itself. I'd do an BufferedAsyncReader of 
>>>> sorts (any name suggestion is welcome as I'm an awful namer).
>>>> >>>>
>>>> >>>>
>>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>> >>>>> Hi Pavel,
>>>> >>>>>
>>>> >>>>> I think Brunoais asking for a double buffering scheme in 
>>>> which the implementation of
>>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>>> application reading from the 1st buffer
>>>> >>>>> and managing the swaps and async reads transparently.
>>>> >>>>> It would not change the API but would change the interactions 
>>>> between the buffered reader
>>>> >>>>> and the underlying stream. It would also increase memory 
>>>> requirements and processing
>>>> >>>>> by introducing or using a separate thread and the necessary 
>>>> synchronization.
>>>> >>>>>
>>>> >>>>> Though I think the formal interface semantics could be 
>>>> maintained, I have doubts
>>>> >>>>> about compatibility and its unintended consequences on 
>>>> existing subclasses,
>>>> >>>>> applications and libraries.
>>>> >>>>>
>>>> >>>>> $.02, Roger
>>>> >>>>>
>>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>>> change the design of an
>>>> >>>>>> _extensible_ type that has been out there for 20 or so 
>>>> years. All these I/O
>>>> >>>>>> streams from java.io <http://java.io> were designed for 
>>>> simple synchronous use case.
>>>> >>>>>>
>>>> >>>>>> It's not that their design is flawed in some way, it's that 
>>>> they doesn't seem to
>>>> >>>>>> suit your needs. Have you considered using 
>>>> java.nio.channels.AsynchronousFileChannel
>>>> >>>>>> in your applications?
>>>> >>>>>>
>>>> >>>>>> -Pavel
>>>> >>>>>>
>>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Any feedback on this? I'm really interested in implementing 
>>>> such BufferedReader/BufferedStreamReader to allow speeding up my 
>>>> applications without having to think in an asynchronous way or 
>>>> multi-threading while programming with it.
>>>> >>>>>>>
>>>> >>>>>>> That's why I'm asking this here.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>> >>>>>>>> Hi,
>>>> >>>>>>>>
>>>> >>>>>>>> I looked at BufferedReader source code for java 9 long 
>>>> with the source code of the channels/streams used. I noticed that, 
>>>> like in java 7, BufferedReader does not use an Async API to load 
>>>> data from files, instead, the data loading is all done 
>>>> synchronously even when the OS allows requesting a file to be read 
>>>> and getting a warning later when the file is effectively read.
>>>> >>>>>>>>
>>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>> >>>>>>>>
>>>> >
>>>>
>>>>
>>>>
>>>
>>
>


From claes.redestad at oracle.com  Wed Oct 26 19:53:16 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 26 Oct 2016 21:53:16 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <859EA247-ABCA-450C-8E45-5D5BAC4D4995@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<d984517b-5743-4eb1-1678-b6244af8c18e@oracle.com>
	<dd917c60-23ed-d103-d93b-7a2e1ca022e0@gmail.com>
	<1dbe9507-0dc3-ca5c-ac14-2c363bb3dd70@oracle.com>
	<859EA247-ABCA-450C-8E45-5D5BAC4D4995@oracle.com>
Message-ID: <581109AC.50709@oracle.com>



On 2016-10-26 21:38, Paul Sandoz wrote:
> Do you observe that MethodHandleStatics.<clinit> can take a large proportion of the VH initialisation time?

Yes, but looking at my startup profiling I also see that that's mostly
due to MethodHandleStatics being the first to touch Integer.parseInt,
which initialize java.lang.CharacterDataLatin1, which takes 66% of the
#bytecode exercised there.

/Claes

From ecki at zusammenkunft.net  Wed Oct 26 20:20:42 2016
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Wed, 26 Oct 2016 20:20:42 +0000 (UTC)
Subject: Request/discussion: BufferedReader reading using async API
	while providing sync API
In-Reply-To: <65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
Message-ID: <57A008FAA9E6E1E3.6E4D2765-C3BE-40AC-98DD-D42511B9EECC@mail.outlook.com>

Hello,
On Linux you could use sysctl to drop caches but I guess one Option is to have a dataset larger than RAM.http://www.digitalinternals.com/unix/linux-clear-memory-cache/403/
Gruss
Bernd
-- 
http://bernd.eckenfels.net




On Wed, Oct 26, 2016 at 10:12 PM +0200, "Brunoais" <brunoaiss at gmail.com> wrote:










Thank you.

Only one thing left. How can I "burn" the OS' file read cache?
I only know how to do that by allocating a very large amount of memory 
based on the information I see in the resource manager (windows) or 
system monitor (linux) of the cached I/O and run the program. In this 
case, I have no idea how much memory each one's computer has so I cannot 
use the same method. How would you do such program excerpt?

As for the rest of the pointers: thank you I'll start building the 
benchmark code based on that information.


On 26/10/2016 18:24, Peter Levart wrote:
> Hi Brunoais,
>
> I'll try to tell what I know from my JMH practice:
>
> On 10/26/2016 10:30 AM, Brunoais wrote:
>> Hey guys. Any idea where I can find instructions on how to use JMH to:
>>
>> 1. Clear OS' file reading cache.
>
> You can create a public void method and make it called by JMH before each:
> - trial (a set of iterations)
> - iteration (a set of test method invocations)
> - invocation
>
> ...simply by annotating it by @Setup( [ Level.Trial | Level.Iteration 
> | Level.Invocation ] ).
>
> So create a method that spawns a script that clears the cache.
>
>> 2. Warm up whatever it needs to (maybe reading from a Channel in memory).
>
> JMH already warms-up the code and VM simply be executing "warmup" 
> iterations before starting real measured iterations. You can control 
> the number of warm-up iterations and real measured iterations by 
> annotating either the class or the method(s) with:
>
> @Warmup(iterations = ...)
> @Measurement(iterations = ...)
>
> If you want to warm-up resources by code that is not equal to code in 
> test method(s) then maybe @Setup methods on different levels could be 
> used for that.
>
>>
>> 3. Create a BufferedInputStream with a FileInputStream inside, with
>>    configurable buffer sizes.
>
> You can annotate a field of int, long or String type of a class 
> annotated with @State annotation (can be the benchmark class itself) 
> with @Param annotation, enumerating values this field will get before 
> executing the @Setup(Level.Trial) method(s). So you enumerate the 
> buffer sizes in @Param annotation and instantiate the 
> BufferedInputStream using the value in @Setup method. Viola.
>
>> 4. Execute iterations to read the file fully.
>
> Then perhaps you could use only one invocation per iteration and 
> measured it using @BenchmarkMode(Mode.SingleShotTime), constructing 
> the loop by yourself.
>
>> 1. Allow setting the byte[] size.
>
> Use @Parameter on a field to hold the byte[] size and create the 
> byte[] in @Setup method...
>
>> 2. On each iteration, burn a set number of CPU cycles.
>
> BlackHole.consumeCPU(tokens)
>
>> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>>    FileChannel.
>
> If you wrap them all into a common API (by delegation), you can use 
> @Parameter String implType, with @Setup method to instantiate the 
> appropriate implementation. Then just invoke the common API in the 
> test method.
>
>>
>> So far I still can't find how to:
>>
>> 1 (clear OS' cache)
>> 3 (the configuration part)
>> 4 (variable number of iterations)
>> 4.1 (the configuration)
>>
>> Can someone please point me in the right direction?
>
> I can create an example test if you like and you can then extend it...
>
> Regards, Peter
>
>>
>>
>> On 26/10/2016 07:57, Brunoais wrote:
>>>
>>> Hey Bernd!
>>>
>>> I don't know how far back you did such thing but I'm getting 
>>> positive results with my non-JMH tests. I do have to evaluate my 
>>> results against logic. After some reads, the OS starts caching the 
>>> file which is not what I want. It's easy to know when that happens, 
>>> though. The times fall from ~30s to ~5s and the HDD keeps near idle 
>>> reading (just looking at the LED is enough to understand).
>>>
>>> If you don't test synchronous work and you only run the reads, you 
>>> will only get marginal results as the OS has no real time to fill 
>>> the buffer.
>>> My research shows the 2 major kernels (windows' and GNU/Linux) have 
>>> non-blocking user-level buffer handling where I give a buffer for 
>>> the OS to read and it keeps filling it and sending messages/signals 
>>> as it writes chunks. Linux has an OS interrupt that only sends the 
>>> signal after it is full, though. There's also another version of 
>>> them where they use an internal buffer of same size as the buffer 
>>> you allocate for the OS and then internally call memcopy() into your 
>>> user-level memory when asked. Tests on the internet show that 
>>> memcopy is as fast (for 0-1 elements) or faster than 
>>> System.arraycopy(). I have no idea if they are true.
>>>
>>> All this was for me to add that, that code is tuned to copy from the 
>>> read buffer only when it is, at least, at half capacity and the 
>>> internal buffer has enough storage space. The process is forced only 
>>> if nothing had been read on the previous fill() call. It is built to 
>>> use JNI as little as possible while providing the major contract 
>>> BufferedInputStream has.
>>> Finally, I never, ever compact the read buffer. It requires doing a 
>>> memcopy which is definitely not necessary.
>>>
>>> Anyway, those tests about time I made were just to get an order of 
>>> magnitude about speed difference. I intended to do them differently 
>>> but JMH looks good so I'll use JMH to test now.
>>>
>>> Short reads only happen when fill(true) is called. That happens for 
>>> desperate get of data.
>>>
>>> I'll look into the avoiding double reading requests. I do think it 
>>> won't bring significant improvements if any at all. It only happens 
>>> when the buffer is nearly empty and any byte of data is welcome "at 
>>> any cost".
>>> Besides, whomever called read at that point would also have had an 
>>> availability() of 0 and still called read()/read(byte[]).
>>>
>>>
>>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>>  Hallo Brunoais,
>>>>
>>>> In the past I die some experiments with non-blocking file channels 
>>>> in the hope to increase throughput in a similiar way then your 
>>>> buffered stream. I also used direct allocated buffers. However my 
>>>> results have not been that encouraging (especially If a upper layer 
>>>> used larger reads). I thought back in the time this was mostly die 
>>>> to the fact that it NOT wraps to real AsyncFIO on most platforms. 
>>>> But maybe I just measured it wrong, so I will have a closer look on 
>>>> your impl.
>>>>
>>>> Generally I would recommend to make the Benchmark a bit more 
>>>> reliable with JMH and in order to do this to externalize the direct 
>>>> buffer allocation (as it ist slow if done repeatingly). This also 
>>>> allows you to publish some results with varrying workloads (on 
>>>> different machines).
>>>>
>>>> I would also measure the readCount to see if short reads happen.
>>>>
>>>>  BTW, I might as well try to only read till the end of the buffer 
>>>> in the backfilling-wraps-around case and not issue two requests, 
>>>> that might remove some additional latency.
>>>>
>>>> Gruss
>>>> Bernd
>>>> -- 
>>>> http://bernd.eckenfels.net
>>>>
>>>> _____________________________
>>>> From: Brunoais >
>>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>>> Subject: Re: Request/discussion: BufferedReader reading using async 
>>>> API while providing sync API
>>>> To: Pavel Rappo >>> >
>>>> Cc: >>> >
>>>>
>>>>
>>>> Attached and sending!
>>>>
>>>>
>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>> > Could you please send a new email on this list with the source 
>>>> attached as a
>>>> > text file?
>>>> >
>>>> >> On 23 Oct 2016, at 19:14, Brunoais >>> > wrote:
>>>> >>
>>>> >> Here's my poc/prototype:
>>>> >> http://pastebin.com/WRpYWDJF
>>>> >>
>>>> >> I've implemented the bare minimum of the class that follows the 
>>>> same contract of BufferedReader while signaling all issues I think 
>>>> it may have or has in comments.
>>>> >> I also wrote some javadoc to help guiding through the class.
>>>> >>
>>>> >> I could have used more fields from BufferedReader but the names 
>>>> were so minimalistic that were confusing me. I intent to change 
>>>> them before sending this to openJDK.
>>>> >>
>>>> >> One of the major problems this has is long overflowing. It is 
>>>> major because it is hidden, it will be extremely rare and it takes 
>>>> a really long time to reproduce. There are different ways of 
>>>> dealing with it. From just documenting to actually making code that 
>>>> works with it.
>>>> >>
>>>> >> I built a simple test code for it to have some ideas about 
>>>> performance and correctness.
>>>> >>
>>>> >> http://pastebin.com/eh6LFgwT
>>>> >>
>>>> >> This doesn't do a through test if it is actually working 
>>>> correctly but I see no reason for it not working correctly after 
>>>> fixing the 2 bugs that test found.
>>>> >>
>>>> >> I'll also leave here some conclusions about speed and resource 
>>>> consumption I found.
>>>> >>
>>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 000B 
>>>> file, I was getting around:
>>>> >>
>>>> >> In all buffers and fake work: 10~15s speed improvement ( from 
>>>> 90% HDD speed to 100% HDD speed)
>>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>>> 90% HDD speed to 100% HDD speed)
>>>> >>
>>>> >> Changing the buffer size was giving different reading speeds but 
>>>> both were quite equal in how much they would change when changing 
>>>> the buffer size.
>>>> >> Finally, I could always confirm that I/O was always the slowest 
>>>> thing while this code was running.
>>>> >>
>>>> >> For the ones wondering about the file size; it is both to avoid 
>>>> OS cache and to make the reading at the main use-case these objects 
>>>> are for (large streams of bytes).
>>>> >>
>>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>>> >>
>>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>>> Reader out there.
>>>> >>> Not specifically FileReader. While you're talking about the 
>>>> case of effective
>>>> >>> reading from a file.
>>>> >>>
>>>> >>> I guess there's one existing possibility to provide exactly 
>>>> what you need (as I
>>>> >>> understand it) under this method:
>>>> >>>
>>>> >>> /**
>>>> >>> * Opens a file for reading, returning a {@code BufferedReader} 
>>>> to read text
>>>> >>> * from the file in an efficient manner...
>>>> >>> ...
>>>> >>> */
>>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>> >>>
>>>> >>> It can return _anything_ as long as it is a BufferedReader. We 
>>>> can do it, but it
>>>> >>> needs to be investigated not only for your favorite OS but for 
>>>> other OSes as
>>>> >>> well. Feel free to prototype this and we can discuss it on the 
>>>> list later.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> -Pavel
>>>> >>>
>>>> >>>> On 21 Oct 2016, at 18:56, Brunoais >>> > wrote:
>>>> >>>>
>>>> >>>> Pavel is right.
>>>> >>>>
>>>> >>>> In reality, I was expecting such BufferedReader to use only a 
>>>> single buffer and have that Buffer being filled asynchronously, not 
>>>> in a different Thread.
>>>> >>>> Additionally, I don't have the intention of having a larger 
>>>> buffer than before unless stated through the API (the constructor).
>>>> >>>>
>>>> >>>> In my idea, internally, it is supposed to use 
>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>> >>>>
>>>> >>>> It does not prevent having two buffers and I do not intent to 
>>>> change BufferedReader itself. I'd do an BufferedAsyncReader of 
>>>> sorts (any name suggestion is welcome as I'm an awful namer).
>>>> >>>>
>>>> >>>>
>>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>> >>>>> Hi Pavel,
>>>> >>>>>
>>>> >>>>> I think Brunoais asking for a double buffering scheme in 
>>>> which the implementation of
>>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>>> application reading from the 1st buffer
>>>> >>>>> and managing the swaps and async reads transparently.
>>>> >>>>> It would not change the API but would change the interactions 
>>>> between the buffered reader
>>>> >>>>> and the underlying stream. It would also increase memory 
>>>> requirements and processing
>>>> >>>>> by introducing or using a separate thread and the necessary 
>>>> synchronization.
>>>> >>>>>
>>>> >>>>> Though I think the formal interface semantics could be 
>>>> maintained, I have doubts
>>>> >>>>> about compatibility and its unintended consequences on 
>>>> existing subclasses,
>>>> >>>>> applications and libraries.
>>>> >>>>>
>>>> >>>>> $.02, Roger
>>>> >>>>>
>>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>>> change the design of an
>>>> >>>>>> _extensible_ type that has been out there for 20 or so 
>>>> years. All these I/O
>>>> >>>>>> streams from java.io  were designed for 
>>>> simple synchronous use case.
>>>> >>>>>>
>>>> >>>>>> It's not that their design is flawed in some way, it's that 
>>>> they doesn't seem to
>>>> >>>>>> suit your needs. Have you considered using 
>>>> java.nio.channels.AsynchronousFileChannel
>>>> >>>>>> in your applications?
>>>> >>>>>>
>>>> >>>>>> -Pavel
>>>> >>>>>>
>>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais >>> > wrote:
>>>> >>>>>>>
>>>> >>>>>>> Any feedback on this? I'm really interested in implementing 
>>>> such BufferedReader/BufferedStreamReader to allow speeding up my 
>>>> applications without having to think in an asynchronous way or 
>>>> multi-threading while programming with it.
>>>> >>>>>>>
>>>> >>>>>>> That's why I'm asking this here.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>> >>>>>>>> Hi,
>>>> >>>>>>>>
>>>> >>>>>>>> I looked at BufferedReader source code for java 9 long 
>>>> with the source code of the channels/streams used. I noticed that, 
>>>> like in java 7, BufferedReader does not use an Async API to load 
>>>> data from files, instead, the data loading is all done 
>>>> synchronously even when the OS allows requesting a file to be read 
>>>> and getting a warning later when the file is effectively read.
>>>> >>>>>>>>
>>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>> >>>>>>>>
>>>> >
>>>>
>>>>
>>>>
>>>
>>
>







From peter.levart at gmail.com  Wed Oct 26 20:41:52 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 26 Oct 2016 22:41:52 +0200
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
Message-ID: <c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>



On 10/26/2016 09:46 PM, Brunoais wrote:
>
> Thank you.
>
> Only one thing left. How can I "burn" the OS' file read cache?
> I only know how to do that by allocating a very large amount of memory 
> based on the information I see in the resource manager (windows) or 
> system monitor (linux) of the cached I/O and run the program. In this 
> case, I have no idea how much memory each one's computer has so I 
> cannot use the same method. How would you do such program excerpt?
>
> As for the rest of the pointers: thank you I'll start building the 
> benchmark code based on that information.
>

Here's a prototype you can extend. It'll make you jump-start in 5th gear:

http://cr.openjdk.java.net/~plevart/misc/FileReadBench/FileReadBench.java

Regards, Peter

>
> On 26/10/2016 18:24, Peter Levart wrote:
>> Hi Brunoais,
>>
>> I'll try to tell what I know from my JMH practice:
>>
>> On 10/26/2016 10:30 AM, Brunoais wrote:
>>> Hey guys. Any idea where I can find instructions on how to use JMH to:
>>>
>>> 1. Clear OS' file reading cache.
>>
>> You can create a public void method and make it called by JMH before 
>> each:
>> - trial (a set of iterations)
>> - iteration (a set of test method invocations)
>> - invocation
>>
>> ...simply by annotating it by @Setup( [ Level.Trial | Level.Iteration 
>> | Level.Invocation ] ).
>>
>> So create a method that spawns a script that clears the cache.
>>
>>> 2. Warm up whatever it needs to (maybe reading from a Channel in 
>>> memory).
>>
>> JMH already warms-up the code and VM simply be executing "warmup" 
>> iterations before starting real measured iterations. You can control 
>> the number of warm-up iterations and real measured iterations by 
>> annotating either the class or the method(s) with:
>>
>> @Warmup(iterations = ...)
>> @Measurement(iterations = ...)
>>
>> If you want to warm-up resources by code that is not equal to code in 
>> test method(s) then maybe @Setup methods on different levels could be 
>> used for that.
>>
>>>
>>> 3. Create a BufferedInputStream with a FileInputStream inside, with
>>>    configurable buffer sizes.
>>
>> You can annotate a field of int, long or String type of a class 
>> annotated with @State annotation (can be the benchmark class itself) 
>> with @Param annotation, enumerating values this field will get before 
>> executing the @Setup(Level.Trial) method(s). So you enumerate the 
>> buffer sizes in @Param annotation and instantiate the 
>> BufferedInputStream using the value in @Setup method. Viola.
>>
>>> 4. Execute iterations to read the file fully.
>>
>> Then perhaps you could use only one invocation per iteration and 
>> measured it using @BenchmarkMode(Mode.SingleShotTime), constructing 
>> the loop by yourself.
>>
>>> 1. Allow setting the byte[] size.
>>
>> Use @Parameter on a field to hold the byte[] size and create the 
>> byte[] in @Setup method...
>>
>>> 2. On each iteration, burn a set number of CPU cycles.
>>
>> BlackHole.consumeCPU(tokens)
>>
>>> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>>>    FileChannel.
>>
>> If you wrap them all into a common API (by delegation), you can use 
>> @Parameter String implType, with @Setup method to instantiate the 
>> appropriate implementation. Then just invoke the common API in the 
>> test method.
>>
>>>
>>> So far I still can't find how to:
>>>
>>> 1 (clear OS' cache)
>>> 3 (the configuration part)
>>> 4 (variable number of iterations)
>>> 4.1 (the configuration)
>>>
>>> Can someone please point me in the right direction?
>>
>> I can create an example test if you like and you can then extend it...
>>
>> Regards, Peter
>>
>>>
>>>
>>> On 26/10/2016 07:57, Brunoais wrote:
>>>>
>>>> Hey Bernd!
>>>>
>>>> I don't know how far back you did such thing but I'm getting 
>>>> positive results with my non-JMH tests. I do have to evaluate my 
>>>> results against logic. After some reads, the OS starts caching the 
>>>> file which is not what I want. It's easy to know when that happens, 
>>>> though. The times fall from ~30s to ~5s and the HDD keeps near idle 
>>>> reading (just looking at the LED is enough to understand).
>>>>
>>>> If you don't test synchronous work and you only run the reads, you 
>>>> will only get marginal results as the OS has no real time to fill 
>>>> the buffer.
>>>> My research shows the 2 major kernels (windows' and GNU/Linux) have 
>>>> non-blocking user-level buffer handling where I give a buffer for 
>>>> the OS to read and it keeps filling it and sending messages/signals 
>>>> as it writes chunks. Linux has an OS interrupt that only sends the 
>>>> signal after it is full, though. There's also another version of 
>>>> them where they use an internal buffer of same size as the buffer 
>>>> you allocate for the OS and then internally call memcopy() into 
>>>> your user-level memory when asked. Tests on the internet show that 
>>>> memcopy is as fast (for 0-1 elements) or faster than 
>>>> System.arraycopy(). I have no idea if they are true.
>>>>
>>>> All this was for me to add that, that code is tuned to copy from 
>>>> the read buffer only when it is, at least, at half capacity and the 
>>>> internal buffer has enough storage space. The process is forced 
>>>> only if nothing had been read on the previous fill() call. It is 
>>>> built to use JNI as little as possible while providing the major 
>>>> contract BufferedInputStream has.
>>>> Finally, I never, ever compact the read buffer. It requires doing a 
>>>> memcopy which is definitely not necessary.
>>>>
>>>> Anyway, those tests about time I made were just to get an order of 
>>>> magnitude about speed difference. I intended to do them differently 
>>>> but JMH looks good so I'll use JMH to test now.
>>>>
>>>> Short reads only happen when fill(true) is called. That happens for 
>>>> desperate get of data.
>>>>
>>>> I'll look into the avoiding double reading requests. I do think it 
>>>> won't bring significant improvements if any at all. It only happens 
>>>> when the buffer is nearly empty and any byte of data is welcome "at 
>>>> any cost".
>>>> Besides, whomever called read at that point would also have had an 
>>>> availability() of 0 and still called read()/read(byte[]).
>>>>
>>>>
>>>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>>>  Hallo Brunoais,
>>>>>
>>>>> In the past I die some experiments with non-blocking file channels 
>>>>> in the hope to increase throughput in a similiar way then your 
>>>>> buffered stream. I also used direct allocated buffers. However my 
>>>>> results have not been that encouraging (especially If a upper 
>>>>> layer used larger reads). I thought back in the time this was 
>>>>> mostly die to the fact that it NOT wraps to real AsyncFIO on most 
>>>>> platforms. But maybe I just measured it wrong, so I will have a 
>>>>> closer look on your impl.
>>>>>
>>>>> Generally I would recommend to make the Benchmark a bit more 
>>>>> reliable with JMH and in order to do this to externalize the 
>>>>> direct buffer allocation (as it ist slow if done repeatingly). 
>>>>> This also allows you to publish some results with varrying 
>>>>> workloads (on different machines).
>>>>>
>>>>> I would also measure the readCount to see if short reads happen.
>>>>>
>>>>>  BTW, I might as well try to only read till the end of the buffer 
>>>>> in the backfilling-wraps-around case and not issue two requests, 
>>>>> that might remove some additional latency.
>>>>>
>>>>> Gruss
>>>>> Bernd
>>>>> -- 
>>>>> http://bernd.eckenfels.net
>>>>>
>>>>> _____________________________
>>>>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>>>> Subject: Re: Request/discussion: BufferedReader reading using 
>>>>> async API while providing sync API
>>>>> To: Pavel Rappo <pavel.rappo at oracle.com 
>>>>> <mailto:pavel.rappo at oracle.com>>
>>>>> Cc: <core-libs-dev at openjdk.java.net 
>>>>> <mailto:core-libs-dev at openjdk.java.net>>
>>>>>
>>>>>
>>>>> Attached and sending!
>>>>>
>>>>>
>>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>> > Could you please send a new email on this list with the source 
>>>>> attached as a
>>>>> > text file?
>>>>> >
>>>>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>> >>
>>>>> >> Here's my poc/prototype:
>>>>> >> http://pastebin.com/WRpYWDJF
>>>>> >>
>>>>> >> I've implemented the bare minimum of the class that follows the 
>>>>> same contract of BufferedReader while signaling all issues I think 
>>>>> it may have or has in comments.
>>>>> >> I also wrote some javadoc to help guiding through the class.
>>>>> >>
>>>>> >> I could have used more fields from BufferedReader but the names 
>>>>> were so minimalistic that were confusing me. I intent to change 
>>>>> them before sending this to openJDK.
>>>>> >>
>>>>> >> One of the major problems this has is long overflowing. It is 
>>>>> major because it is hidden, it will be extremely rare and it takes 
>>>>> a really long time to reproduce. There are different ways of 
>>>>> dealing with it. From just documenting to actually making code 
>>>>> that works with it.
>>>>> >>
>>>>> >> I built a simple test code for it to have some ideas about 
>>>>> performance and correctness.
>>>>> >>
>>>>> >> http://pastebin.com/eh6LFgwT
>>>>> >>
>>>>> >> This doesn't do a through test if it is actually working 
>>>>> correctly but I see no reason for it not working correctly after 
>>>>> fixing the 2 bugs that test found.
>>>>> >>
>>>>> >> I'll also leave here some conclusions about speed and resource 
>>>>> consumption I found.
>>>>> >>
>>>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 
>>>>> 000B file, I was getting around:
>>>>> >>
>>>>> >> In all buffers and fake work: 10~15s speed improvement ( from 
>>>>> 90% HDD speed to 100% HDD speed)
>>>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>>>> 90% HDD speed to 100% HDD speed)
>>>>> >>
>>>>> >> Changing the buffer size was giving different reading speeds 
>>>>> but both were quite equal in how much they would change when 
>>>>> changing the buffer size.
>>>>> >> Finally, I could always confirm that I/O was always the slowest 
>>>>> thing while this code was running.
>>>>> >>
>>>>> >> For the ones wondering about the file size; it is both to avoid 
>>>>> OS cache and to make the reading at the main use-case these 
>>>>> objects are for (large streams of bytes).
>>>>> >>
>>>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>> >>
>>>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>>>> Reader out there.
>>>>> >>> Not specifically FileReader. While you're talking about the 
>>>>> case of effective
>>>>> >>> reading from a file.
>>>>> >>>
>>>>> >>> I guess there's one existing possibility to provide exactly 
>>>>> what you need (as I
>>>>> >>> understand it) under this method:
>>>>> >>>
>>>>> >>> /**
>>>>> >>> * Opens a file for reading, returning a {@code BufferedReader} 
>>>>> to read text
>>>>> >>> * from the file in an efficient manner...
>>>>> >>> ...
>>>>> >>> */
>>>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>> >>>
>>>>> >>> It can return _anything_ as long as it is a BufferedReader. We 
>>>>> can do it, but it
>>>>> >>> needs to be investigated not only for your favorite OS but for 
>>>>> other OSes as
>>>>> >>> well. Feel free to prototype this and we can discuss it on the 
>>>>> list later.
>>>>> >>>
>>>>> >>> Thanks,
>>>>> >>> -Pavel
>>>>> >>>
>>>>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>> >>>>
>>>>> >>>> Pavel is right.
>>>>> >>>>
>>>>> >>>> In reality, I was expecting such BufferedReader to use only a 
>>>>> single buffer and have that Buffer being filled asynchronously, 
>>>>> not in a different Thread.
>>>>> >>>> Additionally, I don't have the intention of having a larger 
>>>>> buffer than before unless stated through the API (the constructor).
>>>>> >>>>
>>>>> >>>> In my idea, internally, it is supposed to use 
>>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>> >>>>
>>>>> >>>> It does not prevent having two buffers and I do not intent to 
>>>>> change BufferedReader itself. I'd do an BufferedAsyncReader of 
>>>>> sorts (any name suggestion is welcome as I'm an awful namer).
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>> >>>>> Hi Pavel,
>>>>> >>>>>
>>>>> >>>>> I think Brunoais asking for a double buffering scheme in 
>>>>> which the implementation of
>>>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>>>> application reading from the 1st buffer
>>>>> >>>>> and managing the swaps and async reads transparently.
>>>>> >>>>> It would not change the API but would change the 
>>>>> interactions between the buffered reader
>>>>> >>>>> and the underlying stream. It would also increase memory 
>>>>> requirements and processing
>>>>> >>>>> by introducing or using a separate thread and the necessary 
>>>>> synchronization.
>>>>> >>>>>
>>>>> >>>>> Though I think the formal interface semantics could be 
>>>>> maintained, I have doubts
>>>>> >>>>> about compatibility and its unintended consequences on 
>>>>> existing subclasses,
>>>>> >>>>> applications and libraries.
>>>>> >>>>>
>>>>> >>>>> $.02, Roger
>>>>> >>>>>
>>>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>>>> change the design of an
>>>>> >>>>>> _extensible_ type that has been out there for 20 or so 
>>>>> years. All these I/O
>>>>> >>>>>> streams from java.io <http://java.io> were designed for 
>>>>> simple synchronous use case.
>>>>> >>>>>>
>>>>> >>>>>> It's not that their design is flawed in some way, it's that 
>>>>> they doesn't seem to
>>>>> >>>>>> suit your needs. Have you considered using 
>>>>> java.nio.channels.AsynchronousFileChannel
>>>>> >>>>>> in your applications?
>>>>> >>>>>>
>>>>> >>>>>> -Pavel
>>>>> >>>>>>
>>>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Any feedback on this? I'm really interested in 
>>>>> implementing such BufferedReader/BufferedStreamReader to allow 
>>>>> speeding up my applications without having to think in an 
>>>>> asynchronous way or multi-threading while programming with it.
>>>>> >>>>>>>
>>>>> >>>>>>> That's why I'm asking this here.
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>> >>>>>>>> Hi,
>>>>> >>>>>>>>
>>>>> >>>>>>>> I looked at BufferedReader source code for java 9 long 
>>>>> with the source code of the channels/streams used. I noticed that, 
>>>>> like in java 7, BufferedReader does not use an Async API to load 
>>>>> data from files, instead, the data loading is all done 
>>>>> synchronously even when the OS allows requesting a file to be read 
>>>>> and getting a warning later when the file is effectively read.
>>>>> >>>>>>>>
>>>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>> >>>>>>>>
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>


From brunoaiss at gmail.com  Wed Oct 26 21:18:53 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Wed, 26 Oct 2016 22:18:53 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
	<c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
Message-ID: <90bd4ff6-8bcc-f0e2-026d-ef9be59fc228@gmail.com>

You didn't need to.... Thank you, I'll use it.


On 26/10/2016 21:41, Peter Levart wrote:
>
>
>
> On 10/26/2016 09:46 PM, Brunoais wrote:
>>
>> Thank you.
>>
>> Only one thing left. How can I "burn" the OS' file read cache?
>> I only know how to do that by allocating a very large amount of 
>> memory based on the information I see in the resource manager 
>> (windows) or system monitor (linux) of the cached I/O and run the 
>> program. In this case, I have no idea how much memory each one's 
>> computer has so I cannot use the same method. How would you do such 
>> program excerpt?
>>
>> As for the rest of the pointers: thank you I'll start building the 
>> benchmark code based on that information.
>>
>
> Here's a prototype you can extend. It'll make you jump-start in 5th gear:
>
> http://cr.openjdk.java.net/~plevart/misc/FileReadBench/FileReadBench.java
>
> Regards, Peter
>
>>
>> On 26/10/2016 18:24, Peter Levart wrote:
>>> Hi Brunoais,
>>>
>>> I'll try to tell what I know from my JMH practice:
>>>
>>> On 10/26/2016 10:30 AM, Brunoais wrote:
>>>> Hey guys. Any idea where I can find instructions on how to use JMH to:
>>>>
>>>> 1. Clear OS' file reading cache.
>>>
>>> You can create a public void method and make it called by JMH before 
>>> each:
>>> - trial (a set of iterations)
>>> - iteration (a set of test method invocations)
>>> - invocation
>>>
>>> ...simply by annotating it by @Setup( [ Level.Trial | 
>>> Level.Iteration | Level.Invocation ] ).
>>>
>>> So create a method that spawns a script that clears the cache.
>>>
>>>> 2. Warm up whatever it needs to (maybe reading from a Channel in 
>>>> memory).
>>>
>>> JMH already warms-up the code and VM simply be executing "warmup" 
>>> iterations before starting real measured iterations. You can control 
>>> the number of warm-up iterations and real measured iterations by 
>>> annotating either the class or the method(s) with:
>>>
>>> @Warmup(iterations = ...)
>>> @Measurement(iterations = ...)
>>>
>>> If you want to warm-up resources by code that is not equal to code 
>>> in test method(s) then maybe @Setup methods on different levels 
>>> could be used for that.
>>>
>>>>
>>>> 3. Create a BufferedInputStream with a FileInputStream inside, with
>>>>    configurable buffer sizes.
>>>
>>> You can annotate a field of int, long or String type of a class 
>>> annotated with @State annotation (can be the benchmark class itself) 
>>> with @Param annotation, enumerating values this field will get 
>>> before executing the @Setup(Level.Trial) method(s). So you enumerate 
>>> the buffer sizes in @Param annotation and instantiate the 
>>> BufferedInputStream using the value in @Setup method. Viola.
>>>
>>>> 4. Execute iterations to read the file fully.
>>>
>>> Then perhaps you could use only one invocation per iteration and 
>>> measured it using @BenchmarkMode(Mode.SingleShotTime), constructing 
>>> the loop by yourself.
>>>
>>>> 1. Allow setting the byte[] size.
>>>
>>> Use @Parameter on a field to hold the byte[] size and create the 
>>> byte[] in @Setup method...
>>>
>>>> 2. On each iteration, burn a set number of CPU cycles.
>>>
>>> BlackHole.consumeCPU(tokens)
>>>
>>>> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>>>>    FileChannel.
>>>
>>> If you wrap them all into a common API (by delegation), you can use 
>>> @Parameter String implType, with @Setup method to instantiate the 
>>> appropriate implementation. Then just invoke the common API in the 
>>> test method.
>>>
>>>>
>>>> So far I still can't find how to:
>>>>
>>>> 1 (clear OS' cache)
>>>> 3 (the configuration part)
>>>> 4 (variable number of iterations)
>>>> 4.1 (the configuration)
>>>>
>>>> Can someone please point me in the right direction?
>>>
>>> I can create an example test if you like and you can then extend it...
>>>
>>> Regards, Peter
>>>
>>>>
>>>>
>>>> On 26/10/2016 07:57, Brunoais wrote:
>>>>>
>>>>> Hey Bernd!
>>>>>
>>>>> I don't know how far back you did such thing but I'm getting 
>>>>> positive results with my non-JMH tests. I do have to evaluate my 
>>>>> results against logic. After some reads, the OS starts caching the 
>>>>> file which is not what I want. It's easy to know when that 
>>>>> happens, though. The times fall from ~30s to ~5s and the HDD keeps 
>>>>> near idle reading (just looking at the LED is enough to understand).
>>>>>
>>>>> If you don't test synchronous work and you only run the reads, you 
>>>>> will only get marginal results as the OS has no real time to fill 
>>>>> the buffer.
>>>>> My research shows the 2 major kernels (windows' and GNU/Linux) 
>>>>> have non-blocking user-level buffer handling where I give a buffer 
>>>>> for the OS to read and it keeps filling it and sending 
>>>>> messages/signals as it writes chunks. Linux has an OS interrupt 
>>>>> that only sends the signal after it is full, though. There's also 
>>>>> another version of them where they use an internal buffer of same 
>>>>> size as the buffer you allocate for the OS and then internally 
>>>>> call memcopy() into your user-level memory when asked. Tests on 
>>>>> the internet show that memcopy is as fast (for 0-1 elements) or 
>>>>> faster than System.arraycopy(). I have no idea if they are true.
>>>>>
>>>>> All this was for me to add that, that code is tuned to copy from 
>>>>> the read buffer only when it is, at least, at half capacity and 
>>>>> the internal buffer has enough storage space. The process is 
>>>>> forced only if nothing had been read on the previous fill() call. 
>>>>> It is built to use JNI as little as possible while providing the 
>>>>> major contract BufferedInputStream has.
>>>>> Finally, I never, ever compact the read buffer. It requires doing 
>>>>> a memcopy which is definitely not necessary.
>>>>>
>>>>> Anyway, those tests about time I made were just to get an order of 
>>>>> magnitude about speed difference. I intended to do them 
>>>>> differently but JMH looks good so I'll use JMH to test now.
>>>>>
>>>>> Short reads only happen when fill(true) is called. That happens 
>>>>> for desperate get of data.
>>>>>
>>>>> I'll look into the avoiding double reading requests. I do think it 
>>>>> won't bring significant improvements if any at all. It only 
>>>>> happens when the buffer is nearly empty and any byte of data is 
>>>>> welcome "at any cost".
>>>>> Besides, whomever called read at that point would also have had an 
>>>>> availability() of 0 and still called read()/read(byte[]).
>>>>>
>>>>>
>>>>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>>>>  Hallo Brunoais,
>>>>>>
>>>>>> In the past I die some experiments with non-blocking file 
>>>>>> channels in the hope to increase throughput in a similiar way 
>>>>>> then your buffered stream. I also used direct allocated buffers. 
>>>>>> However my results have not been that encouraging (especially If 
>>>>>> a upper layer used larger reads). I thought back in the time this 
>>>>>> was mostly die to the fact that it NOT wraps to real AsyncFIO on 
>>>>>> most platforms. But maybe I just measured it wrong, so I will 
>>>>>> have a closer look on your impl.
>>>>>>
>>>>>> Generally I would recommend to make the Benchmark a bit more 
>>>>>> reliable with JMH and in order to do this to externalize the 
>>>>>> direct buffer allocation (as it ist slow if done repeatingly). 
>>>>>> This also allows you to publish some results with varrying 
>>>>>> workloads (on different machines).
>>>>>>
>>>>>> I would also measure the readCount to see if short reads happen.
>>>>>>
>>>>>>  BTW, I might as well try to only read till the end of the buffer 
>>>>>> in the backfilling-wraps-around case and not issue two requests, 
>>>>>> that might remove some additional latency.
>>>>>>
>>>>>> Gruss
>>>>>> Bernd
>>>>>> -- 
>>>>>> http://bernd.eckenfels.net
>>>>>>
>>>>>> _____________________________
>>>>>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>>>>> Subject: Re: Request/discussion: BufferedReader reading using 
>>>>>> async API while providing sync API
>>>>>> To: Pavel Rappo <pavel.rappo at oracle.com 
>>>>>> <mailto:pavel.rappo at oracle.com>>
>>>>>> Cc: <core-libs-dev at openjdk.java.net 
>>>>>> <mailto:core-libs-dev at openjdk.java.net>>
>>>>>>
>>>>>>
>>>>>> Attached and sending!
>>>>>>
>>>>>>
>>>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>>> > Could you please send a new email on this list with the source 
>>>>>> attached as a
>>>>>> > text file?
>>>>>> >
>>>>>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>
>>>>>> >> Here's my poc/prototype:
>>>>>> >> http://pastebin.com/WRpYWDJF
>>>>>> >>
>>>>>> >> I've implemented the bare minimum of the class that follows 
>>>>>> the same contract of BufferedReader while signaling all issues I 
>>>>>> think it may have or has in comments.
>>>>>> >> I also wrote some javadoc to help guiding through the class.
>>>>>> >>
>>>>>> >> I could have used more fields from BufferedReader but the 
>>>>>> names were so minimalistic that were confusing me. I intent to 
>>>>>> change them before sending this to openJDK.
>>>>>> >>
>>>>>> >> One of the major problems this has is long overflowing. It is 
>>>>>> major because it is hidden, it will be extremely rare and it 
>>>>>> takes a really long time to reproduce. There are different ways 
>>>>>> of dealing with it. From just documenting to actually making code 
>>>>>> that works with it.
>>>>>> >>
>>>>>> >> I built a simple test code for it to have some ideas about 
>>>>>> performance and correctness.
>>>>>> >>
>>>>>> >> http://pastebin.com/eh6LFgwT
>>>>>> >>
>>>>>> >> This doesn't do a through test if it is actually working 
>>>>>> correctly but I see no reason for it not working correctly after 
>>>>>> fixing the 2 bugs that test found.
>>>>>> >>
>>>>>> >> I'll also leave here some conclusions about speed and resource 
>>>>>> consumption I found.
>>>>>> >>
>>>>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>>>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 
>>>>>> 000B file, I was getting around:
>>>>>> >>
>>>>>> >> In all buffers and fake work: 10~15s speed improvement ( from 
>>>>>> 90% HDD speed to 100% HDD speed)
>>>>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>>>>> 90% HDD speed to 100% HDD speed)
>>>>>> >>
>>>>>> >> Changing the buffer size was giving different reading speeds 
>>>>>> but both were quite equal in how much they would change when 
>>>>>> changing the buffer size.
>>>>>> >> Finally, I could always confirm that I/O was always the 
>>>>>> slowest thing while this code was running.
>>>>>> >>
>>>>>> >> For the ones wondering about the file size; it is both to 
>>>>>> avoid OS cache and to make the reading at the main use-case these 
>>>>>> objects are for (large streams of bytes).
>>>>>> >>
>>>>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>>> >>
>>>>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>>>>> Reader out there.
>>>>>> >>> Not specifically FileReader. While you're talking about the 
>>>>>> case of effective
>>>>>> >>> reading from a file.
>>>>>> >>>
>>>>>> >>> I guess there's one existing possibility to provide exactly 
>>>>>> what you need (as I
>>>>>> >>> understand it) under this method:
>>>>>> >>>
>>>>>> >>> /**
>>>>>> >>> * Opens a file for reading, returning a {@code 
>>>>>> BufferedReader} to read text
>>>>>> >>> * from the file in an efficient manner...
>>>>>> >>> ...
>>>>>> >>> */
>>>>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>> >>>
>>>>>> >>> It can return _anything_ as long as it is a BufferedReader. 
>>>>>> We can do it, but it
>>>>>> >>> needs to be investigated not only for your favorite OS but 
>>>>>> for other OSes as
>>>>>> >>> well. Feel free to prototype this and we can discuss it on 
>>>>>> the list later.
>>>>>> >>>
>>>>>> >>> Thanks,
>>>>>> >>> -Pavel
>>>>>> >>>
>>>>>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>>>
>>>>>> >>>> Pavel is right.
>>>>>> >>>>
>>>>>> >>>> In reality, I was expecting such BufferedReader to use only 
>>>>>> a single buffer and have that Buffer being filled asynchronously, 
>>>>>> not in a different Thread.
>>>>>> >>>> Additionally, I don't have the intention of having a larger 
>>>>>> buffer than before unless stated through the API (the constructor).
>>>>>> >>>>
>>>>>> >>>> In my idea, internally, it is supposed to use 
>>>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>> >>>>
>>>>>> >>>> It does not prevent having two buffers and I do not intent 
>>>>>> to change BufferedReader itself. I'd do an BufferedAsyncReader of 
>>>>>> sorts (any name suggestion is welcome as I'm an awful namer).
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>> >>>>> Hi Pavel,
>>>>>> >>>>>
>>>>>> >>>>> I think Brunoais asking for a double buffering scheme in 
>>>>>> which the implementation of
>>>>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>>>>> application reading from the 1st buffer
>>>>>> >>>>> and managing the swaps and async reads transparently.
>>>>>> >>>>> It would not change the API but would change the 
>>>>>> interactions between the buffered reader
>>>>>> >>>>> and the underlying stream. It would also increase memory 
>>>>>> requirements and processing
>>>>>> >>>>> by introducing or using a separate thread and the necessary 
>>>>>> synchronization.
>>>>>> >>>>>
>>>>>> >>>>> Though I think the formal interface semantics could be 
>>>>>> maintained, I have doubts
>>>>>> >>>>> about compatibility and its unintended consequences on 
>>>>>> existing subclasses,
>>>>>> >>>>> applications and libraries.
>>>>>> >>>>>
>>>>>> >>>>> $.02, Roger
>>>>>> >>>>>
>>>>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>>>>> change the design of an
>>>>>> >>>>>> _extensible_ type that has been out there for 20 or so 
>>>>>> years. All these I/O
>>>>>> >>>>>> streams from java.io <http://java.io> were designed for 
>>>>>> simple synchronous use case.
>>>>>> >>>>>>
>>>>>> >>>>>> It's not that their design is flawed in some way, it's 
>>>>>> that they doesn't seem to
>>>>>> >>>>>> suit your needs. Have you considered using 
>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>> >>>>>> in your applications?
>>>>>> >>>>>>
>>>>>> >>>>>> -Pavel
>>>>>> >>>>>>
>>>>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>> Any feedback on this? I'm really interested in 
>>>>>> implementing such BufferedReader/BufferedStreamReader to allow 
>>>>>> speeding up my applications without having to think in an 
>>>>>> asynchronous way or multi-threading while programming with it.
>>>>>> >>>>>>>
>>>>>> >>>>>>> That's why I'm asking this here.
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>> >>>>>>>> Hi,
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> I looked at BufferedReader source code for java 9 long 
>>>>>> with the source code of the channels/streams used. I noticed 
>>>>>> that, like in java 7, BufferedReader does not use an Async API to 
>>>>>> load data from files, instead, the data loading is all done 
>>>>>> synchronously even when the OS allows requesting a file to be 
>>>>>> read and getting a warning later when the file is effectively read.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>> >>>>>>>>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From vitalyd at gmail.com  Wed Oct 26 23:06:49 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Wed, 26 Oct 2016 19:06:49 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
Message-ID: <CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>

On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com> wrote:

> It is actually based on the premise that:
>
> 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS
>    buffer size to fill in as the same size as ByteBuffer.

Why do you say that? AFAICT, it issues a read syscall and that will block
if the data isn't in page cache.

> 2. The consecutive calls to ReadableByteChannel.read(ByteBuffer) orders
>    the JVM to order the OS to execute memcpy() to copy from its memory
>    to the shared memory created at ByteBuffer instantiation (in java 8)
>    using Unsafe and then for the JVM to update the ByteBuffer fields.

I think subsequent reads just invoke the same read syscall, passing the
current file offset maintained by the file channel instance.

> 3. The call will not block waiting for I/O and it won't take longer
>    than the JNI interface if no new data exists. However, it will block
>    waiting for the OS to execute memcpy() to the shared memory.

So why do you think it won't block?

>
> Is my premise wrong?
>
> If I read correctly, if I don't use a DirectBuffer, there would be even
> another intermediate buffer to copy data to before giving it to the "user"
> which would be useless.

If you use a HeapByteBuffer, then there's an extra copy from the native
buffer to the Java buffer.

>
>
> On 26/10/2016 11:57, Pavel Rappo wrote:
>
>> I believe I see where you coming from. Please correct me if I'm wrong.
>>
>> Your implementation is based on the premise that a call to
>> ReadableByteChannel.read()
>> _initiates_ the operation and returns immediately. The OS then continues
>> to fill
>> the buffer while there's a free space in the buffer and the channel
>> hasn't encountered EOF.
>>
>> Is that right?
>>
>> On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com> wrote:
>>>
>>> Thank you for your time. I'll try to explain it. I hope I can clear it
>>> up.
>>> First of it, I made a meaning mistake between asynchronous and
>>> non-blocking. This implementation uses a non-blocking algorithm internally
>>> while providing a blocking-like algorithm on the surface. It is
>>> single-threaded and not multi-threaded where one thread fetches data and
>>> blocks waiting and the other accumulates it and provides to whichever wants
>>> it.
>>>
>>> Second of it, I had made a mistake of going after BufferedReader instead
>>> of going after BufferedInputStream. If you want me to go after
>>> BufferedReader it's ok but I only thought that going after
>>> BufferedInputStream would be more generically useful than
>>> BufferedReaderwhen I started the poc.
>>>
>>> On to my code:
>>> Short answers:
>>>         ? The sleep(int) exists because I don't know how to wait until
>>> more data exists in the buffer which is part of read()'s contract.
>>>         ? The ByteBuffer gives a buffer that is filled by the OS (what I
>>> believe Channels do) instead of getting data only         by demand (what I
>>> believe Streams do).
>>> Full answers:
>>> The blockingFill(boolean) method is a method for a busy wait for a fill
>>> which is used exclusively by the read() method. All other methods use the
>>> version that does not sleep (fill(boolean)).
>>> blockingFill(boolean)'s existance like that is only because the read()
>>> method must not return unless either:
>>>
>>>         ? The stream ended.
>>>         ? The next byte is ready for reading.
>>> Additionally, statistically, that while loop will rarely evaluate to
>>> true as reads are in chunks so readPos will be behind writePos most of the
>>> time.
>>> I have no idea if an interrupt will ever happen, to be honest. The main
>>> reasons why I'm using a sleep is because I didn't want a hog onto the CPU
>>> in a full thread usage busy wait and because I didn't find any way of doing
>>> a thread sleep in order to wake up later when the buffer managed by native
>>> code has more data.
>>> The Non-blocking part is managed by the buffer the OS keeps filling most
>>> if not all the time. That buffer is the field
>>>
>>> ByteBuffer readBuffer
>>> That's the gaining part against the plain old Buffered classes.
>>>
>>>
>>> Did that make sense to you? Feel free to ask anything else you need.
>>>
>>> On 25/10/2016 20:52, Pavel Rappo wrote:
>>>
>>>> I've skimmed through the code and I'm not sure I can see any
>>>> asynchronicity
>>>> (you were pointing at the lack of it in BufferedReader).
>>>> And the mechanics of this is very puzzling to me, to be honest:
>>>>      void blockingFill(boolean forced) throws IOException {
>>>>          fill(forced);
>>>>          while (readPos == writePos) {
>>>>              try {
>>>>                  Thread.sleep(100);
>>>>              } catch (InterruptedException e) {
>>>>                  // An interrupt may mean more data is available
>>>>              }
>>>>              fill(forced);
>>>>          }
>>>>      }
>>>> I thought you were suggesting that we should utilize the tools which OS
>>>> provides
>>>> more efficiently. Instead we have something that looks very similarly
>>>> to a
>>>> "busy loop" and... also who and when is supposed to interrupt
>>>> Thread.sleep()?
>>>> Sorry, I'm not following. Could you please explain how this is supposed
>>>> to work?
>>>>
>>>> On 24 Oct 2016, at 15:59, Brunoais <brunoaiss at gmail.com>
>>>>>   wrote:
>>>>> Attached and sending!
>>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>>
>>>>> Could you please send a new email on this list with the source
>>>>>> attached as a
>>>>>> text file?
>>>>>>
>>>>>> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com>
>>>>>>>   wrote:
>>>>>>> Here's my poc/prototype:
>>>>>>>
>>>>>>> http://pastebin.com/WRpYWDJF
>>>>>>>
>>>>>>> I've implemented the bare minimum of the class that follows the same
>>>>>>> contract of BufferedReader while signaling all issues I think it may have
>>>>>>> or has in comments.
>>>>>>> I also wrote some javadoc to help guiding through the class.
>>>>>>> I could have used more fields from BufferedReader but the names were
>>>>>>> so minimalistic that were confusing me. I intent to change them before
>>>>>>> sending this to openJDK.
>>>>>>> One of the major problems this has is long overflowing. It is major
>>>>>>> because it is hidden, it will be extremely rare and it takes a really long
>>>>>>> time to reproduce. There are different ways of dealing with it. From just
>>>>>>> documenting to actually making code that works with it.
>>>>>>> I built a simple test code for it to have some ideas about
>>>>>>> performance and correctness.
>>>>>>>
>>>>>>> http://pastebin.com/eh6LFgwT
>>>>>>>
>>>>>>> This doesn't do a through test if it is actually working correctly
>>>>>>> but I see no reason for it not working correctly after fixing the 2 bugs
>>>>>>> that test found.
>>>>>>> I'll also leave here some conclusions about speed and resource
>>>>>>> consumption I found.
>>>>>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B.
>>>>>>> I noticed that, with my hardware, with the 1 530 000 000B file, I was
>>>>>>> getting around:
>>>>>>> In all buffers and fake work: 10~15s speed improvement ( from 90%
>>>>>>> HDD speed to 100% HDD speed)
>>>>>>> In all buffers and no fake work: 1~2s speed improvement ( from 90%
>>>>>>> HDD speed to 100% HDD speed)
>>>>>>> Changing the buffer size was giving different reading speeds but
>>>>>>> both were quite equal in how much they would change when changing the
>>>>>>> buffer size.
>>>>>>> Finally, I could always confirm that I/O was always the slowest
>>>>>>> thing while this code was running.
>>>>>>> For the ones wondering about the file size; it is both to avoid OS
>>>>>>> cache and to make the reading at the main use-case these objects are for
>>>>>>> (large streams of bytes).
>>>>>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>>>>
>>>>>>> Just to append to my previous email. BufferedReader wraps any Reader
>>>>>>>> out there.
>>>>>>>> Not specifically FileReader. While you're talking about the case of
>>>>>>>> effective
>>>>>>>> reading from a file.
>>>>>>>> I guess there's one existing possibility to provide exactly what
>>>>>>>> you need (as I
>>>>>>>> understand it) under this method:
>>>>>>>> /**
>>>>>>>>   * Opens a file for reading, returning a {@code BufferedReader} to
>>>>>>>> read text
>>>>>>>>   * from the file in an efficient manner...
>>>>>>>>     ...
>>>>>>>>   */
>>>>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>>>> It can return _anything_ as long as it is a BufferedReader. We can
>>>>>>>> do it, but it
>>>>>>>> needs to be investigated not only for your favorite OS but for
>>>>>>>> other OSes as
>>>>>>>> well. Feel free to prototype this and we can discuss it on the list
>>>>>>>> later.
>>>>>>>> Thanks,
>>>>>>>> -Pavel
>>>>>>>>
>>>>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com>
>>>>>>>>>   wrote:
>>>>>>>>> Pavel is right.
>>>>>>>>> In reality, I was expecting such BufferedReader to use only a
>>>>>>>>> single buffer and have that Buffer being filled asynchronously, not in a
>>>>>>>>> different Thread.
>>>>>>>>> Additionally, I don't have the intention of having a larger buffer
>>>>>>>>> than before unless stated through the API (the constructor).
>>>>>>>>> In my idea, internally, it is supposed to use
>>>>>>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>>>>> It does not prevent having two buffers and I do not intent to
>>>>>>>>> change BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any
>>>>>>>>> name suggestion is welcome as I'm an awful namer).
>>>>>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>>>>>
>>>>>>>>> Hi Pavel,
>>>>>>>>>> I think Brunoais asking for a double buffering scheme in which
>>>>>>>>>> the implementation of
>>>>>>>>>> BufferReader fills (a second buffer) in parallel with the
>>>>>>>>>> application reading from the 1st buffer
>>>>>>>>>> and managing the swaps and async reads transparently.
>>>>>>>>>> It would not change the API but would change the interactions
>>>>>>>>>> between the buffered reader
>>>>>>>>>> and the underlying stream.  It would also increase memory
>>>>>>>>>> requirements and processing
>>>>>>>>>> by introducing or using a separate thread and the necessary
>>>>>>>>>> synchronization.
>>>>>>>>>> Though I think the formal interface semantics could be
>>>>>>>>>> maintained, I have doubts
>>>>>>>>>> about compatibility and its unintended consequences on existing
>>>>>>>>>> subclasses,
>>>>>>>>>> applications and libraries.
>>>>>>>>>> $.02, Roger
>>>>>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>>>>>
>>>>>>>>>> Off the top of my head, I would say it's not possible to change
>>>>>>>>>>> the design of an
>>>>>>>>>>> _extensible_ type that has been out there for 20 or so years.
>>>>>>>>>>> All these I/O
>>>>>>>>>>> streams from java.io were designed for simple synchronous use
>>>>>>>>>>> case.
>>>>>>>>>>> It's not that their design is flawed in some way, it's that they
>>>>>>>>>>> doesn't seem to
>>>>>>>>>>> suit your needs. Have you considered using
>>>>>>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>>>>>>> in your applications?
>>>>>>>>>>> -Pavel
>>>>>>>>>>>
>>>>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com>
>>>>>>>>>>>>   wrote:
>>>>>>>>>>>> Any feedback on this? I'm really interested in implementing
>>>>>>>>>>>> such BufferedReader/BufferedStreamReader to allow speeding up
>>>>>>>>>>>> my applications without having to think in an asynchronous way or
>>>>>>>>>>>> multi-threading while programming with it.
>>>>>>>>>>>> That's why I'm asking this here.
>>>>>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> I looked at BufferedReader source code for java 9 long with
>>>>>>>>>>>>> the source code of the channels/streams used. I noticed that, like in java
>>>>>>>>>>>>> 7, BufferedReader does not use an Async API to load data from files,
>>>>>>>>>>>>> instead, the data loading is all done synchronously even when the OS allows
>>>>>>>>>>>>> requesting a file to be read and getting a warning later when the file is
>>>>>>>>>>>>> effectively read.
>>>>>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>>>>>>
>>>>>>>>>>>>> <BufferedNonBlockStream.java><Tests.java>
>>>>>
>>>>>
>>
>

-- 
Sent from my phone

From mandy.chung at oracle.com  Wed Oct 26 23:17:55 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 26 Oct 2016 16:17:55 -0700
Subject: RFR: 8163162: The separation between system loggers and
	application loggers should take the extension loader in consideration.
In-Reply-To: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
References: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
Message-ID: <672131AC-C031-4178-ABA8-E99C02CBA313@oracle.com>


> On Oct 26, 2016, at 6:58 AM, Daniel Fuchs <daniel.fuchs at oracle.com> wrote:
> 
> With the deprivileging of some JDK modules, classes loaded
> by the Platform class loader should get the same kind of
> loggers than classes loaded by the Boot class loader (null loader).
> 
> http://cr.openjdk.java.net/~dfuchs/webrev_8163162/webrev.00/
> 

The patch looks okay.

Nit:
  final boolean isSystem = AccessController.doPrivileged(?);
  return isSystem;

It could simply be:
    returns AccessController.doPrivileged(?);

Nit: The test uses @compile.  It can be @build that will avoid recompilation if the source is not modified.

  37  * @compile systempkg/log/SystemLoggerAccessor.java SystemLoggerInPlatformLoader.java

> Yes - I've been bitten before with using lambda
> in logging code - especially in those parts that
> can be invoked early during platform class
> initialization - so I tend to avoid using them
> in places that are in the code path triggered
> before the full initialization of the logging
> system.

I thought we identified a known issue and you have workaround it but still use method reference.  It?s okay to leave this one as is.  We had reworked the system initialization to enable lambda to be used very early when module system is initialized (after VM init phase 1 completes).  If you run into any issue, it?s likely a bug.

Mandy


From kumar.x.srinivasan at oracle.com  Thu Oct 27 01:20:34 2016
From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan)
Date: Wed, 26 Oct 2016 18:20:34 -0700
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
Message-ID: <58115662.6080005@oracle.com>

Hello,

Please review enclosed fix for: 
https://bugs.openjdk.java.net/browse/JDK-8168010

Based on this discussion, 
http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html
these options are slated for deprecation, only at the documentation level,
and will continue to operate with no perceivable behavior change, in JDK9.

Thanks
Kumar

diff --git 
a/src/java.base/share/classes/sun/launcher/resources/launcher.properties 
b/src/java.base/share/classes/sun/launcher/resources/launcher.properties
--- a/src/java.base/share/classes/sun/launcher/resources/launcher.properties
+++ b/src/java.base/share/classes/sun/launcher/resources/launcher.properties
@@ -1,5 +1,5 @@
  #
-# Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights 
reserved.
+# Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights 
reserved.
  # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  #
  # This code is free software; you can redistribute it and/or modify it
@@ -31,7 +31,7 @@
  \           (to execute the main class in a module)\n\
  where options include:\n

-java.launcher.opt.datamodel  =\    -d{0}\t  use a {0}-bit data model if 
available\n
+java.launcher.opt.datamodel  =\    -d{0}\t  Deprecated, will be removed 
in a future release\n
  java.launcher.opt.vmselect   =\    {0}\t  to select the "{1}" VM\n
  java.launcher.opt.hotspot    =\    {0}\t  is a synonym for the "{1}" 
VM  [deprecated]\n


From david.holmes at oracle.com  Thu Oct 27 02:57:23 2016
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 27 Oct 2016 12:57:23 +1000
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
Message-ID: <c6562d42-7e18-7c86-287c-0d1cc901517c@oracle.com>

On 25/10/2016 10:36 PM, Peter Levart wrote:
> Hi Claes,
>
>
> On 10/25/2016 01:09 PM, Aleksey Shipilev wrote:
>> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>>> small applications:
>>>
>>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
>> I would generally disagree to avoid Atomics to improve startup. In this
>> case, we always lock on close(), even for a short time. I wonder if
>> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
>> instead of introducing performance bugs with synchronized-s.
>>
>> Thanks,
>> -Aleksey
>>
>>
>
> In addition, there is no need for this:
>
>  379                     boolean closed;
>  380                     synchronized (closeLock) {
>  381                         closed = this.closed;
>  382                     }
>
> A simple:
>
>         boolean closed = this.closed;
>
> is equivalent, since this.closed is volatile.

Sorry but where exactly is the code referenced above?

Thanks,
David

>
> But something else bothers me with this code. There is a race. Here are
> the relevant parts:
>
>  316     public void close() throws IOException {
>  317         if (closed) {
>  318             return;
>  319         }
>  320         synchronized (closeLock) {
>  321             if (closed) {
>  322                 return;
>  323             }
>  324             closed = true;
>  325         }
>  326
>  327         FileChannel fc = channel;
>  328         if (fc != null) {
>  329            fc.close();
>  330         }
>  331
>  332         fd.closeAll(new Closeable() {
>  333             public void close() throws IOException {
>  334                close0();
>  335            }
>  336         });
>  337     }
>
> and:
>
>  372     public FileChannel getChannel() {
>  373         FileChannel fc = this.channel;
>  374         if (fc == null) {
>  375             synchronized (this) {
>  376                 fc = this.channel;
>  377                 if (fc == null) {
>  378                     this.channel = fc = FileChannelImpl.open(fd,
> path, true, false, this);
>  379                     boolean closed;
>  380                     synchronized (closeLock) {
>  381                         closed = this.closed;
>  382                     }
>  383                     if (closed) {
>  384                         try {
>  385                             fc.close();
>  386                         } catch (IOException ioe) {
>  387                             throw new InternalError(ioe); // should
> not happen
>  388                         }
>  389                     }
>  390                 }
>  391             }
>  392         }
>  393         return fc;
>  394     }
>
>
> Suppose Thread 1 is executing close() up to line 326, Then Thread 2
> kicks-in and executes getChannel() for the 1st time on this
> FileInputStream. It opens FileChannelImpl and finds closed flag already
> set, so it closes the channel. Thread 1 then resumes from line 326 and
> finds 'channel' field already set, so it closes the channel once more.
>
> FileChannelImpl.close() may be idempotent, but why not making sure it is
> called just once?
>
> In order to fix this, you should read the 'channel' field in close()
> while holding closeLock and you should publish the 'channel' field in
> getChannel() while holding closeLock. Like this:
>
> diff -r 2e7a303cd1ec
> src/java.base/share/classes/java/io/FileInputStream.java
> --- a/src/java.base/share/classes/java/io/FileInputStream.java  Wed Oct
> 19 11:45:43 2016 +0800
> +++ b/src/java.base/share/classes/java/io/FileInputStream.java  Tue Oct
> 25 14:33:05 2016 +0200
> @@ -26,7 +26,6 @@
>  package java.io;
>
>  import java.nio.channels.FileChannel;
> -import java.util.concurrent.atomic.AtomicBoolean;
>  import sun.nio.ch.FileChannelImpl;
>
>
> @@ -60,7 +59,9 @@
>
>      private volatile FileChannel channel;
>
> -    private final AtomicBoolean closed = new AtomicBoolean(false);
> +    private final Object closeLock = new Object();
> +
> +    private volatile boolean closed;
>
>      /**
>       * Creates a <code>FileInputStream</code> by
> @@ -313,12 +314,18 @@
>       * @spec JSR-51
>       */
>      public void close() throws IOException {
> -        if (!closed.compareAndSet(false, true)) {
> -            // if compareAndSet() returns false closed was already true
> +        if (closed) {
>              return;
>          }
> +        FileChannel fc;
> +        synchronized (closeLock) {
> +            if (closed) {
> +                return;
> +            }
> +            closed = true;
> +            fc = channel;
> +        }
>
> -        FileChannel fc = channel;
>          if (fc != null) {
>             fc.close();
>          }
> @@ -369,8 +376,13 @@
>              synchronized (this) {
>                  fc = this.channel;
>                  if (fc == null) {
> -                    this.channel = fc = FileChannelImpl.open(fd, path,
> true, false, this);
> -                    if (closed.get()) {
> +                    fc = FileChannelImpl.open(fd, path, true, false,
> this);
> +                    boolean closed;
> +                    synchronized (closeLock) {
> +                        closed = this.closed;
> +                        this.channel = fc;
> +                    }
> +                    if (closed) {
>                          try {
>                              fc.close();
>                          } catch (IOException ioe) {
>
>
> Regards, Peter
>
>

From peter.levart at gmail.com  Thu Oct 27 05:46:17 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 27 Oct 2016 07:46:17 +0200
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <c6562d42-7e18-7c86-287c-0d1cc901517c@oracle.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<c6562d42-7e18-7c86-287c-0d1cc901517c@oracle.com>
Message-ID: <0b5ad904-822c-1d6c-5e44-e1c103c77964@gmail.com>

Hi David,


On 10/27/2016 04:57 AM, David Holmes wrote:
> On 25/10/2016 10:36 PM, Peter Levart wrote:
>> Hi Claes,
>>
>>
>> On 10/25/2016 01:09 PM, Aleksey Shipilev wrote:
>>> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>>>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>>>> small applications:
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
>>> I would generally disagree to avoid Atomics to improve startup. In this
>>> case, we always lock on close(), even for a short time. I wonder if
>>> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
>>> instead of introducing performance bugs with synchronized-s.
>>>
>>> Thanks,
>>> -Aleksey
>>>
>>>
>>
>> In addition, there is no need for this:
>>
>>  379                     boolean closed;
>>  380                     synchronized (closeLock) {
>>  381                         closed = this.closed;
>>  382                     }
>>
>> A simple:
>>
>>         boolean closed = this.closed;
>>
>> is equivalent, since this.closed is volatile.
>
> Sorry but where exactly is the code referenced above?

It was in the previous webrev in the patched FileInputStream. But the 
webrev has been changed in-place...

Regards, Peter

>
> Thanks,
> David
>
>>
>> But something else bothers me with this code. There is a race. Here are
>> the relevant parts:
>>
>>  316     public void close() throws IOException {
>>  317         if (closed) {
>>  318             return;
>>  319         }
>>  320         synchronized (closeLock) {
>>  321             if (closed) {
>>  322                 return;
>>  323             }
>>  324             closed = true;
>>  325         }
>>  326
>>  327         FileChannel fc = channel;
>>  328         if (fc != null) {
>>  329            fc.close();
>>  330         }
>>  331
>>  332         fd.closeAll(new Closeable() {
>>  333             public void close() throws IOException {
>>  334                close0();
>>  335            }
>>  336         });
>>  337     }
>>
>> and:
>>
>>  372     public FileChannel getChannel() {
>>  373         FileChannel fc = this.channel;
>>  374         if (fc == null) {
>>  375             synchronized (this) {
>>  376                 fc = this.channel;
>>  377                 if (fc == null) {
>>  378                     this.channel = fc = FileChannelImpl.open(fd,
>> path, true, false, this);
>>  379                     boolean closed;
>>  380                     synchronized (closeLock) {
>>  381                         closed = this.closed;
>>  382                     }
>>  383                     if (closed) {
>>  384                         try {
>>  385                             fc.close();
>>  386                         } catch (IOException ioe) {
>>  387                             throw new InternalError(ioe); // should
>> not happen
>>  388                         }
>>  389                     }
>>  390                 }
>>  391             }
>>  392         }
>>  393         return fc;
>>  394     }
>>
>>
>> Suppose Thread 1 is executing close() up to line 326, Then Thread 2
>> kicks-in and executes getChannel() for the 1st time on this
>> FileInputStream. It opens FileChannelImpl and finds closed flag already
>> set, so it closes the channel. Thread 1 then resumes from line 326 and
>> finds 'channel' field already set, so it closes the channel once more.
>>
>> FileChannelImpl.close() may be idempotent, but why not making sure it is
>> called just once?
>>
>> In order to fix this, you should read the 'channel' field in close()
>> while holding closeLock and you should publish the 'channel' field in
>> getChannel() while holding closeLock. Like this:
>>
>> diff -r 2e7a303cd1ec
>> src/java.base/share/classes/java/io/FileInputStream.java
>> --- a/src/java.base/share/classes/java/io/FileInputStream.java Wed Oct
>> 19 11:45:43 2016 +0800
>> +++ b/src/java.base/share/classes/java/io/FileInputStream.java Tue Oct
>> 25 14:33:05 2016 +0200
>> @@ -26,7 +26,6 @@
>>  package java.io;
>>
>>  import java.nio.channels.FileChannel;
>> -import java.util.concurrent.atomic.AtomicBoolean;
>>  import sun.nio.ch.FileChannelImpl;
>>
>>
>> @@ -60,7 +59,9 @@
>>
>>      private volatile FileChannel channel;
>>
>> -    private final AtomicBoolean closed = new AtomicBoolean(false);
>> +    private final Object closeLock = new Object();
>> +
>> +    private volatile boolean closed;
>>
>>      /**
>>       * Creates a <code>FileInputStream</code> by
>> @@ -313,12 +314,18 @@
>>       * @spec JSR-51
>>       */
>>      public void close() throws IOException {
>> -        if (!closed.compareAndSet(false, true)) {
>> -            // if compareAndSet() returns false closed was already true
>> +        if (closed) {
>>              return;
>>          }
>> +        FileChannel fc;
>> +        synchronized (closeLock) {
>> +            if (closed) {
>> +                return;
>> +            }
>> +            closed = true;
>> +            fc = channel;
>> +        }
>>
>> -        FileChannel fc = channel;
>>          if (fc != null) {
>>             fc.close();
>>          }
>> @@ -369,8 +376,13 @@
>>              synchronized (this) {
>>                  fc = this.channel;
>>                  if (fc == null) {
>> -                    this.channel = fc = FileChannelImpl.open(fd, path,
>> true, false, this);
>> -                    if (closed.get()) {
>> +                    fc = FileChannelImpl.open(fd, path, true, false,
>> this);
>> +                    boolean closed;
>> +                    synchronized (closeLock) {
>> +                        closed = this.closed;
>> +                        this.channel = fc;
>> +                    }
>> +                    if (closed) {
>>                          try {
>>                              fc.close();
>>                          } catch (IOException ioe) {
>>
>>
>> Regards, Peter
>>
>>


From mandy.chung at oracle.com  Thu Oct 27 05:59:40 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 26 Oct 2016 22:59:40 -0700
Subject: Review Request: JDK-8168205: Should not default class path to CWD if
	-cp is not specified but -m is specified
Message-ID: <653C6BC1-90EE-4CF8-A3D0-CC63D98886B9@oracle.com>

Webrev:
   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.00/index.html

If -cp is not specified and -m is not specified, the builtin system
class loader will default the class path to the current working 
directory.  If -m is specified, no -cp and CLASSPATH environment 
variable is not set, it should mean no class path.  This patch fixes
the case if -m is specified and the value of java.class.path is empty,
e.g. via -Djava.class.path option, then no class path should be set.

This patch also updates the launcher code used for generating launcher
for JDK tools. As the JDK tool no longer passes any class path, it
removes APP_CLASSPATH macro.

Mandy

From brunoaiss at gmail.com  Thu Oct 27 06:20:20 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 07:20:20 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
Message-ID: <c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>

Did you read the C code?
Have you got any idea how many functions Windows or Linux (nearly all 
flavors) have for the read operation towards a file?

I have already done that homework myself. I may not have read JVM's 
source code but I know well that there's functions on both Windows and 
Linux that provide such interface I mentioned although they require a 
slightly different treatment (and different constants).


On 27/10/2016 00:06, Vitaly Davidovich wrote:
>
>
> On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
>
>     It is actually based on the premise that:
>
>     1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS
>        buffer size to fill in as the same size as ByteBuffer.
>
> Why do you say that? AFAICT, it issues a read syscall and that will 
> block if the data isn't in page cache.
>
>     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
>     orders
>        the JVM to order the OS to execute memcpy() to copy from its memory
>        to the shared memory created at ByteBuffer instantiation (in
>     java 8)
>        using Unsafe and then for the JVM to update the ByteBuffer fields.
>
> I think subsequent reads just invoke the same read syscall, passing 
> the current file offset maintained by the file channel instance.
>
>     3. The call will not block waiting for I/O and it won't take longer
>        than the JNI interface if no new data exists. However, it will
>     block
>        waiting for the OS to execute memcpy() to the shared memory.
>
> So why do you think it won't block?
>
>
>     Is my premise wrong?
>
>     If I read correctly, if I don't use a DirectBuffer, there would be
>     even another intermediate buffer to copy data to before giving it
>     to the "user" which would be useless.
>
> If you use a HeapByteBuffer, then there's an extra copy from the 
> native buffer to the Java buffer.
>
>
>
>     On 26/10/2016 11:57, Pavel Rappo wrote:
>
>         I believe I see where you coming from. Please correct me if
>         I'm wrong.
>
>         Your implementation is based on the premise that a call to
>         ReadableByteChannel.read()
>         _initiates_ the operation and returns immediately. The OS then
>         continues to fill
>         the buffer while there's a free space in the buffer and the
>         channel hasn't encountered EOF.
>
>         Is that right?
>
>             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
>             wrote:
>
>             Thank you for your time. I'll try to explain it. I hope I
>             can clear it up.
>             First of it, I made a meaning mistake between asynchronous
>             and non-blocking. This implementation uses a non-blocking
>             algorithm internally while providing a blocking-like
>             algorithm on the surface. It is single-threaded and not
>             multi-threaded where one thread fetches data and blocks
>             waiting and the other accumulates it and provides to
>             whichever wants it.
>
>             Second of it, I had made a mistake of going after
>             BufferedReader instead of going after BufferedInputStream.
>             If you want me to go after BufferedReader it's ok but I
>             only thought that going after BufferedInputStream would be
>             more generically useful than BufferedReaderwhen I started
>             the poc.
>
>             On to my code:
>             Short answers:
>                     ? The sleep(int) exists because I don't know how
>             to wait until more data exists in the buffer which is part
>             of read()'s contract.
>                     ? The ByteBuffer gives a buffer that is filled by
>             the OS (what I believe Channels do) instead of getting
>             data only         by demand (what I believe Streams do).
>             Full answers:
>             The blockingFill(boolean) method is a method for a busy
>             wait for a fill which is used exclusively by the read()
>             method. All other methods use the version that does not
>             sleep (fill(boolean)).
>             blockingFill(boolean)'s existance like that is only
>             because the read() method must not return unless either:
>
>                     ? The stream ended.
>                     ? The next byte is ready for reading.
>             Additionally, statistically, that while loop will rarely
>             evaluate to true as reads are in chunks so readPos will be
>             behind writePos most of the time.
>             I have no idea if an interrupt will ever happen, to be
>             honest. The main reasons why I'm using a sleep is because
>             I didn't want a hog onto the CPU in a full thread usage
>             busy wait and because I didn't find any way of doing a
>             thread sleep in order to wake up later when the buffer
>             managed by native code has more data.
>             The Non-blocking part is managed by the buffer the OS
>             keeps filling most if not all the time. That buffer is the
>             field
>
>             ByteBuffer readBuffer
>             That's the gaining part against the plain old Buffered
>             classes.
>
>
>             Did that make sense to you? Feel free to ask anything else
>             you need.
>
>             On 25/10/2016 20:52, Pavel Rappo wrote:
>
>                 I've skimmed through the code and I'm not sure I can
>                 see any asynchronicity
>                 (you were pointing at the lack of it in BufferedReader).
>                 And the mechanics of this is very puzzling to me, to
>                 be honest:
>                      void blockingFill(boolean forced) throws
>                 IOException {
>                          fill(forced);
>                          while (readPos == writePos) {
>                              try {
>                                  Thread.sleep(100);
>                              } catch (InterruptedException e) {
>                                  // An interrupt may mean more data is
>                 available
>                              }
>                              fill(forced);
>                          }
>                      }
>                 I thought you were suggesting that we should utilize
>                 the tools which OS provides
>                 more efficiently. Instead we have something that looks
>                 very similarly to a
>                 "busy loop" and... also who and when is supposed to
>                 interrupt Thread.sleep()?
>                 Sorry, I'm not following. Could you please explain how
>                 this is supposed to work?
>
>                     On 24 Oct 2016, at 15:59, Brunoais
>                     <brunoaiss at gmail.com>
>                       wrote:
>                     Attached and sending!
>                     On 24/10/2016 13:48, Pavel Rappo wrote:
>
>                         Could you please send a new email on this list
>                         with the source attached as a
>                         text file?
>
>                             On 23 Oct 2016, at 19:14, Brunoais
>                             <brunoaiss at gmail.com>
>                               wrote:
>                             Here's my poc/prototype:
>
>                             http://pastebin.com/WRpYWDJF
>
>                             I've implemented the bare minimum of the
>                             class that follows the same contract of
>                             BufferedReader while signaling all issues
>                             I think it may have or has in comments.
>                             I also wrote some javadoc to help guiding
>                             through the class.
>                             I could have used more fields from
>                             BufferedReader but the names were so
>                             minimalistic that were confusing me. I
>                             intent to change them before sending this
>                             to openJDK.
>                             One of the major problems this has is long
>                             overflowing. It is major because it is
>                             hidden, it will be extremely rare and it
>                             takes a really long time to reproduce.
>                             There are different ways of dealing with
>                             it. From just documenting to actually
>                             making code that works with it.
>                             I built a simple test code for it to have
>                             some ideas about performance and correctness.
>
>                             http://pastebin.com/eh6LFgwT
>
>                             This doesn't do a through test if it is
>                             actually working correctly but I see no
>                             reason for it not working correctly after
>                             fixing the 2 bugs that test found.
>                             I'll also leave here some conclusions
>                             about speed and resource consumption I found.
>                             I made tests with default buffer sizes,
>                             5000B 15_000B and 500_000B. I noticed
>                             that, with my hardware, with the 1 530 000
>                             000B file, I was getting around:
>                             In all buffers and fake work: 10~15s speed
>                             improvement ( from 90% HDD speed to 100%
>                             HDD speed)
>                             In all buffers and no fake work: 1~2s
>                             speed improvement ( from 90% HDD speed to
>                             100% HDD speed)
>                             Changing the buffer size was giving
>                             different reading speeds but both were
>                             quite equal in how much they would change
>                             when changing the buffer size.
>                             Finally, I could always confirm that I/O
>                             was always the slowest thing while this
>                             code was running.
>                             For the ones wondering about the file
>                             size; it is both to avoid OS cache and to
>                             make the reading at the main use-case
>                             these objects are for (large streams of
>                             bytes).
>                             @Pavel, are you open for discussion now
>                             ;)? Need anything else?
>                             On 21/10/2016 19:21, Pavel Rappo wrote:
>
>                                 Just to append to my previous email.
>                                 BufferedReader wraps any Reader out there.
>                                 Not specifically FileReader. While
>                                 you're talking about the case of effective
>                                 reading from a file.
>                                 I guess there's one existing
>                                 possibility to provide exactly what
>                                 you need (as I
>                                 understand it) under this method:
>                                 /**
>                                   * Opens a file for reading,
>                                 returning a {@code BufferedReader} to
>                                 read text
>                                   * from the file in an efficient
>                                 manner...
>                                     ...
>                                   */
>                                 java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>                                 It can return _anything_ as long as it
>                                 is a BufferedReader. We can do it, but it
>                                 needs to be investigated not only for
>                                 your favorite OS but for other OSes as
>                                 well. Feel free to prototype this and
>                                 we can discuss it on the list later.
>                                 Thanks,
>                                 -Pavel
>
>                                     On 21 Oct 2016, at 18:56, Brunoais
>                                     <brunoaiss at gmail.com>
>                                       wrote:
>                                     Pavel is right.
>                                     In reality, I was expecting such
>                                     BufferedReader to use only a
>                                     single buffer and have that Buffer
>                                     being filled asynchronously, not
>                                     in a different Thread.
>                                     Additionally, I don't have the
>                                     intention of having a larger
>                                     buffer than before unless stated
>                                     through the API (the constructor).
>                                     In my idea, internally, it is
>                                     supposed to use
>                                     java.nio.channels.AsynchronousFileChannel
>                                     or equivalent.
>                                     It does not prevent having two
>                                     buffers and I do not intent to
>                                     change BufferedReader itself. I'd
>                                     do an BufferedAsyncReader of sorts
>                                     (any name suggestion is welcome as
>                                     I'm an awful namer).
>                                     On 21/10/2016 18:38, Roger Riggs
>                                     wrote:
>
>                                         Hi Pavel,
>                                         I think Brunoais asking for a
>                                         double buffering scheme in
>                                         which the implementation of
>                                         BufferReader fills (a second
>                                         buffer) in parallel with the
>                                         application reading from the
>                                         1st buffer
>                                         and managing the swaps and
>                                         async reads transparently.
>                                         It would not change the API
>                                         but would change the
>                                         interactions between the
>                                         buffered reader
>                                         and the underlying stream.  It
>                                         would also increase memory
>                                         requirements and processing
>                                         by introducing or using a
>                                         separate thread and the
>                                         necessary synchronization.
>                                         Though I think the formal
>                                         interface semantics could be
>                                         maintained, I have doubts
>                                         about compatibility and its
>                                         unintended consequences on
>                                         existing subclasses,
>                                         applications and libraries.
>                                         $.02, Roger
>                                         On 10/21/16 1:22 PM, Pavel
>                                         Rappo wrote:
>
>                                             Off the top of my head, I
>                                             would say it's not
>                                             possible to change the
>                                             design of an
>                                             _extensible_ type that has
>                                             been out there for 20 or
>                                             so years. All these I/O
>                                             streams from java.io
>                                             <http://java.io> were
>                                             designed for simple
>                                             synchronous use case.
>                                             It's not that their design
>                                             is flawed in some way,
>                                             it's that they doesn't seem to
>                                             suit your needs. Have you
>                                             considered using
>                                             java.nio.channels.AsynchronousFileChannel
>                                             in your applications?
>                                             -Pavel
>
>                                                 On 21 Oct 2016, at
>                                                 17:08, Brunoais
>                                                 <brunoaiss at gmail.com>
>                                                   wrote:
>                                                 Any feedback on this?
>                                                 I'm really interested
>                                                 in implementing such
>                                                 BufferedReader/BufferedStreamReader
>                                                 to allow speeding up
>                                                 my applications
>                                                 without having to
>                                                 think in an
>                                                 asynchronous way or
>                                                 multi-threading while
>                                                 programming with it.
>                                                 That's why I'm asking
>                                                 this here.
>                                                 On 13/10/2016 14:45,
>                                                 Brunoais wrote:
>
>                                                     Hi,
>                                                     I looked at
>                                                     BufferedReader
>                                                     source code for
>                                                     java 9 long with
>                                                     the source code of
>                                                     the
>                                                     channels/streams
>                                                     used. I noticed
>                                                     that, like in java
>                                                     7, BufferedReader
>                                                     does not use an
>                                                     Async API to load
>                                                     data from files,
>                                                     instead, the data
>                                                     loading is all
>                                                     done synchronously
>                                                     even when the OS
>                                                     allows requesting
>                                                     a file to be read
>                                                     and getting a
>                                                     warning later when
>                                                     the file is
>                                                     effectively read.
>                                                     Why Is
>                                                     BufferedReader not
>                                                     async while
>                                                     providing a sync API?
>
>                     <BufferedNonBlockStream.java><Tests.java>
>
>
>
>
>
> -- 
> Sent from my phone


From Alan.Bateman at oracle.com  Thu Oct 27 08:03:55 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 27 Oct 2016 09:03:55 +0100
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
In-Reply-To: <58115662.6080005@oracle.com>
References: <58115662.6080005@oracle.com>
Message-ID: <7875836b-0cbe-1218-7bee-270dc2115858@oracle.com>

On 27/10/2016 02:20, Kumar Srinivasan wrote:

> Hello,
>
> Please review enclosed fix for: 
> https://bugs.openjdk.java.net/browse/JDK-8168010
>
> Based on this discussion, 
> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html
> these options are slated for deprecation, only at the documentation 
> level,
> and will continue to operate with no perceivable behavior change, in 
> JDK9.
This looks okay to me although I've no doubt that it will go unnoticed 
until the options are actually removed.

-Alan

From ecki at zusammenkunft.net  Thu Oct 27 09:28:30 2016
From: ecki at zusammenkunft.net (Bernd Eckenfels)
Date: Thu, 27 Oct 2016 11:28:30 +0200
Subject: Request/discussion: BufferedReader reading using async API
	while providing sync API
In-Reply-To: <c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
Message-ID: <20161027112830.00004636.ecki@zusammenkunft.net>

Hello,

the NIO2 code for files is pretty disappointing as it
does not use file completion ports on Windows or AIO on Linux.

You use the file channel, in that case it is blocking. It does not
implement SelectableChannel.

AsyncFileChannel uses Java threads in the background for the
completion handling (and last time I checked they are not signaled by
the OS). Performance with that was not very good for sequential
processing in my tests.


Gruss
Bernd


  Am Thu,
27 Oct 2016 07:20:20 +0100 schrieb Brunoais <brunoaiss at gmail.com>:

> Did you read the C code?
> Have you got any idea how many functions Windows or Linux (nearly all 
> flavors) have for the read operation towards a file?
> 
> I have already done that homework myself. I may not have read JVM's 
> source code but I know well that there's functions on both Windows
> and Linux that provide such interface I mentioned although they
> require a slightly different treatment (and different constants).
> 
> 
> On 27/10/2016 00:06, Vitaly Davidovich wrote:
> >
> >
> > On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com 
> > <mailto:brunoaiss at gmail.com>> wrote:
> >
> >     It is actually based on the premise that:
> >
> >     1. The first call to ReadableByteChannel.read(ByteBuffer) sets
> > the OS buffer size to fill in as the same size as ByteBuffer.
> >
> > Why do you say that? AFAICT, it issues a read syscall and that will 
> > block if the data isn't in page cache.
> >
> >     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
> >     orders
> >        the JVM to order the OS to execute memcpy() to copy from its
> > memory to the shared memory created at ByteBuffer instantiation (in
> >     java 8)
> >        using Unsafe and then for the JVM to update the ByteBuffer
> > fields.
> >
> > I think subsequent reads just invoke the same read syscall, passing 
> > the current file offset maintained by the file channel instance.
> >
> >     3. The call will not block waiting for I/O and it won't take
> > longer than the JNI interface if no new data exists. However, it
> > will block
> >        waiting for the OS to execute memcpy() to the shared memory.
> >
> > So why do you think it won't block?
> >
> >
> >     Is my premise wrong?
> >
> >     If I read correctly, if I don't use a DirectBuffer, there would
> > be even another intermediate buffer to copy data to before giving it
> >     to the "user" which would be useless.
> >
> > If you use a HeapByteBuffer, then there's an extra copy from the 
> > native buffer to the Java buffer.
> >
> >
> >
> >     On 26/10/2016 11:57, Pavel Rappo wrote:
> >
> >         I believe I see where you coming from. Please correct me if
> >         I'm wrong.
> >
> >         Your implementation is based on the premise that a call to
> >         ReadableByteChannel.read()
> >         _initiates_ the operation and returns immediately. The OS
> > then continues to fill
> >         the buffer while there's a free space in the buffer and the
> >         channel hasn't encountered EOF.
> >
> >         Is that right?
> >
> >             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
> >             wrote:
> >
> >             Thank you for your time. I'll try to explain it. I hope
> > I can clear it up.
> >             First of it, I made a meaning mistake between
> > asynchronous and non-blocking. This implementation uses a
> > non-blocking algorithm internally while providing a blocking-like
> >             algorithm on the surface. It is single-threaded and not
> >             multi-threaded where one thread fetches data and blocks
> >             waiting and the other accumulates it and provides to
> >             whichever wants it.
> >
> >             Second of it, I had made a mistake of going after
> >             BufferedReader instead of going after
> > BufferedInputStream. If you want me to go after BufferedReader it's
> > ok but I only thought that going after BufferedInputStream would be
> >             more generically useful than BufferedReaderwhen I
> > started the poc.
> >
> >             On to my code:
> >             Short answers:
> >                     ? The sleep(int) exists because I don't know how
> >             to wait until more data exists in the buffer which is
> > part of read()'s contract.
> >                     ? The ByteBuffer gives a buffer that is filled
> > by the OS (what I believe Channels do) instead of getting
> >             data only         by demand (what I believe Streams do).
> >             Full answers:
> >             The blockingFill(boolean) method is a method for a busy
> >             wait for a fill which is used exclusively by the read()
> >             method. All other methods use the version that does not
> >             sleep (fill(boolean)).
> >             blockingFill(boolean)'s existance like that is only
> >             because the read() method must not return unless either:
> >
> >                     ? The stream ended.
> >                     ? The next byte is ready for reading.
> >             Additionally, statistically, that while loop will rarely
> >             evaluate to true as reads are in chunks so readPos will
> > be behind writePos most of the time.
> >             I have no idea if an interrupt will ever happen, to be
> >             honest. The main reasons why I'm using a sleep is
> > because I didn't want a hog onto the CPU in a full thread usage
> >             busy wait and because I didn't find any way of doing a
> >             thread sleep in order to wake up later when the buffer
> >             managed by native code has more data.
> >             The Non-blocking part is managed by the buffer the OS
> >             keeps filling most if not all the time. That buffer is
> > the field
> >
> >             ByteBuffer readBuffer
> >             That's the gaining part against the plain old Buffered
> >             classes.
> >
> >
> >             Did that make sense to you? Feel free to ask anything
> > else you need.
> >
> >             On 25/10/2016 20:52, Pavel Rappo wrote:
> >
> >                 I've skimmed through the code and I'm not sure I can
> >                 see any asynchronicity
> >                 (you were pointing at the lack of it in
> > BufferedReader). And the mechanics of this is very puzzling to me,
> > to be honest:
> >                      void blockingFill(boolean forced) throws
> >                 IOException {
> >                          fill(forced);
> >                          while (readPos == writePos) {
> >                              try {
> >                                  Thread.sleep(100);
> >                              } catch (InterruptedException e) {
> >                                  // An interrupt may mean more data
> > is available
> >                              }
> >                              fill(forced);
> >                          }
> >                      }
> >                 I thought you were suggesting that we should utilize
> >                 the tools which OS provides
> >                 more efficiently. Instead we have something that
> > looks very similarly to a
> >                 "busy loop" and... also who and when is supposed to
> >                 interrupt Thread.sleep()?
> >                 Sorry, I'm not following. Could you please explain
> > how this is supposed to work?
> >
> >                     On 24 Oct 2016, at 15:59, Brunoais
> >                     <brunoaiss at gmail.com>
> >                       wrote:
> >                     Attached and sending!
> >                     On 24/10/2016 13:48, Pavel Rappo wrote:
> >
> >                         Could you please send a new email on this
> > list with the source attached as a
> >                         text file?
> >
> >                             On 23 Oct 2016, at 19:14, Brunoais
> >                             <brunoaiss at gmail.com>
> >                               wrote:
> >                             Here's my poc/prototype:
> >
> >                             http://pastebin.com/WRpYWDJF
> >
> >                             I've implemented the bare minimum of the
> >                             class that follows the same contract of
> >                             BufferedReader while signaling all
> > issues I think it may have or has in comments.
> >                             I also wrote some javadoc to help
> > guiding through the class.
> >                             I could have used more fields from
> >                             BufferedReader but the names were so
> >                             minimalistic that were confusing me. I
> >                             intent to change them before sending
> > this to openJDK.
> >                             One of the major problems this has is
> > long overflowing. It is major because it is
> >                             hidden, it will be extremely rare and it
> >                             takes a really long time to reproduce.
> >                             There are different ways of dealing with
> >                             it. From just documenting to actually
> >                             making code that works with it.
> >                             I built a simple test code for it to
> > have some ideas about performance and correctness.
> >
> >                             http://pastebin.com/eh6LFgwT
> >
> >                             This doesn't do a through test if it is
> >                             actually working correctly but I see no
> >                             reason for it not working correctly
> > after fixing the 2 bugs that test found.
> >                             I'll also leave here some conclusions
> >                             about speed and resource consumption I
> > found. I made tests with default buffer sizes,
> >                             5000B 15_000B and 500_000B. I noticed
> >                             that, with my hardware, with the 1 530
> > 000 000B file, I was getting around:
> >                             In all buffers and fake work: 10~15s
> > speed improvement ( from 90% HDD speed to 100%
> >                             HDD speed)
> >                             In all buffers and no fake work: 1~2s
> >                             speed improvement ( from 90% HDD speed
> > to 100% HDD speed)
> >                             Changing the buffer size was giving
> >                             different reading speeds but both were
> >                             quite equal in how much they would
> > change when changing the buffer size.
> >                             Finally, I could always confirm that I/O
> >                             was always the slowest thing while this
> >                             code was running.
> >                             For the ones wondering about the file
> >                             size; it is both to avoid OS cache and
> > to make the reading at the main use-case
> >                             these objects are for (large streams of
> >                             bytes).
> >                             @Pavel, are you open for discussion now
> >                             ;)? Need anything else?
> >                             On 21/10/2016 19:21, Pavel Rappo wrote:
> >
> >                                 Just to append to my previous email.
> >                                 BufferedReader wraps any Reader out
> > there. Not specifically FileReader. While
> >                                 you're talking about the case of
> > effective reading from a file.
> >                                 I guess there's one existing
> >                                 possibility to provide exactly what
> >                                 you need (as I
> >                                 understand it) under this method:
> >                                 /**
> >                                   * Opens a file for reading,
> >                                 returning a {@code BufferedReader}
> > to read text
> >                                   * from the file in an efficient
> >                                 manner...
> >                                     ...
> >                                   */
> >                                 java.nio.file.Files#newBufferedReader(java.nio.file.Path)
> >                                 It can return _anything_ as long as
> > it is a BufferedReader. We can do it, but it
> >                                 needs to be investigated not only
> > for your favorite OS but for other OSes as
> >                                 well. Feel free to prototype this
> > and we can discuss it on the list later.
> >                                 Thanks,
> >                                 -Pavel
> >
> >                                     On 21 Oct 2016, at 18:56,
> > Brunoais <brunoaiss at gmail.com>
> >                                       wrote:
> >                                     Pavel is right.
> >                                     In reality, I was expecting such
> >                                     BufferedReader to use only a
> >                                     single buffer and have that
> > Buffer being filled asynchronously, not
> >                                     in a different Thread.
> >                                     Additionally, I don't have the
> >                                     intention of having a larger
> >                                     buffer than before unless stated
> >                                     through the API (the
> > constructor). In my idea, internally, it is
> >                                     supposed to use
> >                                     java.nio.channels.AsynchronousFileChannel
> >                                     or equivalent.
> >                                     It does not prevent having two
> >                                     buffers and I do not intent to
> >                                     change BufferedReader itself.
> > I'd do an BufferedAsyncReader of sorts
> >                                     (any name suggestion is welcome
> > as I'm an awful namer).
> >                                     On 21/10/2016 18:38, Roger Riggs
> >                                     wrote:
> >
> >                                         Hi Pavel,
> >                                         I think Brunoais asking for
> > a double buffering scheme in
> >                                         which the implementation of
> >                                         BufferReader fills (a second
> >                                         buffer) in parallel with the
> >                                         application reading from the
> >                                         1st buffer
> >                                         and managing the swaps and
> >                                         async reads transparently.
> >                                         It would not change the API
> >                                         but would change the
> >                                         interactions between the
> >                                         buffered reader
> >                                         and the underlying stream.
> > It would also increase memory
> >                                         requirements and processing
> >                                         by introducing or using a
> >                                         separate thread and the
> >                                         necessary synchronization.
> >                                         Though I think the formal
> >                                         interface semantics could be
> >                                         maintained, I have doubts
> >                                         about compatibility and its
> >                                         unintended consequences on
> >                                         existing subclasses,
> >                                         applications and libraries.
> >                                         $.02, Roger
> >                                         On 10/21/16 1:22 PM, Pavel
> >                                         Rappo wrote:
> >
> >                                             Off the top of my head,
> > I would say it's not
> >                                             possible to change the
> >                                             design of an
> >                                             _extensible_ type that
> > has been out there for 20 or
> >                                             so years. All these I/O
> >                                             streams from java.io
> >                                             <http://java.io> were
> >                                             designed for simple
> >                                             synchronous use case.
> >                                             It's not that their
> > design is flawed in some way,
> >                                             it's that they doesn't
> > seem to suit your needs. Have you
> >                                             considered using
> >                                             java.nio.channels.AsynchronousFileChannel
> >                                             in your applications?
> >                                             -Pavel
> >
> >                                                 On 21 Oct 2016, at
> >                                                 17:08, Brunoais
> >                                                 <brunoaiss at gmail.com>
> >                                                   wrote:
> >                                                 Any feedback on
> > this? I'm really interested
> >                                                 in implementing such
> >                                                 BufferedReader/BufferedStreamReader
> >                                                 to allow speeding up
> >                                                 my applications
> >                                                 without having to
> >                                                 think in an
> >                                                 asynchronous way or
> >                                                 multi-threading
> > while programming with it.
> >                                                 That's why I'm
> > asking this here.
> >                                                 On 13/10/2016 14:45,
> >                                                 Brunoais wrote:
> >
> >                                                     Hi,
> >                                                     I looked at
> >                                                     BufferedReader
> >                                                     source code for
> >                                                     java 9 long with
> >                                                     the source code
> > of the
> >                                                     channels/streams
> >                                                     used. I noticed
> >                                                     that, like in
> > java 7, BufferedReader
> >                                                     does not use an
> >                                                     Async API to
> > load data from files,
> >                                                     instead, the
> > data loading is all
> >                                                     done
> > synchronously even when the OS
> >                                                     allows
> > requesting a file to be read
> >                                                     and getting a
> >                                                     warning later
> > when the file is
> >                                                     effectively
> > read. Why Is
> >                                                     BufferedReader
> > not async while
> >                                                     providing a
> > sync API?
> >
> >                     <BufferedNonBlockStream.java><Tests.java>
> >
> >
> >
> >
> >
> > -- 
> > Sent from my phone
> 


From Alan.Bateman at oracle.com  Thu Oct 27 09:44:15 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 27 Oct 2016 10:44:15 +0100
Subject: Review Request: JDK-8168205: Should not default class path to CWD
	if -cp is not specified but -m is specified
In-Reply-To: <0C45950A-566E-4673-91CE-99EC59848F64@oracle.com>
References: <0C45950A-566E-4673-91CE-99EC59848F64@oracle.com>
Message-ID: <80fdbd10-a88e-bf85-ffe9-3af907a2053e@oracle.com>

On 27/10/2016 07:05, Mandy Chung wrote:

> Webrev:
>    http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.00/index.html
>
> If -cp is not specified and -m is not specified, the builtin system
> class loader will default the class path to the current working
> directory.  If -m is specified, no -cp and CLASSPATH environment
> variable is not set, it should mean no class path.  This patch fixes
> the case if -m is specified and the value of java.class.path is empty,
> e.g. via -Djava.class.path option, then no class path should be set.
>
> This patch also updates the launcher code used for generating launcher
> for JDK tools. As the JDK tool no longer passes any class path, it
> removes APP_CLASSPATH macro.
>
The updated comment in ClassLoaders might be a bit clearer if you drop 
"if defined".

A minor comment on the test is that it could use 
ProcessTools.executeTestJava to avoid needing JAVA_TOOL.

-Alan

From Alan.Bateman at oracle.com  Thu Oct 27 09:46:36 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 27 Oct 2016 10:46:36 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <20161027112830.00004636.ecki@zusammenkunft.net>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<20161027112830.00004636.ecki@zusammenkunft.net>
Message-ID: <85b7a9cb-e989-35e9-97ca-39b32fc83f54@oracle.com>

On 27/10/2016 10:28, Bernd Eckenfels wrote:

> :
>
> AsyncFileChannel uses Java threads in the background for the
> completion handling (and last time I checked they are not signaled by
> the OS). Performance with that was not very good for sequential
> processing in my tests.
>
>
It does use completion ports on Windows but there has be a thread to 
read the completion status. In any case, it's not really suitable here 
as I think the original poster is looking read-ahead sequential access 
ather than concurrent access to different parts of the file.

-Alan

From daniel.fuchs at oracle.com  Thu Oct 27 10:30:16 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 27 Oct 2016 11:30:16 +0100
Subject: RFR: 8163162: The separation between system loggers and
	application loggers should take the extension loader in consideration.
In-Reply-To: <672131AC-C031-4178-ABA8-E99C02CBA313@oracle.com>
References: <963c9911-93e4-703d-dc51-86ea9988731c@oracle.com>
	<672131AC-C031-4178-ABA8-E99C02CBA313@oracle.com>
Message-ID: <f758ff2a-4b93-26f8-9b34-d972aff54197@oracle.com>

On 27/10/16 00:17, Mandy Chung wrote:
>
>> On Oct 26, 2016, at 6:58 AM, Daniel Fuchs <daniel.fuchs at oracle.com> wrote:
>>
>> With the deprivileging of some JDK modules, classes loaded
>> by the Platform class loader should get the same kind of
>> loggers than classes loaded by the Boot class loader (null loader).
>>
>> http://cr.openjdk.java.net/~dfuchs/webrev_8163162/webrev.00/
>>
>
> The patch looks okay.
>
> Nit:
>   final boolean isSystem = AccessController.doPrivileged(?);
>   return isSystem;
>
> It could simply be:
>     returns AccessController.doPrivileged(?);
>
> Nit: The test uses @compile.  It can be @build that will avoid recompilation if the source is not modified.
>
>   37  * @compile systempkg/log/SystemLoggerAccessor.java SystemLoggerInPlatformLoader.java

Thanks Mandy - I will do these before pushing.

best regards,

-- daniel


>> Yes - I've been bitten before with using lambda
>> in logging code - especially in those parts that
>> can be invoked early during platform class
>> initialization - so I tend to avoid using them
>> in places that are in the code path triggered
>> before the full initialization of the logging
>> system.
>
> I thought we identified a known issue and you have workaround it but still use method reference.  It?s okay to leave this one as is.  We had reworked the system initialization to enable lambda to be used very early when module system is initialized (after VM init phase 1 completes).  If you run into any issue, it?s likely a bug.
>
> Mandy
>


From vitalyd at gmail.com  Thu Oct 27 10:47:09 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Thu, 27 Oct 2016 06:47:09 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
Message-ID: <CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>

On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com> wrote:

> Did you read the C code?

I looked at the Linux code in the JDK.

> Have you got any idea how many functions Windows or Linux (nearly all
> flavors) have for the read operation towards a file?

I do.

>
> I have already done that homework myself. I may not have read JVM's source
> code but I know well that there's functions on both Windows and Linux that
> provide such interface I mentioned although they require a slightly
> different treatment (and different constants).

You should read the JDK (native) source code instead of guessing/assuming.
On Linux, it doesn't use aio facilities for files.  The kernel io scheduler
may issue readahead behind the scenes, but there's no nonblocking file io
that's at the heart of your premise.

>
>
> On 27/10/2016 00:06, Vitaly Davidovich wrote:
>
>>
>>
>> On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com <mailto:
>> brunoaiss at gmail.com>> wrote:
>>
>>     It is actually based on the premise that:
>>
>>     1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS
>>        buffer size to fill in as the same size as ByteBuffer.
>>
>> Why do you say that? AFAICT, it issues a read syscall and that will block
>> if the data isn't in page cache.
>>
>>     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
>>     orders
>>        the JVM to order the OS to execute memcpy() to copy from its memory
>>        to the shared memory created at ByteBuffer instantiation (in
>>     java 8)
>>        using Unsafe and then for the JVM to update the ByteBuffer fields.
>>
>> I think subsequent reads just invoke the same read syscall, passing the
>> current file offset maintained by the file channel instance.
>>
>>     3. The call will not block waiting for I/O and it won't take longer
>>        than the JNI interface if no new data exists. However, it will
>>     block
>>        waiting for the OS to execute memcpy() to the shared memory.
>>
>> So why do you think it won't block?
>>
>>
>>     Is my premise wrong?
>>
>>     If I read correctly, if I don't use a DirectBuffer, there would be
>>     even another intermediate buffer to copy data to before giving it
>>     to the "user" which would be useless.
>>
>> If you use a HeapByteBuffer, then there's an extra copy from the native
>> buffer to the Java buffer.
>>
>>
>>
>>     On 26/10/2016 11:57, Pavel Rappo wrote:
>>
>>         I believe I see where you coming from. Please correct me if
>>         I'm wrong.
>>
>>         Your implementation is based on the premise that a call to
>>         ReadableByteChannel.read()
>>         _initiates_ the operation and returns immediately. The OS then
>>         continues to fill
>>         the buffer while there's a free space in the buffer and the
>>         channel hasn't encountered EOF.
>>
>>         Is that right?
>>
>>             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
>>             wrote:
>>
>>             Thank you for your time. I'll try to explain it. I hope I
>>             can clear it up.
>>             First of it, I made a meaning mistake between asynchronous
>>             and non-blocking. This implementation uses a non-blocking
>>             algorithm internally while providing a blocking-like
>>             algorithm on the surface. It is single-threaded and not
>>             multi-threaded where one thread fetches data and blocks
>>             waiting and the other accumulates it and provides to
>>             whichever wants it.
>>
>>             Second of it, I had made a mistake of going after
>>             BufferedReader instead of going after BufferedInputStream.
>>             If you want me to go after BufferedReader it's ok but I
>>             only thought that going after BufferedInputStream would be
>>             more generically useful than BufferedReaderwhen I started
>>             the poc.
>>
>>             On to my code:
>>             Short answers:
>>                     ? The sleep(int) exists because I don't know how
>>             to wait until more data exists in the buffer which is part
>>             of read()'s contract.
>>                     ? The ByteBuffer gives a buffer that is filled by
>>             the OS (what I believe Channels do) instead of getting
>>             data only         by demand (what I believe Streams do).
>>             Full answers:
>>             The blockingFill(boolean) method is a method for a busy
>>             wait for a fill which is used exclusively by the read()
>>             method. All other methods use the version that does not
>>             sleep (fill(boolean)).
>>             blockingFill(boolean)'s existance like that is only
>>             because the read() method must not return unless either:
>>
>>                     ? The stream ended.
>>                     ? The next byte is ready for reading.
>>             Additionally, statistically, that while loop will rarely
>>             evaluate to true as reads are in chunks so readPos will be
>>             behind writePos most of the time.
>>             I have no idea if an interrupt will ever happen, to be
>>             honest. The main reasons why I'm using a sleep is because
>>             I didn't want a hog onto the CPU in a full thread usage
>>             busy wait and because I didn't find any way of doing a
>>             thread sleep in order to wake up later when the buffer
>>             managed by native code has more data.
>>             The Non-blocking part is managed by the buffer the OS
>>             keeps filling most if not all the time. That buffer is the
>>             field
>>
>>             ByteBuffer readBuffer
>>             That's the gaining part against the plain old Buffered
>>             classes.
>>
>>
>>             Did that make sense to you? Feel free to ask anything else
>>             you need.
>>
>>             On 25/10/2016 20:52, Pavel Rappo wrote:
>>
>>                 I've skimmed through the code and I'm not sure I can
>>                 see any asynchronicity
>>                 (you were pointing at the lack of it in BufferedReader).
>>                 And the mechanics of this is very puzzling to me, to
>>                 be honest:
>>                      void blockingFill(boolean forced) throws
>>                 IOException {
>>                          fill(forced);
>>                          while (readPos == writePos) {
>>                              try {
>>                                  Thread.sleep(100);
>>                              } catch (InterruptedException e) {
>>                                  // An interrupt may mean more data is
>>                 available
>>                              }
>>                              fill(forced);
>>                          }
>>                      }
>>                 I thought you were suggesting that we should utilize
>>                 the tools which OS provides
>>                 more efficiently. Instead we have something that looks
>>                 very similarly to a
>>                 "busy loop" and... also who and when is supposed to
>>                 interrupt Thread.sleep()?
>>                 Sorry, I'm not following. Could you please explain how
>>                 this is supposed to work?
>>
>>                     On 24 Oct 2016, at 15:59, Brunoais
>>                     <brunoaiss at gmail.com>
>>                       wrote:
>>                     Attached and sending!
>>                     On 24/10/2016 13:48, Pavel Rappo wrote:
>>
>>                         Could you please send a new email on this list
>>                         with the source attached as a
>>                         text file?
>>
>>                             On 23 Oct 2016, at 19:14, Brunoais
>>                             <brunoaiss at gmail.com>
>>                               wrote:
>>                             Here's my poc/prototype:
>>
>>                             http://pastebin.com/WRpYWDJF
>>
>>                             I've implemented the bare minimum of the
>>                             class that follows the same contract of
>>                             BufferedReader while signaling all issues
>>                             I think it may have or has in comments.
>>                             I also wrote some javadoc to help guiding
>>                             through the class.
>>                             I could have used more fields from
>>                             BufferedReader but the names were so
>>                             minimalistic that were confusing me. I
>>                             intent to change them before sending this
>>                             to openJDK.
>>                             One of the major problems this has is long
>>                             overflowing. It is major because it is
>>                             hidden, it will be extremely rare and it
>>                             takes a really long time to reproduce.
>>                             There are different ways of dealing with
>>                             it. From just documenting to actually
>>                             making code that works with it.
>>                             I built a simple test code for it to have
>>                             some ideas about performance and correctness.
>>
>>                             http://pastebin.com/eh6LFgwT
>>
>>                             This doesn't do a through test if it is
>>                             actually working correctly but I see no
>>                             reason for it not working correctly after
>>                             fixing the 2 bugs that test found.
>>                             I'll also leave here some conclusions
>>                             about speed and resource consumption I found.
>>                             I made tests with default buffer sizes,
>>                             5000B 15_000B and 500_000B. I noticed
>>                             that, with my hardware, with the 1 530 000
>>                             000B file, I was getting around:
>>                             In all buffers and fake work: 10~15s speed
>>                             improvement ( from 90% HDD speed to 100%
>>                             HDD speed)
>>                             In all buffers and no fake work: 1~2s
>>                             speed improvement ( from 90% HDD speed to
>>                             100% HDD speed)
>>                             Changing the buffer size was giving
>>                             different reading speeds but both were
>>                             quite equal in how much they would change
>>                             when changing the buffer size.
>>                             Finally, I could always confirm that I/O
>>                             was always the slowest thing while this
>>                             code was running.
>>                             For the ones wondering about the file
>>                             size; it is both to avoid OS cache and to
>>                             make the reading at the main use-case
>>                             these objects are for (large streams of
>>                             bytes).
>>                             @Pavel, are you open for discussion now
>>                             ;)? Need anything else?
>>                             On 21/10/2016 19:21, Pavel Rappo wrote:
>>
>>                                 Just to append to my previous email.
>>                                 BufferedReader wraps any Reader out there.
>>                                 Not specifically FileReader. While
>>                                 you're talking about the case of effective
>>                                 reading from a file.
>>                                 I guess there's one existing
>>                                 possibility to provide exactly what
>>                                 you need (as I
>>                                 understand it) under this method:
>>                                 /**
>>                                   * Opens a file for reading,
>>                                 returning a {@code BufferedReader} to
>>                                 read text
>>                                   * from the file in an efficient
>>                                 manner...
>>                                     ...
>>                                   */
>>                                 java.nio.file.Files#newBuffere
>> dReader(java.nio.file.Path)
>>                                 It can return _anything_ as long as it
>>                                 is a BufferedReader. We can do it, but it
>>                                 needs to be investigated not only for
>>                                 your favorite OS but for other OSes as
>>                                 well. Feel free to prototype this and
>>                                 we can discuss it on the list later.
>>                                 Thanks,
>>                                 -Pavel
>>
>>                                     On 21 Oct 2016, at 18:56, Brunoais
>>                                     <brunoaiss at gmail.com>
>>                                       wrote:
>>                                     Pavel is right.
>>                                     In reality, I was expecting such
>>                                     BufferedReader to use only a
>>                                     single buffer and have that Buffer
>>                                     being filled asynchronously, not
>>                                     in a different Thread.
>>                                     Additionally, I don't have the
>>                                     intention of having a larger
>>                                     buffer than before unless stated
>>                                     through the API (the constructor).
>>                                     In my idea, internally, it is
>>                                     supposed to use
>>                                     java.nio.channels.Asynchronous
>> FileChannel
>>                                     or equivalent.
>>                                     It does not prevent having two
>>                                     buffers and I do not intent to
>>                                     change BufferedReader itself. I'd
>>                                     do an BufferedAsyncReader of sorts
>>                                     (any name suggestion is welcome as
>>                                     I'm an awful namer).
>>                                     On 21/10/2016 18:38, Roger Riggs
>>                                     wrote:
>>
>>                                         Hi Pavel,
>>                                         I think Brunoais asking for a
>>                                         double buffering scheme in
>>                                         which the implementation of
>>                                         BufferReader fills (a second
>>                                         buffer) in parallel with the
>>                                         application reading from the
>>                                         1st buffer
>>                                         and managing the swaps and
>>                                         async reads transparently.
>>                                         It would not change the API
>>                                         but would change the
>>                                         interactions between the
>>                                         buffered reader
>>                                         and the underlying stream.  It
>>                                         would also increase memory
>>                                         requirements and processing
>>                                         by introducing or using a
>>                                         separate thread and the
>>                                         necessary synchronization.
>>                                         Though I think the formal
>>                                         interface semantics could be
>>                                         maintained, I have doubts
>>                                         about compatibility and its
>>                                         unintended consequences on
>>                                         existing subclasses,
>>                                         applications and libraries.
>>                                         $.02, Roger
>>                                         On 10/21/16 1:22 PM, Pavel
>>                                         Rappo wrote:
>>
>>                                             Off the top of my head, I
>>                                             would say it's not
>>                                             possible to change the
>>                                             design of an
>>                                             _extensible_ type that has
>>                                             been out there for 20 or
>>                                             so years. All these I/O
>>                                             streams from java.io
>>                                             <http://java.io> were
>>                                             designed for simple
>>                                             synchronous use case.
>>                                             It's not that their design
>>                                             is flawed in some way,
>>                                             it's that they doesn't seem to
>>                                             suit your needs. Have you
>>                                             considered using
>>                                             java.nio.channels.Asynchronous
>> FileChannel
>>                                             in your applications?
>>                                             -Pavel
>>
>>                                                 On 21 Oct 2016, at
>>                                                 17:08, Brunoais
>>                                                 <brunoaiss at gmail.com>
>>                                                   wrote:
>>                                                 Any feedback on this?
>>                                                 I'm really interested
>>                                                 in implementing such
>>
>> BufferedReader/BufferedStreamReader
>>                                                 to allow speeding up
>>                                                 my applications
>>                                                 without having to
>>                                                 think in an
>>                                                 asynchronous way or
>>                                                 multi-threading while
>>                                                 programming with it.
>>                                                 That's why I'm asking
>>                                                 this here.
>>                                                 On 13/10/2016 14:45,
>>                                                 Brunoais wrote:
>>
>>                                                     Hi,
>>                                                     I looked at
>>                                                     BufferedReader
>>                                                     source code for
>>                                                     java 9 long with
>>                                                     the source code of
>>                                                     the
>>                                                     channels/streams
>>                                                     used. I noticed
>>                                                     that, like in java
>>                                                     7, BufferedReader
>>                                                     does not use an
>>                                                     Async API to load
>>                                                     data from files,
>>                                                     instead, the data
>>                                                     loading is all
>>                                                     done synchronously
>>                                                     even when the OS
>>                                                     allows requesting
>>                                                     a file to be read
>>                                                     and getting a
>>                                                     warning later when
>>                                                     the file is
>>                                                     effectively read.
>>                                                     Why Is
>>                                                     BufferedReader not
>>                                                     async while
>>                                                     providing a sync API?
>>
>>                     <BufferedNonBlockStream.java><Tests.java>
>>
>>
>>
>>
>>
>> --
>> Sent from my phone
>>
>
>

-- 
Sent from my phone

From brunoaiss at gmail.com  Thu Oct 27 12:20:53 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 13:20:53 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
	<c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
Message-ID: <aeac24a7-63c9-1640-f1a9-b120b91972fe@gmail.com>

Hey.

Any idea how to skip tests? When testing for BufferedInputStream, the 
directBufferSize is not used so testing with different directBufferSize 
makes no sense.

I already tried "return 0" on the benchmarked test but jmh fills the 
output with " (*interrupt*)" if I do that.


On 26/10/2016 21:41, Peter Levart wrote:
>
>
>
> On 10/26/2016 09:46 PM, Brunoais wrote:
>>
>> Thank you.
>>
>> Only one thing left. How can I "burn" the OS' file read cache?
>> I only know how to do that by allocating a very large amount of 
>> memory based on the information I see in the resource manager 
>> (windows) or system monitor (linux) of the cached I/O and run the 
>> program. In this case, I have no idea how much memory each one's 
>> computer has so I cannot use the same method. How would you do such 
>> program excerpt?
>>
>> As for the rest of the pointers: thank you I'll start building the 
>> benchmark code based on that information.
>>
>
> Here's a prototype you can extend. It'll make you jump-start in 5th gear:
>
> http://cr.openjdk.java.net/~plevart/misc/FileReadBench/FileReadBench.java
>
> Regards, Peter
>
>>
>> On 26/10/2016 18:24, Peter Levart wrote:
>>> Hi Brunoais,
>>>
>>> I'll try to tell what I know from my JMH practice:
>>>
>>> On 10/26/2016 10:30 AM, Brunoais wrote:
>>>> Hey guys. Any idea where I can find instructions on how to use JMH to:
>>>>
>>>> 1. Clear OS' file reading cache.
>>>
>>> You can create a public void method and make it called by JMH before 
>>> each:
>>> - trial (a set of iterations)
>>> - iteration (a set of test method invocations)
>>> - invocation
>>>
>>> ...simply by annotating it by @Setup( [ Level.Trial | 
>>> Level.Iteration | Level.Invocation ] ).
>>>
>>> So create a method that spawns a script that clears the cache.
>>>
>>>> 2. Warm up whatever it needs to (maybe reading from a Channel in 
>>>> memory).
>>>
>>> JMH already warms-up the code and VM simply be executing "warmup" 
>>> iterations before starting real measured iterations. You can control 
>>> the number of warm-up iterations and real measured iterations by 
>>> annotating either the class or the method(s) with:
>>>
>>> @Warmup(iterations = ...)
>>> @Measurement(iterations = ...)
>>>
>>> If you want to warm-up resources by code that is not equal to code 
>>> in test method(s) then maybe @Setup methods on different levels 
>>> could be used for that.
>>>
>>>>
>>>> 3. Create a BufferedInputStream with a FileInputStream inside, with
>>>>    configurable buffer sizes.
>>>
>>> You can annotate a field of int, long or String type of a class 
>>> annotated with @State annotation (can be the benchmark class itself) 
>>> with @Param annotation, enumerating values this field will get 
>>> before executing the @Setup(Level.Trial) method(s). So you enumerate 
>>> the buffer sizes in @Param annotation and instantiate the 
>>> BufferedInputStream using the value in @Setup method. Viola.
>>>
>>>> 4. Execute iterations to read the file fully.
>>>
>>> Then perhaps you could use only one invocation per iteration and 
>>> measured it using @BenchmarkMode(Mode.SingleShotTime), constructing 
>>> the loop by yourself.
>>>
>>>> 1. Allow setting the byte[] size.
>>>
>>> Use @Parameter on a field to hold the byte[] size and create the 
>>> byte[] in @Setup method...
>>>
>>>> 2. On each iteration, burn a set number of CPU cycles.
>>>
>>> BlackHole.consumeCPU(tokens)
>>>
>>>> 5. Re-execute 1, 3 and 4 but with a BufferedNonBlockStream and a
>>>>    FileChannel.
>>>
>>> If you wrap them all into a common API (by delegation), you can use 
>>> @Parameter String implType, with @Setup method to instantiate the 
>>> appropriate implementation. Then just invoke the common API in the 
>>> test method.
>>>
>>>>
>>>> So far I still can't find how to:
>>>>
>>>> 1 (clear OS' cache)
>>>> 3 (the configuration part)
>>>> 4 (variable number of iterations)
>>>> 4.1 (the configuration)
>>>>
>>>> Can someone please point me in the right direction?
>>>
>>> I can create an example test if you like and you can then extend it...
>>>
>>> Regards, Peter
>>>
>>>>
>>>>
>>>> On 26/10/2016 07:57, Brunoais wrote:
>>>>>
>>>>> Hey Bernd!
>>>>>
>>>>> I don't know how far back you did such thing but I'm getting 
>>>>> positive results with my non-JMH tests. I do have to evaluate my 
>>>>> results against logic. After some reads, the OS starts caching the 
>>>>> file which is not what I want. It's easy to know when that 
>>>>> happens, though. The times fall from ~30s to ~5s and the HDD keeps 
>>>>> near idle reading (just looking at the LED is enough to understand).
>>>>>
>>>>> If you don't test synchronous work and you only run the reads, you 
>>>>> will only get marginal results as the OS has no real time to fill 
>>>>> the buffer.
>>>>> My research shows the 2 major kernels (windows' and GNU/Linux) 
>>>>> have non-blocking user-level buffer handling where I give a buffer 
>>>>> for the OS to read and it keeps filling it and sending 
>>>>> messages/signals as it writes chunks. Linux has an OS interrupt 
>>>>> that only sends the signal after it is full, though. There's also 
>>>>> another version of them where they use an internal buffer of same 
>>>>> size as the buffer you allocate for the OS and then internally 
>>>>> call memcopy() into your user-level memory when asked. Tests on 
>>>>> the internet show that memcopy is as fast (for 0-1 elements) or 
>>>>> faster than System.arraycopy(). I have no idea if they are true.
>>>>>
>>>>> All this was for me to add that, that code is tuned to copy from 
>>>>> the read buffer only when it is, at least, at half capacity and 
>>>>> the internal buffer has enough storage space. The process is 
>>>>> forced only if nothing had been read on the previous fill() call. 
>>>>> It is built to use JNI as little as possible while providing the 
>>>>> major contract BufferedInputStream has.
>>>>> Finally, I never, ever compact the read buffer. It requires doing 
>>>>> a memcopy which is definitely not necessary.
>>>>>
>>>>> Anyway, those tests about time I made were just to get an order of 
>>>>> magnitude about speed difference. I intended to do them 
>>>>> differently but JMH looks good so I'll use JMH to test now.
>>>>>
>>>>> Short reads only happen when fill(true) is called. That happens 
>>>>> for desperate get of data.
>>>>>
>>>>> I'll look into the avoiding double reading requests. I do think it 
>>>>> won't bring significant improvements if any at all. It only 
>>>>> happens when the buffer is nearly empty and any byte of data is 
>>>>> welcome "at any cost".
>>>>> Besides, whomever called read at that point would also have had an 
>>>>> availability() of 0 and still called read()/read(byte[]).
>>>>>
>>>>>
>>>>> On 26/10/2016 06:14, Bernd Eckenfels wrote:
>>>>>>  Hallo Brunoais,
>>>>>>
>>>>>> In the past I die some experiments with non-blocking file 
>>>>>> channels in the hope to increase throughput in a similiar way 
>>>>>> then your buffered stream. I also used direct allocated buffers. 
>>>>>> However my results have not been that encouraging (especially If 
>>>>>> a upper layer used larger reads). I thought back in the time this 
>>>>>> was mostly die to the fact that it NOT wraps to real AsyncFIO on 
>>>>>> most platforms. But maybe I just measured it wrong, so I will 
>>>>>> have a closer look on your impl.
>>>>>>
>>>>>> Generally I would recommend to make the Benchmark a bit more 
>>>>>> reliable with JMH and in order to do this to externalize the 
>>>>>> direct buffer allocation (as it ist slow if done repeatingly). 
>>>>>> This also allows you to publish some results with varrying 
>>>>>> workloads (on different machines).
>>>>>>
>>>>>> I would also measure the readCount to see if short reads happen.
>>>>>>
>>>>>>  BTW, I might as well try to only read till the end of the buffer 
>>>>>> in the backfilling-wraps-around case and not issue two requests, 
>>>>>> that might remove some additional latency.
>>>>>>
>>>>>> Gruss
>>>>>> Bernd
>>>>>> -- 
>>>>>> http://bernd.eckenfels.net
>>>>>>
>>>>>> _____________________________
>>>>>> From: Brunoais <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>>> Sent: Montag, Oktober 24, 2016 6:30 PM
>>>>>> Subject: Re: Request/discussion: BufferedReader reading using 
>>>>>> async API while providing sync API
>>>>>> To: Pavel Rappo <pavel.rappo at oracle.com 
>>>>>> <mailto:pavel.rappo at oracle.com>>
>>>>>> Cc: <core-libs-dev at openjdk.java.net 
>>>>>> <mailto:core-libs-dev at openjdk.java.net>>
>>>>>>
>>>>>>
>>>>>> Attached and sending!
>>>>>>
>>>>>>
>>>>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>>> > Could you please send a new email on this list with the source 
>>>>>> attached as a
>>>>>> > text file?
>>>>>> >
>>>>>> >> On 23 Oct 2016, at 19:14, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>
>>>>>> >> Here's my poc/prototype:
>>>>>> >> http://pastebin.com/WRpYWDJF
>>>>>> >>
>>>>>> >> I've implemented the bare minimum of the class that follows 
>>>>>> the same contract of BufferedReader while signaling all issues I 
>>>>>> think it may have or has in comments.
>>>>>> >> I also wrote some javadoc to help guiding through the class.
>>>>>> >>
>>>>>> >> I could have used more fields from BufferedReader but the 
>>>>>> names were so minimalistic that were confusing me. I intent to 
>>>>>> change them before sending this to openJDK.
>>>>>> >>
>>>>>> >> One of the major problems this has is long overflowing. It is 
>>>>>> major because it is hidden, it will be extremely rare and it 
>>>>>> takes a really long time to reproduce. There are different ways 
>>>>>> of dealing with it. From just documenting to actually making code 
>>>>>> that works with it.
>>>>>> >>
>>>>>> >> I built a simple test code for it to have some ideas about 
>>>>>> performance and correctness.
>>>>>> >>
>>>>>> >> http://pastebin.com/eh6LFgwT
>>>>>> >>
>>>>>> >> This doesn't do a through test if it is actually working 
>>>>>> correctly but I see no reason for it not working correctly after 
>>>>>> fixing the 2 bugs that test found.
>>>>>> >>
>>>>>> >> I'll also leave here some conclusions about speed and resource 
>>>>>> consumption I found.
>>>>>> >>
>>>>>> >> I made tests with default buffer sizes, 5000B 15_000B and 
>>>>>> 500_000B. I noticed that, with my hardware, with the 1 530 000 
>>>>>> 000B file, I was getting around:
>>>>>> >>
>>>>>> >> In all buffers and fake work: 10~15s speed improvement ( from 
>>>>>> 90% HDD speed to 100% HDD speed)
>>>>>> >> In all buffers and no fake work: 1~2s speed improvement ( from 
>>>>>> 90% HDD speed to 100% HDD speed)
>>>>>> >>
>>>>>> >> Changing the buffer size was giving different reading speeds 
>>>>>> but both were quite equal in how much they would change when 
>>>>>> changing the buffer size.
>>>>>> >> Finally, I could always confirm that I/O was always the 
>>>>>> slowest thing while this code was running.
>>>>>> >>
>>>>>> >> For the ones wondering about the file size; it is both to 
>>>>>> avoid OS cache and to make the reading at the main use-case these 
>>>>>> objects are for (large streams of bytes).
>>>>>> >>
>>>>>> >> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>>> >>
>>>>>> >> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>>> >>> Just to append to my previous email. BufferedReader wraps any 
>>>>>> Reader out there.
>>>>>> >>> Not specifically FileReader. While you're talking about the 
>>>>>> case of effective
>>>>>> >>> reading from a file.
>>>>>> >>>
>>>>>> >>> I guess there's one existing possibility to provide exactly 
>>>>>> what you need (as I
>>>>>> >>> understand it) under this method:
>>>>>> >>>
>>>>>> >>> /**
>>>>>> >>> * Opens a file for reading, returning a {@code 
>>>>>> BufferedReader} to read text
>>>>>> >>> * from the file in an efficient manner...
>>>>>> >>> ...
>>>>>> >>> */
>>>>>> >>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>> >>>
>>>>>> >>> It can return _anything_ as long as it is a BufferedReader. 
>>>>>> We can do it, but it
>>>>>> >>> needs to be investigated not only for your favorite OS but 
>>>>>> for other OSes as
>>>>>> >>> well. Feel free to prototype this and we can discuss it on 
>>>>>> the list later.
>>>>>> >>>
>>>>>> >>> Thanks,
>>>>>> >>> -Pavel
>>>>>> >>>
>>>>>> >>>> On 21 Oct 2016, at 18:56, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>>>
>>>>>> >>>> Pavel is right.
>>>>>> >>>>
>>>>>> >>>> In reality, I was expecting such BufferedReader to use only 
>>>>>> a single buffer and have that Buffer being filled asynchronously, 
>>>>>> not in a different Thread.
>>>>>> >>>> Additionally, I don't have the intention of having a larger 
>>>>>> buffer than before unless stated through the API (the constructor).
>>>>>> >>>>
>>>>>> >>>> In my idea, internally, it is supposed to use 
>>>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>> >>>>
>>>>>> >>>> It does not prevent having two buffers and I do not intent 
>>>>>> to change BufferedReader itself. I'd do an BufferedAsyncReader of 
>>>>>> sorts (any name suggestion is welcome as I'm an awful namer).
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>> >>>>> Hi Pavel,
>>>>>> >>>>>
>>>>>> >>>>> I think Brunoais asking for a double buffering scheme in 
>>>>>> which the implementation of
>>>>>> >>>>> BufferReader fills (a second buffer) in parallel with the 
>>>>>> application reading from the 1st buffer
>>>>>> >>>>> and managing the swaps and async reads transparently.
>>>>>> >>>>> It would not change the API but would change the 
>>>>>> interactions between the buffered reader
>>>>>> >>>>> and the underlying stream. It would also increase memory 
>>>>>> requirements and processing
>>>>>> >>>>> by introducing or using a separate thread and the necessary 
>>>>>> synchronization.
>>>>>> >>>>>
>>>>>> >>>>> Though I think the formal interface semantics could be 
>>>>>> maintained, I have doubts
>>>>>> >>>>> about compatibility and its unintended consequences on 
>>>>>> existing subclasses,
>>>>>> >>>>> applications and libraries.
>>>>>> >>>>>
>>>>>> >>>>> $.02, Roger
>>>>>> >>>>>
>>>>>> >>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>> >>>>>> Off the top of my head, I would say it's not possible to 
>>>>>> change the design of an
>>>>>> >>>>>> _extensible_ type that has been out there for 20 or so 
>>>>>> years. All these I/O
>>>>>> >>>>>> streams from java.io <http://java.io> were designed for 
>>>>>> simple synchronous use case.
>>>>>> >>>>>>
>>>>>> >>>>>> It's not that their design is flawed in some way, it's 
>>>>>> that they doesn't seem to
>>>>>> >>>>>> suit your needs. Have you considered using 
>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>> >>>>>> in your applications?
>>>>>> >>>>>>
>>>>>> >>>>>> -Pavel
>>>>>> >>>>>>
>>>>>> >>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoaiss at gmail.com 
>>>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>> Any feedback on this? I'm really interested in 
>>>>>> implementing such BufferedReader/BufferedStreamReader to allow 
>>>>>> speeding up my applications without having to think in an 
>>>>>> asynchronous way or multi-threading while programming with it.
>>>>>> >>>>>>>
>>>>>> >>>>>>> That's why I'm asking this here.
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>> >>>>>>>> Hi,
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> I looked at BufferedReader source code for java 9 long 
>>>>>> with the source code of the channels/streams used. I noticed 
>>>>>> that, like in java 7, BufferedReader does not use an Async API to 
>>>>>> load data from files, instead, the data loading is all done 
>>>>>> synchronously even when the OS allows requesting a file to be 
>>>>>> read and getting a warning later when the file is effectively read.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>> >>>>>>>>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From brunoaiss at gmail.com  Thu Oct 27 12:34:05 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 13:34:05 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
Message-ID: <7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>

Oh... I see. In that case, it means something is terribly wrong. It can 
be my initial tests, though.

I'm testing on both linux and windows and I'm getting performance gains 
from using the FileChannel compared to using FileInputStream... The 
tests also make sense based on my predictions O_O...


On 27/10/2016 11:47, Vitaly Davidovich wrote:
>
>
> On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
>
>     Did you read the C code?
>
> I looked at the Linux code in the JDK.
>
>     Have you got any idea how many functions Windows or Linux (nearly
>     all flavors) have for the read operation towards a file?
>
> I do.
>
>
>     I have already done that homework myself. I may not have read
>     JVM's source code but I know well that there's functions on both
>     Windows and Linux that provide such interface I mentioned although
>     they require a slightly different treatment (and different constants).
>
> You should read the JDK (native) source code instead of 
> guessing/assuming.  On Linux, it doesn't use aio facilities for 
> files.  The kernel io scheduler may issue readahead behind the scenes, 
> but there's no nonblocking file io that's at the heart of your premise.
>
>
>
>     On 27/10/2016 00:06, Vitaly Davidovich wrote:
>
>
>
>         On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com
>         <mailto:brunoaiss at gmail.com>> wrote:
>
>             It is actually based on the premise that:
>
>             1. The first call to ReadableByteChannel.read(ByteBuffer)
>         sets the OS
>                buffer size to fill in as the same size as ByteBuffer.
>
>         Why do you say that? AFAICT, it issues a read syscall and that
>         will block if the data isn't in page cache.
>
>             2. The consecutive calls to
>         ReadableByteChannel.read(ByteBuffer)
>             orders
>                the JVM to order the OS to execute memcpy() to copy
>         from its memory
>                to the shared memory created at ByteBuffer
>         instantiation (in
>             java 8)
>                using Unsafe and then for the JVM to update the
>         ByteBuffer fields.
>
>         I think subsequent reads just invoke the same read syscall,
>         passing the current file offset maintained by the file channel
>         instance.
>
>             3. The call will not block waiting for I/O and it won't
>         take longer
>                than the JNI interface if no new data exists. However,
>         it will
>             block
>                waiting for the OS to execute memcpy() to the shared
>         memory.
>
>         So why do you think it won't block?
>
>
>             Is my premise wrong?
>
>             If I read correctly, if I don't use a DirectBuffer, there
>         would be
>             even another intermediate buffer to copy data to before
>         giving it
>             to the "user" which would be useless.
>
>         If you use a HeapByteBuffer, then there's an extra copy from
>         the native buffer to the Java buffer.
>
>
>
>             On 26/10/2016 11:57, Pavel Rappo wrote:
>
>                 I believe I see where you coming from. Please correct
>         me if
>                 I'm wrong.
>
>                 Your implementation is based on the premise that a call to
>                 ReadableByteChannel.read()
>                 _initiates_ the operation and returns immediately. The
>         OS then
>                 continues to fill
>                 the buffer while there's a free space in the buffer
>         and the
>                 channel hasn't encountered EOF.
>
>                 Is that right?
>
>                     On 25 Oct 2016, at 22:16, Brunoais
>         <brunoaiss at gmail.com>
>                     wrote:
>
>                     Thank you for your time. I'll try to explain it. I
>         hope I
>                     can clear it up.
>                     First of it, I made a meaning mistake between
>         asynchronous
>                     and non-blocking. This implementation uses a
>         non-blocking
>                     algorithm internally while providing a blocking-like
>                     algorithm on the surface. It is single-threaded
>         and not
>                     multi-threaded where one thread fetches data and
>         blocks
>                     waiting and the other accumulates it and provides to
>                     whichever wants it.
>
>                     Second of it, I had made a mistake of going after
>                     BufferedReader instead of going after
>         BufferedInputStream.
>                     If you want me to go after BufferedReader it's ok
>         but I
>                     only thought that going after BufferedInputStream
>         would be
>                     more generically useful than BufferedReaderwhen I
>         started
>                     the poc.
>
>                     On to my code:
>                     Short answers:
>                             ? The sleep(int) exists because I don't
>         know how
>                     to wait until more data exists in the buffer which
>         is part
>                     of read()'s contract.
>                             ? The ByteBuffer gives a buffer that is
>         filled by
>                     the OS (what I believe Channels do) instead of getting
>                     data only         by demand (what I believe
>         Streams do).
>                     Full answers:
>                     The blockingFill(boolean) method is a method for a
>         busy
>                     wait for a fill which is used exclusively by the
>         read()
>                     method. All other methods use the version that
>         does not
>                     sleep (fill(boolean)).
>                     blockingFill(boolean)'s existance like that is only
>                     because the read() method must not return unless
>         either:
>
>                             ? The stream ended.
>                             ? The next byte is ready for reading.
>                     Additionally, statistically, that while loop will
>         rarely
>                     evaluate to true as reads are in chunks so readPos
>         will be
>                     behind writePos most of the time.
>                     I have no idea if an interrupt will ever happen, to be
>                     honest. The main reasons why I'm using a sleep is
>         because
>                     I didn't want a hog onto the CPU in a full thread
>         usage
>                     busy wait and because I didn't find any way of doing a
>                     thread sleep in order to wake up later when the buffer
>                     managed by native code has more data.
>                     The Non-blocking part is managed by the buffer the OS
>                     keeps filling most if not all the time. That
>         buffer is the
>                     field
>
>                     ByteBuffer readBuffer
>                     That's the gaining part against the plain old Buffered
>                     classes.
>
>
>                     Did that make sense to you? Feel free to ask
>         anything else
>                     you need.
>
>                     On 25/10/2016 20:52, Pavel Rappo wrote:
>
>                         I've skimmed through the code and I'm not sure
>         I can
>                         see any asynchronicity
>                         (you were pointing at the lack of it in
>         BufferedReader).
>                         And the mechanics of this is very puzzling to
>         me, to
>                         be honest:
>                              void blockingFill(boolean forced) throws
>                         IOException {
>                                  fill(forced);
>                                  while (readPos == writePos) {
>                                      try {
>                                          Thread.sleep(100);
>                                      } catch (InterruptedException e) {
>                                          // An interrupt may mean more
>         data is
>                         available
>                                      }
>                                      fill(forced);
>                                  }
>                              }
>                         I thought you were suggesting that we should
>         utilize
>                         the tools which OS provides
>                         more efficiently. Instead we have something
>         that looks
>                         very similarly to a
>                         "busy loop" and... also who and when is
>         supposed to
>                         interrupt Thread.sleep()?
>                         Sorry, I'm not following. Could you please
>         explain how
>                         this is supposed to work?
>
>                             On 24 Oct 2016, at 15:59, Brunoais
>                             <brunoaiss at gmail.com>
>                               wrote:
>                             Attached and sending!
>                             On 24/10/2016 13:48, Pavel Rappo wrote:
>
>                                 Could you please send a new email on
>         this list
>                                 with the source attached as a
>                                 text file?
>
>                                     On 23 Oct 2016, at 19:14, Brunoais
>                                     <brunoaiss at gmail.com>
>                                       wrote:
>                                     Here's my poc/prototype:
>
>         http://pastebin.com/WRpYWDJF
>
>                                     I've implemented the bare minimum
>         of the
>                                     class that follows the same
>         contract of
>                                     BufferedReader while signaling all
>         issues
>                                     I think it may have or has in
>         comments.
>                                     I also wrote some javadoc to help
>         guiding
>                                     through the class.
>                                     I could have used more fields from
>                                     BufferedReader but the names were so
>                                     minimalistic that were confusing me. I
>                                     intent to change them before
>         sending this
>                                     to openJDK.
>                                     One of the major problems this has
>         is long
>                                     overflowing. It is major because it is
>                                     hidden, it will be extremely rare
>         and it
>                                     takes a really long time to reproduce.
>                                     There are different ways of
>         dealing with
>                                     it. From just documenting to actually
>                                     making code that works with it.
>                                     I built a simple test code for it
>         to have
>                                     some ideas about performance and
>         correctness.
>
>         http://pastebin.com/eh6LFgwT
>
>                                     This doesn't do a through test if
>         it is
>                                     actually working correctly but I
>         see no
>                                     reason for it not working
>         correctly after
>                                     fixing the 2 bugs that test found.
>                                     I'll also leave here some conclusions
>                                     about speed and resource
>         consumption I found.
>                                     I made tests with default buffer
>         sizes,
>                                     5000B 15_000B and 500_000B. I noticed
>                                     that, with my hardware, with the 1
>         530 000
>                                     000B file, I was getting around:
>                                     In all buffers and fake work:
>         10~15s speed
>                                     improvement ( from 90% HDD speed
>         to 100%
>                                     HDD speed)
>                                     In all buffers and no fake work: 1~2s
>                                     speed improvement ( from 90% HDD
>         speed to
>                                     100% HDD speed)
>                                     Changing the buffer size was giving
>                                     different reading speeds but both were
>                                     quite equal in how much they would
>         change
>                                     when changing the buffer size.
>                                     Finally, I could always confirm
>         that I/O
>                                     was always the slowest thing while
>         this
>                                     code was running.
>                                     For the ones wondering about the file
>                                     size; it is both to avoid OS cache
>         and to
>                                     make the reading at the main use-case
>                                     these objects are for (large
>         streams of
>                                     bytes).
>                                     @Pavel, are you open for
>         discussion now
>                                     ;)? Need anything else?
>                                     On 21/10/2016 19:21, Pavel Rappo
>         wrote:
>
>                                         Just to append to my previous
>         email.
>                                         BufferedReader wraps any
>         Reader out there.
>                                         Not specifically FileReader. While
>                                         you're talking about the case
>         of effective
>                                         reading from a file.
>                                         I guess there's one existing
>                                         possibility to provide exactly
>         what
>                                         you need (as I
>                                         understand it) under this method:
>                                         /**
>                                           * Opens a file for reading,
>                                         returning a {@code
>         BufferedReader} to
>                                         read text
>                                           * from the file in an efficient
>                                         manner...
>                                             ...
>                                           */
>                                        
>         java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>                                         It can return _anything_ as
>         long as it
>                                         is a BufferedReader. We can do
>         it, but it
>                                         needs to be investigated not
>         only for
>                                         your favorite OS but for other
>         OSes as
>                                         well. Feel free to prototype
>         this and
>                                         we can discuss it on the list
>         later.
>                                         Thanks,
>                                         -Pavel
>
>                                             On 21 Oct 2016, at 18:56,
>         Brunoais
>                                             <brunoaiss at gmail.com>
>                                               wrote:
>                                             Pavel is right.
>                                             In reality, I was
>         expecting such
>                                             BufferedReader to use only a
>                                             single buffer and have
>         that Buffer
>                                             being filled
>         asynchronously, not
>                                             in a different Thread.
>                                             Additionally, I don't have the
>                                             intention of having a larger
>                                             buffer than before unless
>         stated
>                                             through the API (the
>         constructor).
>                                             In my idea, internally, it is
>                                             supposed to use
>         java.nio.channels.AsynchronousFileChannel
>                                             or equivalent.
>                                             It does not prevent having two
>                                             buffers and I do not intent to
>                                             change BufferedReader
>         itself. I'd
>                                             do an BufferedAsyncReader
>         of sorts
>                                             (any name suggestion is
>         welcome as
>                                             I'm an awful namer).
>                                             On 21/10/2016 18:38, Roger
>         Riggs
>                                             wrote:
>
>                                                 Hi Pavel,
>                                                 I think Brunoais
>         asking for a
>                                                 double buffering scheme in
>                                                 which the
>         implementation of
>                                                 BufferReader fills (a
>         second
>                                                 buffer) in parallel
>         with the
>                                                 application reading
>         from the
>                                                 1st buffer
>                                                 and managing the swaps and
>                                                 async reads transparently.
>                                                 It would not change
>         the API
>                                                 but would change the
>                                                 interactions between the
>                                                 buffered reader
>                                                 and the underlying
>         stream.  It
>                                                 would also increase memory
>                                                 requirements and
>         processing
>                                                 by introducing or using a
>                                                 separate thread and the
>                                                 necessary synchronization.
>                                                 Though I think the formal
>                                                 interface semantics
>         could be
>                                                 maintained, I have doubts
>                                                 about compatibility
>         and its
>                                                 unintended consequences on
>                                                 existing subclasses,
>                                                 applications and
>         libraries.
>                                                 $.02, Roger
>                                                 On 10/21/16 1:22 PM, Pavel
>                                                 Rappo wrote:
>
>                                                     Off the top of my
>         head, I
>                                                     would say it's not
>                                                     possible to change the
>                                                     design of an
>                                                     _extensible_ type
>         that has
>                                                     been out there for
>         20 or
>                                                     so years. All
>         these I/O
>                                                     streams from
>         java.io <http://java.io>
>                                                     <http://java.io> were
>                                                     designed for simple
>                                                     synchronous use case.
>                                                     It's not that
>         their design
>                                                     is flawed in some way,
>                                                     it's that they
>         doesn't seem to
>                                                     suit your needs.
>         Have you
>                                                     considered using
>         java.nio.channels.AsynchronousFileChannel
>                                                     in your applications?
>                                                     -Pavel
>
>                                                         On 21 Oct 2016, at
>                                                         17:08, Brunoais
>                                                        
>         <brunoaiss at gmail.com>
>                                                           wrote:
>                                                         Any feedback
>         on this?
>                                                         I'm really
>         interested
>                                                         in
>         implementing such
>         BufferedReader/BufferedStreamReader
>                                                         to allow
>         speeding up
>                                                         my applications
>                                                         without having to
>                                                         think in an
>                                                         asynchronous
>         way or
>         multi-threading while
>                                                         programming
>         with it.
>                                                         That's why I'm
>         asking
>                                                         this here.
>                                                         On 13/10/2016
>         14:45,
>                                                         Brunoais wrote:
>
>                                                             Hi,
>                                                             I looked at
>         BufferedReader
>                                                             source
>         code for
>                                                             java 9
>         long with
>                                                             the source
>         code of
>                                                             the
>         channels/streams
>                                                             used. I
>         noticed
>                                                             that, like
>         in java
>                                                             7,
>         BufferedReader
>                                                             does not
>         use an
>                                                             Async API
>         to load
>                                                             data from
>         files,
>                                                             instead,
>         the data
>                                                             loading is all
>                                                             done
>         synchronously
>                                                             even when
>         the OS
>                                                             allows
>         requesting
>                                                             a file to
>         be read
>                                                             and getting a
>                                                             warning
>         later when
>                                                             the file is
>         effectively read.
>                                                             Why Is
>         BufferedReader not
>                                                             async while
>                                                             providing
>         a sync API?
>
>                             <BufferedNonBlockStream.java><Tests.java>
>
>
>
>
>
>         -- 
>         Sent from my phone
>
>
>
>
> -- 
> Sent from my phone


From vitalyd at gmail.com  Thu Oct 27 12:53:09 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Thu, 27 Oct 2016 08:53:09 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
Message-ID: <CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>

On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com> wrote:

> Oh... I see. In that case, it means something is terribly wrong. It can be
> my initial tests, though.
>
> I'm testing on both linux and windows and I'm getting performance gains
> from using the FileChannel compared to using FileInputStream... The tests
> also make sense based on my predictions O_O...
>
FileInputStream requires copying native buffers holding the read data to
the java byte[].  If you're using direct ByteBuffer for FileChannel, that
whole memcpy is skipped.  Try comparing FileChannel with HeapByteBuffer
instead.

>
> On 27/10/2016 11:47, Vitaly Davidovich wrote:
>
>
>
> On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com> wrote:
>
>> Did you read the C code?
>
> I looked at the Linux code in the JDK.
>
>> Have you got any idea how many functions Windows or Linux (nearly all
>> flavors) have for the read operation towards a file?
>
> I do.
>
>>
>> I have already done that homework myself. I may not have read JVM's
>> source code but I know well that there's functions on both Windows and
>> Linux that provide such interface I mentioned although they require a
>> slightly different treatment (and different constants).
>
> You should read the JDK (native) source code instead of
> guessing/assuming.  On Linux, it doesn't use aio facilities for files.  The
> kernel io scheduler may issue readahead behind the scenes, but there's no
> nonblocking file io that's at the heart of your premise.
>
>>
>>
>> On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>
>>>
>>>
>>> On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com <mailto:
>>> brunoaiss at gmail.com>> wrote:
>>>
>>>     It is actually based on the premise that:
>>>
>>>     1. The first call to ReadableByteChannel.read(ByteBuffer) sets the
>>> OS
>>>        buffer size to fill in as the same size as ByteBuffer.
>>>
>>> Why do you say that? AFAICT, it issues a read syscall and that will
>>> block if the data isn't in page cache.
>>>
>>>     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
>>>     orders
>>>        the JVM to order the OS to execute memcpy() to copy from its
>>> memory
>>>        to the shared memory created at ByteBuffer instantiation (in
>>>     java 8)
>>>        using Unsafe and then for the JVM to update the ByteBuffer fields.
>>>
>>> I think subsequent reads just invoke the same read syscall, passing the
>>> current file offset maintained by the file channel instance.
>>>
>>>     3. The call will not block waiting for I/O and it won't take longer
>>>        than the JNI interface if no new data exists. However, it will
>>>     block
>>>        waiting for the OS to execute memcpy() to the shared memory.
>>>
>>> So why do you think it won't block?
>>>
>>>
>>>     Is my premise wrong?
>>>
>>>     If I read correctly, if I don't use a DirectBuffer, there would be
>>>     even another intermediate buffer to copy data to before giving it
>>>     to the "user" which would be useless.
>>>
>>> If you use a HeapByteBuffer, then there's an extra copy from the native
>>> buffer to the Java buffer.
>>>
>>>
>>>
>>>     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>
>>>         I believe I see where you coming from. Please correct me if
>>>         I'm wrong.
>>>
>>>         Your implementation is based on the premise that a call to
>>>         ReadableByteChannel.read()
>>>         _initiates_ the operation and returns immediately. The OS then
>>>         continues to fill
>>>         the buffer while there's a free space in the buffer and the
>>>         channel hasn't encountered EOF.
>>>
>>>         Is that right?
>>>
>>>             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
>>>             wrote:
>>>
>>>             Thank you for your time. I'll try to explain it. I hope I
>>>             can clear it up.
>>>             First of it, I made a meaning mistake between asynchronous
>>>             and non-blocking. This implementation uses a non-blocking
>>>             algorithm internally while providing a blocking-like
>>>             algorithm on the surface. It is single-threaded and not
>>>             multi-threaded where one thread fetches data and blocks
>>>             waiting and the other accumulates it and provides to
>>>             whichever wants it.
>>>
>>>             Second of it, I had made a mistake of going after
>>>             BufferedReader instead of going after BufferedInputStream.
>>>             If you want me to go after BufferedReader it's ok but I
>>>             only thought that going after BufferedInputStream would be
>>>             more generically useful than BufferedReaderwhen I started
>>>             the poc.
>>>
>>>             On to my code:
>>>             Short answers:
>>>                     ? The sleep(int) exists because I don't know how
>>>             to wait until more data exists in the buffer which is part
>>>             of read()'s contract.
>>>                     ? The ByteBuffer gives a buffer that is filled by
>>>             the OS (what I believe Channels do) instead of getting
>>>             data only         by demand (what I believe Streams do).
>>>             Full answers:
>>>             The blockingFill(boolean) method is a method for a busy
>>>             wait for a fill which is used exclusively by the read()
>>>             method. All other methods use the version that does not
>>>             sleep (fill(boolean)).
>>>             blockingFill(boolean)'s existance like that is only
>>>             because the read() method must not return unless either:
>>>
>>>                     ? The stream ended.
>>>                     ? The next byte is ready for reading.
>>>             Additionally, statistically, that while loop will rarely
>>>             evaluate to true as reads are in chunks so readPos will be
>>>             behind writePos most of the time.
>>>             I have no idea if an interrupt will ever happen, to be
>>>             honest. The main reasons why I'm using a sleep is because
>>>             I didn't want a hog onto the CPU in a full thread usage
>>>             busy wait and because I didn't find any way of doing a
>>>             thread sleep in order to wake up later when the buffer
>>>             managed by native code has more data.
>>>             The Non-blocking part is managed by the buffer the OS
>>>             keeps filling most if not all the time. That buffer is the
>>>             field
>>>
>>>             ByteBuffer readBuffer
>>>             That's the gaining part against the plain old Buffered
>>>             classes.
>>>
>>>
>>>             Did that make sense to you? Feel free to ask anything else
>>>             you need.
>>>
>>>             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>
>>>                 I've skimmed through the code and I'm not sure I can
>>>                 see any asynchronicity
>>>                 (you were pointing at the lack of it in BufferedReader).
>>>                 And the mechanics of this is very puzzling to me, to
>>>                 be honest:
>>>                      void blockingFill(boolean forced) throws
>>>                 IOException {
>>>                          fill(forced);
>>>                          while (readPos == writePos) {
>>>                              try {
>>>                                  Thread.sleep(100);
>>>                              } catch (InterruptedException e) {
>>>                                  // An interrupt may mean more data is
>>>                 available
>>>                              }
>>>                              fill(forced);
>>>                          }
>>>                      }
>>>                 I thought you were suggesting that we should utilize
>>>                 the tools which OS provides
>>>                 more efficiently. Instead we have something that looks
>>>                 very similarly to a
>>>                 "busy loop" and... also who and when is supposed to
>>>                 interrupt Thread.sleep()?
>>>                 Sorry, I'm not following. Could you please explain how
>>>                 this is supposed to work?
>>>
>>>                     On 24 Oct 2016, at 15:59, Brunoais
>>>                     <brunoaiss at gmail.com>
>>>                       wrote:
>>>                     Attached and sending!
>>>                     On 24/10/2016 13:48, Pavel Rappo wrote:
>>>
>>>                         Could you please send a new email on this list
>>>                         with the source attached as a
>>>                         text file?
>>>
>>>                             On 23 Oct 2016, at 19:14, Brunoais
>>>                             <brunoaiss at gmail.com>
>>>                               wrote:
>>>                             Here's my poc/prototype:
>>>
>>>                             http://pastebin.com/WRpYWDJF
>>>
>>>                             I've implemented the bare minimum of the
>>>                             class that follows the same contract of
>>>                             BufferedReader while signaling all issues
>>>                             I think it may have or has in comments.
>>>                             I also wrote some javadoc to help guiding
>>>                             through the class.
>>>                             I could have used more fields from
>>>                             BufferedReader but the names were so
>>>                             minimalistic that were confusing me. I
>>>                             intent to change them before sending this
>>>                             to openJDK.
>>>                             One of the major problems this has is long
>>>                             overflowing. It is major because it is
>>>                             hidden, it will be extremely rare and it
>>>                             takes a really long time to reproduce.
>>>                             There are different ways of dealing with
>>>                             it. From just documenting to actually
>>>                             making code that works with it.
>>>                             I built a simple test code for it to have
>>>                             some ideas about performance and correctness.
>>>
>>>                             http://pastebin.com/eh6LFgwT
>>>
>>>                             This doesn't do a through test if it is
>>>                             actually working correctly but I see no
>>>                             reason for it not working correctly after
>>>                             fixing the 2 bugs that test found.
>>>                             I'll also leave here some conclusions
>>>                             about speed and resource consumption I found.
>>>                             I made tests with default buffer sizes,
>>>                             5000B 15_000B and 500_000B. I noticed
>>>                             that, with my hardware, with the 1 530 000
>>>                             000B file, I was getting around:
>>>                             In all buffers and fake work: 10~15s speed
>>>                             improvement ( from 90% HDD speed to 100%
>>>                             HDD speed)
>>>                             In all buffers and no fake work: 1~2s
>>>                             speed improvement ( from 90% HDD speed to
>>>                             100% HDD speed)
>>>                             Changing the buffer size was giving
>>>                             different reading speeds but both were
>>>                             quite equal in how much they would change
>>>                             when changing the buffer size.
>>>                             Finally, I could always confirm that I/O
>>>                             was always the slowest thing while this
>>>                             code was running.
>>>                             For the ones wondering about the file
>>>                             size; it is both to avoid OS cache and to
>>>                             make the reading at the main use-case
>>>                             these objects are for (large streams of
>>>                             bytes).
>>>                             @Pavel, are you open for discussion now
>>>                             ;)? Need anything else?
>>>                             On 21/10/2016 19:21, Pavel Rappo wrote:
>>>
>>>                                 Just to append to my previous email.
>>>                                 BufferedReader wraps any Reader out
>>> there.
>>>                                 Not specifically FileReader. While
>>>                                 you're talking about the case of
>>> effective
>>>                                 reading from a file.
>>>                                 I guess there's one existing
>>>                                 possibility to provide exactly what
>>>                                 you need (as I
>>>                                 understand it) under this method:
>>>                                 /**
>>>                                   * Opens a file for reading,
>>>                                 returning a {@code BufferedReader} to
>>>                                 read text
>>>                                   * from the file in an efficient
>>>                                 manner...
>>>                                     ...
>>>                                   */
>>>                                 java.nio.file.Files#newBuffere
>>> dReader(java.nio.file.Path)
>>>                                 It can return _anything_ as long as it
>>>                                 is a BufferedReader. We can do it, but it
>>>                                 needs to be investigated not only for
>>>                                 your favorite OS but for other OSes as
>>>                                 well. Feel free to prototype this and
>>>                                 we can discuss it on the list later.
>>>                                 Thanks,
>>>                                 -Pavel
>>>
>>>                                     On 21 Oct 2016, at 18:56, Brunoais
>>>                                     <brunoaiss at gmail.com>
>>>                                       wrote:
>>>                                     Pavel is right.
>>>                                     In reality, I was expecting such
>>>                                     BufferedReader to use only a
>>>                                     single buffer and have that Buffer
>>>                                     being filled asynchronously, not
>>>                                     in a different Thread.
>>>                                     Additionally, I don't have the
>>>                                     intention of having a larger
>>>                                     buffer than before unless stated
>>>                                     through the API (the constructor).
>>>                                     In my idea, internally, it is
>>>                                     supposed to use
>>>                                     java.nio.channels.Asynchronous
>>> FileChannel
>>>                                     or equivalent.
>>>                                     It does not prevent having two
>>>                                     buffers and I do not intent to
>>>                                     change BufferedReader itself. I'd
>>>                                     do an BufferedAsyncReader of sorts
>>>                                     (any name suggestion is welcome as
>>>                                     I'm an awful namer).
>>>                                     On 21/10/2016 18:38, Roger Riggs
>>>                                     wrote:
>>>
>>>                                         Hi Pavel,
>>>                                         I think Brunoais asking for a
>>>                                         double buffering scheme in
>>>                                         which the implementation of
>>>                                         BufferReader fills (a second
>>>                                         buffer) in parallel with the
>>>                                         application reading from the
>>>                                         1st buffer
>>>                                         and managing the swaps and
>>>                                         async reads transparently.
>>>                                         It would not change the API
>>>                                         but would change the
>>>                                         interactions between the
>>>                                         buffered reader
>>>                                         and the underlying stream.  It
>>>                                         would also increase memory
>>>                                         requirements and processing
>>>                                         by introducing or using a
>>>                                         separate thread and the
>>>                                         necessary synchronization.
>>>                                         Though I think the formal
>>>                                         interface semantics could be
>>>                                         maintained, I have doubts
>>>                                         about compatibility and its
>>>                                         unintended consequences on
>>>                                         existing subclasses,
>>>                                         applications and libraries.
>>>                                         $.02, Roger
>>>                                         On 10/21/16 1:22 PM, Pavel
>>>                                         Rappo wrote:
>>>
>>>                                             Off the top of my head, I
>>>                                             would say it's not
>>>                                             possible to change the
>>>                                             design of an
>>>                                             _extensible_ type that has
>>>                                             been out there for 20 or
>>>                                             so years. All these I/O
>>>                                             streams from java.io
>>>                                             <http://java.io> were
>>>                                             designed for simple
>>>                                             synchronous use case.
>>>                                             It's not that their design
>>>                                             is flawed in some way,
>>>                                             it's that they doesn't seem
>>> to
>>>                                             suit your needs. Have you
>>>                                             considered using
>>>
>>> java.nio.channels.AsynchronousFileChannel
>>>                                             in your applications?
>>>                                             -Pavel
>>>
>>>                                                 On 21 Oct 2016, at
>>>                                                 17:08, Brunoais
>>>                                                 <brunoaiss at gmail.com>
>>>                                                   wrote:
>>>                                                 Any feedback on this?
>>>                                                 I'm really interested
>>>                                                 in implementing such
>>>
>>> BufferedReader/BufferedStreamReader
>>>                                                 to allow speeding up
>>>                                                 my applications
>>>                                                 without having to
>>>                                                 think in an
>>>                                                 asynchronous way or
>>>                                                 multi-threading while
>>>                                                 programming with it.
>>>                                                 That's why I'm asking
>>>                                                 this here.
>>>                                                 On 13/10/2016 14:45,
>>>                                                 Brunoais wrote:
>>>
>>>                                                     Hi,
>>>                                                     I looked at
>>>                                                     BufferedReader
>>>                                                     source code for
>>>                                                     java 9 long with
>>>                                                     the source code of
>>>                                                     the
>>>                                                     channels/streams
>>>                                                     used. I noticed
>>>                                                     that, like in java
>>>                                                     7, BufferedReader
>>>                                                     does not use an
>>>                                                     Async API to load
>>>                                                     data from files,
>>>                                                     instead, the data
>>>                                                     loading is all
>>>                                                     done synchronously
>>>                                                     even when the OS
>>>                                                     allows requesting
>>>                                                     a file to be read
>>>                                                     and getting a
>>>                                                     warning later when
>>>                                                     the file is
>>>                                                     effectively read.
>>>                                                     Why Is
>>>                                                     BufferedReader not
>>>                                                     async while
>>>                                                     providing a sync API?
>>>
>>>                     <BufferedNonBlockStream.java><Tests.java>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Sent from my phone
>>>
>>
>>
>
> --
> Sent from my phone
>
>
>

From brunoaiss at gmail.com  Thu Oct 27 13:06:38 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 14:06:38 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
Message-ID: <a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>

Thanks for the heads up.

I'll try that later. These tests are still useful then. Meanwhile, I'll 
end up also checking how FileChannel queries the OS on windows. I'm 
getting 100% HDD reads... Could it be that the OS reads the file ahead 
on its own?... Anyway, I'll look into it. Thanks for the heads up.


On 27/10/2016 13:53, Vitaly Davidovich wrote:
>
>
> On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
>
>     Oh... I see. In that case, it means something is terribly wrong.
>     It can be my initial tests, though.
>
>     I'm testing on both linux and windows and I'm getting performance
>     gains from using the FileChannel compared to using
>     FileInputStream... The tests also make sense based on my
>     predictions O_O...
>
> FileInputStream requires copying native buffers holding the read data 
> to the java byte[].  If you're using direct ByteBuffer for 
> FileChannel, that whole memcpy is skipped. Try comparing FileChannel 
> with HeapByteBuffer instead.
>
>
>     On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>
>>
>>     On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>     <mailto:brunoaiss at gmail.com>> wrote:
>>
>>         Did you read the C code?
>>
>>     I looked at the Linux code in the JDK.
>>
>>         Have you got any idea how many functions Windows or Linux
>>         (nearly all flavors) have for the read operation towards a file?
>>
>>     I do.
>>
>>
>>         I have already done that homework myself. I may not have read
>>         JVM's source code but I know well that there's functions on
>>         both Windows and Linux that provide such interface I
>>         mentioned although they require a slightly different
>>         treatment (and different constants).
>>
>>     You should read the JDK (native) source code instead of
>>     guessing/assuming.  On Linux, it doesn't use aio facilities for
>>     files.  The kernel io scheduler may issue readahead behind the
>>     scenes, but there's no nonblocking file io that's at the heart of
>>     your premise.
>>
>>
>>
>>         On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>
>>
>>
>>             On Wednesday, October 26, 2016, Brunoais
>>             <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>> wrote:
>>
>>                 It is actually based on the premise that:
>>
>>                 1. The first call to
>>             ReadableByteChannel.read(ByteBuffer) sets the OS
>>                    buffer size to fill in as the same size as ByteBuffer.
>>
>>             Why do you say that? AFAICT, it issues a read syscall and
>>             that will block if the data isn't in page cache.
>>
>>                 2. The consecutive calls to
>>             ReadableByteChannel.read(ByteBuffer)
>>                 orders
>>                    the JVM to order the OS to execute memcpy() to
>>             copy from its memory
>>                    to the shared memory created at ByteBuffer
>>             instantiation (in
>>                 java 8)
>>                    using Unsafe and then for the JVM to update the
>>             ByteBuffer fields.
>>
>>             I think subsequent reads just invoke the same read
>>             syscall, passing the current file offset maintained by
>>             the file channel instance.
>>
>>                 3. The call will not block waiting for I/O and it
>>             won't take longer
>>                    than the JNI interface if no new data exists.
>>             However, it will
>>                 block
>>                    waiting for the OS to execute memcpy() to the
>>             shared memory.
>>
>>             So why do you think it won't block?
>>
>>
>>                 Is my premise wrong?
>>
>>                 If I read correctly, if I don't use a DirectBuffer,
>>             there would be
>>                 even another intermediate buffer to copy data to
>>             before giving it
>>                 to the "user" which would be useless.
>>
>>             If you use a HeapByteBuffer, then there's an extra copy
>>             from the native buffer to the Java buffer.
>>
>>
>>
>>                 On 26/10/2016 11:57, Pavel Rappo wrote:
>>
>>                     I believe I see where you coming from. Please
>>             correct me if
>>                     I'm wrong.
>>
>>                     Your implementation is based on the premise that
>>             a call to
>>                     ReadableByteChannel.read()
>>                     _initiates_ the operation and returns
>>             immediately. The OS then
>>                     continues to fill
>>                     the buffer while there's a free space in the
>>             buffer and the
>>                     channel hasn't encountered EOF.
>>
>>                     Is that right?
>>
>>                         On 25 Oct 2016, at 22:16, Brunoais
>>             <brunoaiss at gmail.com>
>>                         wrote:
>>
>>                         Thank you for your time. I'll try to explain
>>             it. I hope I
>>                         can clear it up.
>>                         First of it, I made a meaning mistake between
>>             asynchronous
>>                         and non-blocking. This implementation uses a
>>             non-blocking
>>                         algorithm internally while providing a
>>             blocking-like
>>                         algorithm on the surface. It is
>>             single-threaded and not
>>                         multi-threaded where one thread fetches data
>>             and blocks
>>                         waiting and the other accumulates it and
>>             provides to
>>                         whichever wants it.
>>
>>                         Second of it, I had made a mistake of going after
>>                         BufferedReader instead of going after
>>             BufferedInputStream.
>>                         If you want me to go after BufferedReader
>>             it's ok but I
>>                         only thought that going after
>>             BufferedInputStream would be
>>                         more generically useful than
>>             BufferedReaderwhen I started
>>                         the poc.
>>
>>                         On to my code:
>>                         Short answers:
>>                                 ? The sleep(int) exists because I
>>             don't know how
>>                         to wait until more data exists in the buffer
>>             which is part
>>                         of read()'s contract.
>>                                 ? The ByteBuffer gives a buffer that
>>             is filled by
>>                         the OS (what I believe Channels do) instead
>>             of getting
>>                         data only         by demand (what I believe
>>             Streams do).
>>                         Full answers:
>>                         The blockingFill(boolean) method is a method
>>             for a busy
>>                         wait for a fill which is used exclusively by
>>             the read()
>>                         method. All other methods use the version
>>             that does not
>>                         sleep (fill(boolean)).
>>                         blockingFill(boolean)'s existance like that
>>             is only
>>                         because the read() method must not return
>>             unless either:
>>
>>                                 ? The stream ended.
>>                                 ? The next byte is ready for reading.
>>                         Additionally, statistically, that while loop
>>             will rarely
>>                         evaluate to true as reads are in chunks so
>>             readPos will be
>>                         behind writePos most of the time.
>>                         I have no idea if an interrupt will ever
>>             happen, to be
>>                         honest. The main reasons why I'm using a
>>             sleep is because
>>                         I didn't want a hog onto the CPU in a full
>>             thread usage
>>                         busy wait and because I didn't find any way
>>             of doing a
>>                         thread sleep in order to wake up later when
>>             the buffer
>>                         managed by native code has more data.
>>                         The Non-blocking part is managed by the
>>             buffer the OS
>>                         keeps filling most if not all the time. That
>>             buffer is the
>>                         field
>>
>>                         ByteBuffer readBuffer
>>                         That's the gaining part against the plain old
>>             Buffered
>>                         classes.
>>
>>
>>                         Did that make sense to you? Feel free to ask
>>             anything else
>>                         you need.
>>
>>                         On 25/10/2016 20:52, Pavel Rappo wrote:
>>
>>                             I've skimmed through the code and I'm not
>>             sure I can
>>                             see any asynchronicity
>>                             (you were pointing at the lack of it in
>>             BufferedReader).
>>                             And the mechanics of this is very
>>             puzzling to me, to
>>                             be honest:
>>                                  void blockingFill(boolean forced) throws
>>                             IOException {
>>                                      fill(forced);
>>                                      while (readPos == writePos) {
>>                                          try {
>>              Thread.sleep(100);
>>                                          } catch
>>             (InterruptedException e) {
>>                                              // An interrupt may mean
>>             more data is
>>                             available
>>                                          }
>>                                          fill(forced);
>>                                      }
>>                                  }
>>                             I thought you were suggesting that we
>>             should utilize
>>                             the tools which OS provides
>>                             more efficiently. Instead we have
>>             something that looks
>>                             very similarly to a
>>                             "busy loop" and... also who and when is
>>             supposed to
>>                             interrupt Thread.sleep()?
>>                             Sorry, I'm not following. Could you
>>             please explain how
>>                             this is supposed to work?
>>
>>                                 On 24 Oct 2016, at 15:59, Brunoais
>>                                 <brunoaiss at gmail.com>
>>                                   wrote:
>>                                 Attached and sending!
>>                                 On 24/10/2016 13:48, Pavel Rappo wrote:
>>
>>                                     Could you please send a new email
>>             on this list
>>                                     with the source attached as a
>>                                     text file?
>>
>>                                         On 23 Oct 2016, at 19:14,
>>             Brunoais
>>                                         <brunoaiss at gmail.com>
>>                                           wrote:
>>                                         Here's my poc/prototype:
>>
>>             http://pastebin.com/WRpYWDJF
>>
>>                                         I've implemented the bare
>>             minimum of the
>>                                         class that follows the same
>>             contract of
>>                                         BufferedReader while
>>             signaling all issues
>>                                         I think it may have or has in
>>             comments.
>>                                         I also wrote some javadoc to
>>             help guiding
>>                                         through the class.
>>                                         I could have used more fields
>>             from
>>                                         BufferedReader but the names
>>             were so
>>                                         minimalistic that were
>>             confusing me. I
>>                                         intent to change them before
>>             sending this
>>                                         to openJDK.
>>                                         One of the major problems
>>             this has is long
>>                                         overflowing. It is major
>>             because it is
>>                                         hidden, it will be extremely
>>             rare and it
>>                                         takes a really long time to
>>             reproduce.
>>                                         There are different ways of
>>             dealing with
>>                                         it. From just documenting to
>>             actually
>>                                         making code that works with it.
>>                                         I built a simple test code
>>             for it to have
>>                                         some ideas about performance
>>             and correctness.
>>
>>             http://pastebin.com/eh6LFgwT
>>
>>                                         This doesn't do a through
>>             test if it is
>>                                         actually working correctly
>>             but I see no
>>                                         reason for it not working
>>             correctly after
>>                                         fixing the 2 bugs that test
>>             found.
>>                                         I'll also leave here some
>>             conclusions
>>                                         about speed and resource
>>             consumption I found.
>>                                         I made tests with default
>>             buffer sizes,
>>                                         5000B 15_000B and 500_000B. I
>>             noticed
>>                                         that, with my hardware, with
>>             the 1 530 000
>>                                         000B file, I was getting around:
>>                                         In all buffers and fake work:
>>             10~15s speed
>>                                         improvement ( from 90% HDD
>>             speed to 100%
>>                                         HDD speed)
>>                                         In all buffers and no fake
>>             work: 1~2s
>>                                         speed improvement ( from 90%
>>             HDD speed to
>>                                         100% HDD speed)
>>                                         Changing the buffer size was
>>             giving
>>                                         different reading speeds but
>>             both were
>>                                         quite equal in how much they
>>             would change
>>                                         when changing the buffer size.
>>                                         Finally, I could always
>>             confirm that I/O
>>                                         was always the slowest thing
>>             while this
>>                                         code was running.
>>                                         For the ones wondering about
>>             the file
>>                                         size; it is both to avoid OS
>>             cache and to
>>                                         make the reading at the main
>>             use-case
>>                                         these objects are for (large
>>             streams of
>>                                         bytes).
>>                                         @Pavel, are you open for
>>             discussion now
>>                                         ;)? Need anything else?
>>                                         On 21/10/2016 19:21, Pavel
>>             Rappo wrote:
>>
>>                                             Just to append to my
>>             previous email.
>>                                             BufferedReader wraps any
>>             Reader out there.
>>                                             Not specifically
>>             FileReader. While
>>                                             you're talking about the
>>             case of effective
>>                                             reading from a file.
>>                                             I guess there's one existing
>>                                             possibility to provide
>>             exactly what
>>                                             you need (as I
>>                                             understand it) under this
>>             method:
>>                                             /**
>>                                               * Opens a file for reading,
>>                                             returning a {@code
>>             BufferedReader} to
>>                                             read text
>>                                               * from the file in an
>>             efficient
>>                                             manner...
>>                                                 ...
>>                                               */
>>             java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>                                             It can return _anything_
>>             as long as it
>>                                             is a BufferedReader. We
>>             can do it, but it
>>                                             needs to be investigated
>>             not only for
>>                                             your favorite OS but for
>>             other OSes as
>>                                             well. Feel free to
>>             prototype this and
>>                                             we can discuss it on the
>>             list later.
>>                                             Thanks,
>>                                             -Pavel
>>
>>                                                 On 21 Oct 2016, at
>>             18:56, Brunoais
>>                                                 <brunoaiss at gmail.com>
>>                                                   wrote:
>>                                                 Pavel is right.
>>                                                 In reality, I was
>>             expecting such
>>             BufferedReader to use only a
>>                                                 single buffer and
>>             have that Buffer
>>                                                 being filled
>>             asynchronously, not
>>                                                 in a different Thread.
>>             Additionally, I don't have the
>>                                                 intention of having a
>>             larger
>>                                                 buffer than before
>>             unless stated
>>                                                 through the API (the
>>             constructor).
>>                                                 In my idea,
>>             internally, it is
>>                                                 supposed to use
>>             java.nio.channels.AsynchronousFileChannel
>>                                                 or equivalent.
>>                                                 It does not prevent
>>             having two
>>                                                 buffers and I do not
>>             intent to
>>                                                 change BufferedReader
>>             itself. I'd
>>                                                 do an
>>             BufferedAsyncReader of sorts
>>                                                 (any name suggestion
>>             is welcome as
>>                                                 I'm an awful namer).
>>                                                 On 21/10/2016 18:38,
>>             Roger Riggs
>>                                                 wrote:
>>
>>                                                     Hi Pavel,
>>                                                     I think Brunoais
>>             asking for a
>>                                                     double buffering
>>             scheme in
>>                                                     which the
>>             implementation of
>>             BufferReader fills (a second
>>             buffer) in parallel with the
>>             application reading from the
>>                                                     1st buffer
>>                                                     and managing the
>>             swaps and
>>                                                     async reads
>>             transparently.
>>                                                     It would not
>>             change the API
>>                                                     but would change the
>>             interactions between the
>>             buffered reader
>>                                                     and the
>>             underlying stream.  It
>>                                                     would also
>>             increase memory
>>             requirements and processing
>>                                                     by introducing or
>>             using a
>>             separate thread and the
>>             necessary synchronization.
>>                                                     Though I think
>>             the formal
>>             interface semantics could be
>>             maintained, I have doubts
>>                                                     about
>>             compatibility and its
>>             unintended consequences on
>>             existing subclasses,
>>             applications and libraries.
>>                                                     $.02, Roger
>>                                                     On 10/21/16 1:22
>>             PM, Pavel
>>                                                     Rappo wrote:
>>
>>             Off the top of my head, I
>>             would say it's not
>>             possible to change the
>>             design of an
>>             _extensible_ type that has
>>             been out there for 20 or
>>                                                         so years. All
>>             these I/O
>>             streams from java.io <http://java.io>
>>             <http://java.io> were
>>             designed for simple
>>             synchronous use case.
>>             It's not that their design
>>                                                         is flawed in
>>             some way,
>>             it's that they doesn't seem to
>>             suit your needs. Have you
>>             considered using
>>             java.nio.channels.AsynchronousFileChannel
>>                                                         in your
>>             applications?
>>             -Pavel
>>
>>               On 21 Oct 2016, at
>>               17:08, Brunoais
>>               <brunoaiss at gmail.com>
>>                 wrote:
>>               Any feedback on this?
>>               I'm really interested
>>               in implementing such
>>               BufferedReader/BufferedStreamReader
>>               to allow speeding up
>>               my applications
>>               without having to
>>               think in an
>>               asynchronous way or
>>               multi-threading while
>>               programming with it.
>>               That's why I'm asking
>>               this here.
>>               On 13/10/2016 14:45,
>>               Brunoais wrote:
>>
>>                   Hi,
>>                   I looked at
>>                   BufferedReader
>>                   source code for
>>                   java 9 long with
>>                   the source code of
>>                   the
>>                   channels/streams
>>                   used. I noticed
>>                   that, like in java
>>                   7, BufferedReader
>>                   does not use an
>>                   Async API to load
>>                   data from files,
>>                   instead, the data
>>                   loading is all
>>                   done synchronously
>>                   even when the OS
>>                   allows requesting
>>                   a file to be read
>>                   and getting a
>>                   warning later when
>>                   the file is
>>                   effectively read.
>>                   Why Is
>>                   BufferedReader not
>>                   async while
>>                   providing a sync API?
>>
>>             <BufferedNonBlockStream.java><Tests.java>
>>
>>
>>
>>
>>
>>             -- 
>>             Sent from my phone
>>
>>
>>
>>
>>     -- 
>>     Sent from my phone
>
>


From vitalyd at gmail.com  Thu Oct 27 13:11:12 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Thu, 27 Oct 2016 09:11:12 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
Message-ID: <CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>

I don't know about Windows specifically, but generally file systems across
major OS's will implement readahead in their IO scheduler when they detect
sequential scans.

On Linux, you can also strace your test to confirm which syscalls are
emitted (you should be seeing plain read()'s there, with FileInputStream
and FileChannel).

On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com> wrote:

> Thanks for the heads up.
>
> I'll try that later. These tests are still useful then. Meanwhile, I'll
> end up also checking how FileChannel queries the OS on windows. I'm getting
> 100% HDD reads... Could it be that the OS reads the file ahead on its
> own?... Anyway, I'll look into it. Thanks for the heads up.
>
> On 27/10/2016 13:53, Vitaly Davidovich wrote:
>
>
>
> On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com> wrote:
>
>> Oh... I see. In that case, it means something is terribly wrong. It can
>> be my initial tests, though.
>>
>> I'm testing on both linux and windows and I'm getting performance gains
>> from using the FileChannel compared to using FileInputStream... The tests
>> also make sense based on my predictions O_O...
>>
> FileInputStream requires copying native buffers holding the read data to
> the java byte[].  If you're using direct ByteBuffer for FileChannel, that
> whole memcpy is skipped.  Try comparing FileChannel with HeapByteBuffer
> instead.
>
>>
>> On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>
>>
>>
>> On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com> wrote:
>>
>>> Did you read the C code?
>>
>> I looked at the Linux code in the JDK.
>>
>>> Have you got any idea how many functions Windows or Linux (nearly all
>>> flavors) have for the read operation towards a file?
>>
>> I do.
>>
>>>
>>> I have already done that homework myself. I may not have read JVM's
>>> source code but I know well that there's functions on both Windows and
>>> Linux that provide such interface I mentioned although they require a
>>> slightly different treatment (and different constants).
>>
>> You should read the JDK (native) source code instead of
>> guessing/assuming.  On Linux, it doesn't use aio facilities for files.  The
>> kernel io scheduler may issue readahead behind the scenes, but there's no
>> nonblocking file io that's at the heart of your premise.
>>
>>>
>>>
>>> On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>
>>>>
>>>>
>>>> On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com <mailto:
>>>> brunoaiss at gmail.com>> wrote:
>>>>
>>>>     It is actually based on the premise that:
>>>>
>>>>     1. The first call to ReadableByteChannel.read(ByteBuffer) sets the
>>>> OS
>>>>        buffer size to fill in as the same size as ByteBuffer.
>>>>
>>>> Why do you say that? AFAICT, it issues a read syscall and that will
>>>> block if the data isn't in page cache.
>>>>
>>>>     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
>>>>     orders
>>>>        the JVM to order the OS to execute memcpy() to copy from its
>>>> memory
>>>>        to the shared memory created at ByteBuffer instantiation (in
>>>>     java 8)
>>>>        using Unsafe and then for the JVM to update the ByteBuffer
>>>> fields.
>>>>
>>>> I think subsequent reads just invoke the same read syscall, passing the
>>>> current file offset maintained by the file channel instance.
>>>>
>>>>     3. The call will not block waiting for I/O and it won't take longer
>>>>        than the JNI interface if no new data exists. However, it will
>>>>     block
>>>>        waiting for the OS to execute memcpy() to the shared memory.
>>>>
>>>> So why do you think it won't block?
>>>>
>>>>
>>>>     Is my premise wrong?
>>>>
>>>>     If I read correctly, if I don't use a DirectBuffer, there would be
>>>>     even another intermediate buffer to copy data to before giving it
>>>>     to the "user" which would be useless.
>>>>
>>>> If you use a HeapByteBuffer, then there's an extra copy from the native
>>>> buffer to the Java buffer.
>>>>
>>>>
>>>>
>>>>     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>
>>>>         I believe I see where you coming from. Please correct me if
>>>>         I'm wrong.
>>>>
>>>>         Your implementation is based on the premise that a call to
>>>>         ReadableByteChannel.read()
>>>>         _initiates_ the operation and returns immediately. The OS then
>>>>         continues to fill
>>>>         the buffer while there's a free space in the buffer and the
>>>>         channel hasn't encountered EOF.
>>>>
>>>>         Is that right?
>>>>
>>>>             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
>>>>             wrote:
>>>>
>>>>             Thank you for your time. I'll try to explain it. I hope I
>>>>             can clear it up.
>>>>             First of it, I made a meaning mistake between asynchronous
>>>>             and non-blocking. This implementation uses a non-blocking
>>>>             algorithm internally while providing a blocking-like
>>>>             algorithm on the surface. It is single-threaded and not
>>>>             multi-threaded where one thread fetches data and blocks
>>>>             waiting and the other accumulates it and provides to
>>>>             whichever wants it.
>>>>
>>>>             Second of it, I had made a mistake of going after
>>>>             BufferedReader instead of going after BufferedInputStream.
>>>>             If you want me to go after BufferedReader it's ok but I
>>>>             only thought that going after BufferedInputStream would be
>>>>             more generically useful than BufferedReaderwhen I started
>>>>             the poc.
>>>>
>>>>             On to my code:
>>>>             Short answers:
>>>>                     ? The sleep(int) exists because I don't know how
>>>>             to wait until more data exists in the buffer which is part
>>>>             of read()'s contract.
>>>>                     ? The ByteBuffer gives a buffer that is filled by
>>>>             the OS (what I believe Channels do) instead of getting
>>>>             data only         by demand (what I believe Streams do).
>>>>             Full answers:
>>>>             The blockingFill(boolean) method is a method for a busy
>>>>             wait for a fill which is used exclusively by the read()
>>>>             method. All other methods use the version that does not
>>>>             sleep (fill(boolean)).
>>>>             blockingFill(boolean)'s existance like that is only
>>>>             because the read() method must not return unless either:
>>>>
>>>>                     ? The stream ended.
>>>>                     ? The next byte is ready for reading.
>>>>             Additionally, statistically, that while loop will rarely
>>>>             evaluate to true as reads are in chunks so readPos will be
>>>>             behind writePos most of the time.
>>>>             I have no idea if an interrupt will ever happen, to be
>>>>             honest. The main reasons why I'm using a sleep is because
>>>>             I didn't want a hog onto the CPU in a full thread usage
>>>>             busy wait and because I didn't find any way of doing a
>>>>             thread sleep in order to wake up later when the buffer
>>>>             managed by native code has more data.
>>>>             The Non-blocking part is managed by the buffer the OS
>>>>             keeps filling most if not all the time. That buffer is the
>>>>             field
>>>>
>>>>             ByteBuffer readBuffer
>>>>             That's the gaining part against the plain old Buffered
>>>>             classes.
>>>>
>>>>
>>>>             Did that make sense to you? Feel free to ask anything else
>>>>             you need.
>>>>
>>>>             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>
>>>>                 I've skimmed through the code and I'm not sure I can
>>>>                 see any asynchronicity
>>>>                 (you were pointing at the lack of it in BufferedReader).
>>>>                 And the mechanics of this is very puzzling to me, to
>>>>                 be honest:
>>>>                      void blockingFill(boolean forced) throws
>>>>                 IOException {
>>>>                          fill(forced);
>>>>                          while (readPos == writePos) {
>>>>                              try {
>>>>                                  Thread.sleep(100);
>>>>                              } catch (InterruptedException e) {
>>>>                                  // An interrupt may mean more data is
>>>>                 available
>>>>                              }
>>>>                              fill(forced);
>>>>                          }
>>>>                      }
>>>>                 I thought you were suggesting that we should utilize
>>>>                 the tools which OS provides
>>>>                 more efficiently. Instead we have something that looks
>>>>                 very similarly to a
>>>>                 "busy loop" and... also who and when is supposed to
>>>>                 interrupt Thread.sleep()?
>>>>                 Sorry, I'm not following. Could you please explain how
>>>>                 this is supposed to work?
>>>>
>>>>                     On 24 Oct 2016, at 15:59, Brunoais
>>>>                     <brunoaiss at gmail.com>
>>>>                       wrote:
>>>>                     Attached and sending!
>>>>                     On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>
>>>>                         Could you please send a new email on this list
>>>>                         with the source attached as a
>>>>                         text file?
>>>>
>>>>                             On 23 Oct 2016, at 19:14, Brunoais
>>>>                             <brunoaiss at gmail.com>
>>>>                               wrote:
>>>>                             Here's my poc/prototype:
>>>>
>>>>                             http://pastebin.com/WRpYWDJF
>>>>
>>>>                             I've implemented the bare minimum of the
>>>>                             class that follows the same contract of
>>>>                             BufferedReader while signaling all issues
>>>>                             I think it may have or has in comments.
>>>>                             I also wrote some javadoc to help guiding
>>>>                             through the class.
>>>>                             I could have used more fields from
>>>>                             BufferedReader but the names were so
>>>>                             minimalistic that were confusing me. I
>>>>                             intent to change them before sending this
>>>>                             to openJDK.
>>>>                             One of the major problems this has is long
>>>>                             overflowing. It is major because it is
>>>>                             hidden, it will be extremely rare and it
>>>>                             takes a really long time to reproduce.
>>>>                             There are different ways of dealing with
>>>>                             it. From just documenting to actually
>>>>                             making code that works with it.
>>>>                             I built a simple test code for it to have
>>>>                             some ideas about performance and
>>>> correctness.
>>>>
>>>>                             http://pastebin.com/eh6LFgwT
>>>>
>>>>                             This doesn't do a through test if it is
>>>>                             actually working correctly but I see no
>>>>                             reason for it not working correctly after
>>>>                             fixing the 2 bugs that test found.
>>>>                             I'll also leave here some conclusions
>>>>                             about speed and resource consumption I
>>>> found.
>>>>                             I made tests with default buffer sizes,
>>>>                             5000B 15_000B and 500_000B. I noticed
>>>>                             that, with my hardware, with the 1 530 000
>>>>                             000B file, I was getting around:
>>>>                             In all buffers and fake work: 10~15s speed
>>>>                             improvement ( from 90% HDD speed to 100%
>>>>                             HDD speed)
>>>>                             In all buffers and no fake work: 1~2s
>>>>                             speed improvement ( from 90% HDD speed to
>>>>                             100% HDD speed)
>>>>                             Changing the buffer size was giving
>>>>                             different reading speeds but both were
>>>>                             quite equal in how much they would change
>>>>                             when changing the buffer size.
>>>>                             Finally, I could always confirm that I/O
>>>>                             was always the slowest thing while this
>>>>                             code was running.
>>>>                             For the ones wondering about the file
>>>>                             size; it is both to avoid OS cache and to
>>>>                             make the reading at the main use-case
>>>>                             these objects are for (large streams of
>>>>                             bytes).
>>>>                             @Pavel, are you open for discussion now
>>>>                             ;)? Need anything else?
>>>>                             On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>
>>>>                                 Just to append to my previous email.
>>>>                                 BufferedReader wraps any Reader out
>>>> there.
>>>>                                 Not specifically FileReader. While
>>>>                                 you're talking about the case of
>>>> effective
>>>>                                 reading from a file.
>>>>                                 I guess there's one existing
>>>>                                 possibility to provide exactly what
>>>>                                 you need (as I
>>>>                                 understand it) under this method:
>>>>                                 /**
>>>>                                   * Opens a file for reading,
>>>>                                 returning a {@code BufferedReader} to
>>>>                                 read text
>>>>                                   * from the file in an efficient
>>>>                                 manner...
>>>>                                     ...
>>>>                                   */
>>>>                                 java.nio.file.Files#newBuffere
>>>> dReader(java.nio.file.Path)
>>>>                                 It can return _anything_ as long as it
>>>>                                 is a BufferedReader. We can do it, but
>>>> it
>>>>                                 needs to be investigated not only for
>>>>                                 your favorite OS but for other OSes as
>>>>                                 well. Feel free to prototype this and
>>>>                                 we can discuss it on the list later.
>>>>                                 Thanks,
>>>>                                 -Pavel
>>>>
>>>>                                     On 21 Oct 2016, at 18:56, Brunoais
>>>>                                     <brunoaiss at gmail.com>
>>>>                                       wrote:
>>>>                                     Pavel is right.
>>>>                                     In reality, I was expecting such
>>>>                                     BufferedReader to use only a
>>>>                                     single buffer and have that Buffer
>>>>                                     being filled asynchronously, not
>>>>                                     in a different Thread.
>>>>                                     Additionally, I don't have the
>>>>                                     intention of having a larger
>>>>                                     buffer than before unless stated
>>>>                                     through the API (the constructor).
>>>>                                     In my idea, internally, it is
>>>>                                     supposed to use
>>>>                                     java.nio.channels.Asynchronous
>>>> FileChannel
>>>>                                     or equivalent.
>>>>                                     It does not prevent having two
>>>>                                     buffers and I do not intent to
>>>>                                     change BufferedReader itself. I'd
>>>>                                     do an BufferedAsyncReader of sorts
>>>>                                     (any name suggestion is welcome as
>>>>                                     I'm an awful namer).
>>>>                                     On 21/10/2016 18:38, Roger Riggs
>>>>                                     wrote:
>>>>
>>>>                                         Hi Pavel,
>>>>                                         I think Brunoais asking for a
>>>>                                         double buffering scheme in
>>>>                                         which the implementation of
>>>>                                         BufferReader fills (a second
>>>>                                         buffer) in parallel with the
>>>>                                         application reading from the
>>>>                                         1st buffer
>>>>                                         and managing the swaps and
>>>>                                         async reads transparently.
>>>>                                         It would not change the API
>>>>                                         but would change the
>>>>                                         interactions between the
>>>>                                         buffered reader
>>>>                                         and the underlying stream.  It
>>>>                                         would also increase memory
>>>>                                         requirements and processing
>>>>                                         by introducing or using a
>>>>                                         separate thread and the
>>>>                                         necessary synchronization.
>>>>                                         Though I think the formal
>>>>                                         interface semantics could be
>>>>                                         maintained, I have doubts
>>>>                                         about compatibility and its
>>>>                                         unintended consequences on
>>>>                                         existing subclasses,
>>>>                                         applications and libraries.
>>>>                                         $.02, Roger
>>>>                                         On 10/21/16 1:22 PM, Pavel
>>>>                                         Rappo wrote:
>>>>
>>>>                                             Off the top of my head, I
>>>>                                             would say it's not
>>>>                                             possible to change the
>>>>                                             design of an
>>>>                                             _extensible_ type that has
>>>>                                             been out there for 20 or
>>>>                                             so years. All these I/O
>>>>                                             streams from java.io
>>>>                                             <http://java.io> were
>>>>                                             designed for simple
>>>>                                             synchronous use case.
>>>>                                             It's not that their design
>>>>                                             is flawed in some way,
>>>>                                             it's that they doesn't seem
>>>> to
>>>>                                             suit your needs. Have you
>>>>                                             considered using
>>>>
>>>> java.nio.channels.AsynchronousFileChannel
>>>>                                             in your applications?
>>>>                                             -Pavel
>>>>
>>>>                                                 On 21 Oct 2016, at
>>>>                                                 17:08, Brunoais
>>>>                                                 <brunoaiss at gmail.com>
>>>>                                                   wrote:
>>>>                                                 Any feedback on this?
>>>>                                                 I'm really interested
>>>>                                                 in implementing such
>>>>
>>>> BufferedReader/BufferedStreamReader
>>>>                                                 to allow speeding up
>>>>                                                 my applications
>>>>                                                 without having to
>>>>                                                 think in an
>>>>                                                 asynchronous way or
>>>>                                                 multi-threading while
>>>>                                                 programming with it.
>>>>                                                 That's why I'm asking
>>>>                                                 this here.
>>>>                                                 On 13/10/2016 14:45,
>>>>                                                 Brunoais wrote:
>>>>
>>>>                                                     Hi,
>>>>                                                     I looked at
>>>>                                                     BufferedReader
>>>>                                                     source code for
>>>>                                                     java 9 long with
>>>>                                                     the source code of
>>>>                                                     the
>>>>                                                     channels/streams
>>>>                                                     used. I noticed
>>>>                                                     that, like in java
>>>>                                                     7, BufferedReader
>>>>                                                     does not use an
>>>>                                                     Async API to load
>>>>                                                     data from files,
>>>>                                                     instead, the data
>>>>                                                     loading is all
>>>>                                                     done synchronously
>>>>                                                     even when the OS
>>>>                                                     allows requesting
>>>>                                                     a file to be read
>>>>                                                     and getting a
>>>>                                                     warning later when
>>>>                                                     the file is
>>>>                                                     effectively read.
>>>>                                                     Why Is
>>>>                                                     BufferedReader not
>>>>                                                     async while
>>>>                                                     providing a sync
>>>> API?
>>>>
>>>>                     <BufferedNonBlockStream.java><Tests.java>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from my phone
>>>>
>>>
>>>
>>
>> --
>> Sent from my phone
>>
>>
>>
>
>

From peter.levart at gmail.com  Thu Oct 27 13:48:41 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 27 Oct 2016 15:48:41 +0200
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <aeac24a7-63c9-1640-f1a9-b120b91972fe@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
	<c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
	<aeac24a7-63c9-1640-f1a9-b120b91972fe@gmail.com>
Message-ID: <22b7c693-9c9c-0add-011a-6e6552fd4f3a@gmail.com>


On 10/27/2016 02:20 PM, Brunoais wrote:
>
> Hey.
>
> Any idea how to skip tests? When testing for BufferedInputStream, the 
> directBufferSize is not used so testing with different 
> directBufferSize makes no sense.
>
> I already tried "return 0" on the benchmarked test but jmh fills the 
> output with " (*interrupt*)" if I do that.
>

If all combinations of @Param(s) don't make sense then I usually 
collapse two or three of them into one "compound" @Param containing the 
sensible combinations of their components. The type of such @Param field 
is String and the content is a delimited multi-value tuple. the @Setup 
method then parses such tuple into components...

Regards, Peter


From peter.levart at gmail.com  Thu Oct 27 13:55:39 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 27 Oct 2016 15:55:39 +0200
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <22b7c693-9c9c-0add-011a-6e6552fd4f3a@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
	<c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
	<aeac24a7-63c9-1640-f1a9-b120b91972fe@gmail.com>
	<22b7c693-9c9c-0add-011a-6e6552fd4f3a@gmail.com>
Message-ID: <e1e4aa0d-825e-efca-585e-da1cc0740633@gmail.com>



On 10/27/2016 03:48 PM, Peter Levart wrote:
>
> On 10/27/2016 02:20 PM, Brunoais wrote:
>>
>> Hey.
>>
>> Any idea how to skip tests? When testing for BufferedInputStream, the 
>> directBufferSize is not used so testing with different 
>> directBufferSize makes no sense.
>>
>> I already tried "return 0" on the benchmarked test but jmh fills the 
>> output with " (*interrupt*)" if I do that.
>>
>
> If all combinations of @Param(s) don't make sense then I usually 
> collapse two or three of them into one "compound" @Param containing 
> the sensible combinations of their components. The type of such @Param 
> field is String and the content is a delimited multi-value tuple. the 
> @Setup method then parses such tuple into components...
>
> Regards, Peter
>

Another possibility is to invoke the test(s) with command-line overrides 
for values of particular parameters. The benchmark will then run just 
the combinations you specify on the command line (try: java -jar 
benchmarks.jar -help).

Peter

From Roger.Riggs at Oracle.com  Thu Oct 27 15:15:19 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 27 Oct 2016 11:15:19 -0400
Subject: RFR:JDK-8075205 java/net/URLClassLoader/closetest/CloseTest.java
	and GetResourceAsStream.java failed with existing dir
In-Reply-To: <7BCAB4F4-93BF-4B5C-8494-4395292E3F7A@oracle.com>
References: <90BEB814-FB5F-44F6-B691-247186DC8480@oracle.com>
	<019E6AAB-85A3-4D4E-A2A0-2DCBCF028EF3@oracle.com>
	<6AE698AC-5301-46A3-92D2-4C6B64A3A2FB@oracle.com>
	<7BCAB4F4-93BF-4B5C-8494-4395292E3F7A@oracle.com>
Message-ID: <8302eb3c-41a2-f48c-ed7b-b989e1c225eb@Oracle.com>

Hi,

Looks good.

Editorial improvements:

GetResourceAsStream.java:

  - line 44-52:  Add the new variables after the test description like 
CloseTest.java or change CloseTest
    to improve style consistency.

  - Line 54-57: fix indentation

  - consistently use /* ... */ or // comment conventions for methods.

Thanks, Roger





On 10/25/2016 10:50 PM, Srinivasan Raghavan wrote:
> Hi all
>
> Please help with one more reviewer comment.
>
> Regards
> Srinivasan Raghavan
>> On 19-Oct-2016, at 9:38 AM, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
>>
>> Thanks for the review.
>>> On 18-Oct-2016, at 4:16 PM, Chris Hegarty <chris.hegarty at oracle.com> wrote:
>>>
>>>
>>>> On 17 Oct 2016, at 09:51, Srinivasan Raghavan <srinivasan.raghavan at oracle.com> wrote:
>>>>
>>>> Hi all
>>>>
>>>> Please review the fix for the bug
>>>>
>>>> Bug :https://bugs.openjdk.java.net/browse/JDK-8075205
>>>>
>>>> The tests uses classes directory for the output files. This can result in the files being left over after the test is complete which can result in instability. The tests copies the files to be compiled form test src to test classes which can result in copy of permission and result in instability because the test has delete operations. The test fails randomly mostly in copy or delete operation. I propose the test to be refactored to make the use scratch directory as its output directory and eliminate shell by using testlibrary utils.
>>>>
>>>> fix : http://cr.openjdk.java.net/~sraghavan/8075205/webrev.00/
>>> This looks good to me. Thanks Srinivasan.
>>>
>>> -Chris.


From Roger.Riggs at Oracle.com  Thu Oct 27 15:44:57 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 27 Oct 2016 11:44:57 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
Message-ID: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>

Please review a test fix for a timeout on a busy system in 
Process.waitFor a destroyed process.

Webrev:
    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

Thanks, Roger


From mandy.chung at oracle.com  Thu Oct 27 16:53:03 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 27 Oct 2016 09:53:03 -0700
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
In-Reply-To: <58115662.6080005@oracle.com>
References: <58115662.6080005@oracle.com>
Message-ID: <AB2D46BC-39C7-4AAE-9C78-E5BAA9905925@oracle.com>

+1

Mandy

> On Oct 26, 2016, at 6:20 PM, Kumar Srinivasan <kumar.x.srinivasan at oracle.com> wrote:
> 
> Hello,
> 
> Please review enclosed fix for: https://bugs.openjdk.java.net/browse/JDK-8168010
> 
> Based on this discussion, http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html
> these options are slated for deprecation, only at the documentation level,
> and will continue to operate with no perceivable behavior change, in JDK9.
> 
> Thanks
> Kumar
> 
> diff --git a/src/java.base/share/classes/sun/launcher/resources/launcher.properties b/src/java.base/share/classes/sun/launcher/resources/launcher.properties
> --- a/src/java.base/share/classes/sun/launcher/resources/launcher.properties
> +++ b/src/java.base/share/classes/sun/launcher/resources/launcher.properties
> @@ -1,5 +1,5 @@
> #
> -# Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights reserved.
> +# Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights reserved.
> # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> #
> # This code is free software; you can redistribute it and/or modify it
> @@ -31,7 +31,7 @@
> \           (to execute the main class in a module)\n\
> where options include:\n
> 
> -java.launcher.opt.datamodel  =\    -d{0}\t  use a {0}-bit data model if available\n
> +java.launcher.opt.datamodel  =\    -d{0}\t  Deprecated, will be removed in a future release\n
> java.launcher.opt.vmselect   =\    {0}\t  to select the "{1}" VM\n
> java.launcher.opt.hotspot    =\    {0}\t  is a synonym for the "{1}" VM  [deprecated]\n
> 


From david.holmes at oracle.com  Thu Oct 27 16:57:54 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 02:57:54 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
Message-ID: <6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>

Hi Roger,

On 28/10/2016 1:44 AM, Roger Riggs wrote:
> Please review a test fix for a timeout on a busy system in
> Process.waitFor a destroyed process.

Won't that now cause the test to hang until timed-out by the harness?

David

> Webrev:
>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>
> Thanks, Roger
>

From mandy.chung at oracle.com  Thu Oct 27 17:02:25 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 27 Oct 2016 10:02:25 -0700
Subject: Review Request: JDK-8168205: Should not default class path to CWD
	if -cp is not specified but -m is specified
In-Reply-To: <80fdbd10-a88e-bf85-ffe9-3af907a2053e@oracle.com>
References: <0C45950A-566E-4673-91CE-99EC59848F64@oracle.com>
	<80fdbd10-a88e-bf85-ffe9-3af907a2053e@oracle.com>
Message-ID: <5CDB20F8-0CD9-428E-A757-DADB05F591B6@oracle.com>


> On Oct 27, 2016, at 2:44 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> The updated comment in ClassLoaders might be a bit clearer if you drop "if defined?.
> 

OK, dropped ?if defined?.

> A minor comment on the test is that it could use ProcessTools.executeTestJava to avoid needing JAVA_TOOL.

ProcessTools::executeTestJava launches the process inherited with CLASSPATH environment variable set.  I can use ProcessTools::createJavaProcessBuilder that calls JDKToolFinder::getJDKTool to find the launcher instead of having the test finding the launcher (no difference anyway).  I?m okay either way and updated the webrev:

  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.01/

thanks
Mandy

From Roger.Riggs at Oracle.com  Thu Oct 27 17:12:43 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 27 Oct 2016 13:12:43 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
Message-ID: <1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>

Hi David,

On 10/27/2016 12:57 PM, David Holmes wrote:
> Hi Roger,
>
> On 28/10/2016 1:44 AM, Roger Riggs wrote:
>> Please review a test fix for a timeout on a busy system in
>> Process.waitFor a destroyed process.
>
> Won't that now cause the test to hang until timed-out by the harness?
yes, but an in-app timeout is not much different than the harness timeout
so I didn't complicate the test. Under normal system loading it should 
be 1-4 seconds.

I contemplated increasing the timeout but given the issue is system loading
I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
and the harness timeout is 2min.

Roger

>
> David
>
>> Webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>
>> Thanks, Roger
>>


From brunoaiss at gmail.com  Thu Oct 27 17:53:30 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 18:53:30 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <e1e4aa0d-825e-efca-585e-da1cc0740633@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1db04f00-b148-7ffb-2cef-ec32897ad2c3@gmail.com>
	<22f379c5-a5be-4db4-910d-851e09db66b6@gmail.com>
	<E1935BBB-2503-4E91-A37B-C316DE188DE2@oracle.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<57A008FAA9E6E1E3.C109FCD4-E265-4B0D-964E-655FABCDF1B6@mail.outlook.com>
	<ca62c807-1b37-738d-ec4f-75112b98ca80@gmail.com>
	<9a4397ad-fd19-de89-35b7-22102d3c4016@gmail.com>
	<229511e3-6e8d-6bc4-16d3-f24b6a985068@gmail.com>
	<65c6712e-2d86-71be-fa7a-14f40514adad@gmail.com>
	<c780cce7-e2f6-db04-2ba8-71377b66a76e@gmail.com>
	<aeac24a7-63c9-1640-f1a9-b120b91972fe@gmail.com>
	<22b7c693-9c9c-0add-011a-6e6552fd4f3a@gmail.com>
	<e1e4aa0d-825e-efca-585e-da1cc0740633@gmail.com>
Message-ID: <174adfa8-2153-cf35-f518-d349693ffe6b@gmail.com>

Here's my current code for benchmarking. Does this seem correct to you?


On 27/10/2016 14:55, Peter Levart wrote:
>
>
> On 10/27/2016 03:48 PM, Peter Levart wrote:
>>
>> On 10/27/2016 02:20 PM, Brunoais wrote:
>>>
>>> Hey.
>>>
>>> Any idea how to skip tests? When testing for BufferedInputStream, 
>>> the directBufferSize is not used so testing with different 
>>> directBufferSize makes no sense.
>>>
>>> I already tried "return 0" on the benchmarked test but jmh fills the 
>>> output with " (*interrupt*)" if I do that.
>>>
>>
>> If all combinations of @Param(s) don't make sense then I usually 
>> collapse two or three of them into one "compound" @Param containing 
>> the sensible combinations of their components. The type of such 
>> @Param field is String and the content is a delimited multi-value 
>> tuple. the @Setup method then parses such tuple into components...
>>
>> Regards, Peter
>>
>
> Another possibility is to invoke the test(s) with command-line 
> overrides for values of particular parameters. The benchmark will then 
> run just the combinations you specify on the command line (try: java 
> -jar benchmarks.jar -help).
>
> Peter
>

-------------- next part --------------
package org.sample.BufferedNon_BlockIO;

import java.io.BufferedInputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.file.StandardOpenOption;
import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.annotations.Threads;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;

/**
 * A sample file reading benchmark
 */
@BenchmarkMode(Mode.SingleShotTime)
@Fork(value = 1)
@Warmup(iterations = 0, time = 2, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2)
@Threads(1)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class BufferedNonBlockStreamBenchmark_try1 {

	public static boolean isWindows = System.getProperty("os.name").contains("Windows");
	
    @Param({"1gig_file"})
    String file;

//    @Param({"4100", "16400", "131100", "1048600"})
    @Param({"4100", "131100", "1048600"})
    int javaBufSize;
    
    @Param({"100", "1000"})
    int readSize;

//    @Param({"0", "1000", "100000"})
    @Param({"0", "1000"})
    long cpuWork;

    // 												This MUST be smaller than javaBufSize
    //  											@Param({"4096", "16384", "131072", "1048576"})
    @Param({"BufferedInputStream", "BufferedNonBlockStream|4096", "BufferedNonBlockStream|131072", "BufferedNonBlockStream|1048576"})
    String implType;

    interface ReadOp {
        int read(byte[] buf) throws IOException;
    }

    private ReadOp read;
    private Closeable close;
    private byte[] buf;

    @Setup(Level.Iteration)
    public void setup() throws IOException, InterruptedException {

        clearCache();
        
        String[] type_readSize = implType.split("\\|");

        switch (type_readSize[0]) {
            case "BufferedInputStream":{
                BufferedInputStream in = new BufferedInputStream(
                    new FileInputStream(file), javaBufSize);
                read = in::read;
                close = in::close;
            }
                break;
            case "BufferedNonBlockStream": {
            	BufferedNonBlockStream in = new BufferedNonBlockStream(
        			FileChannel.open(new File(file).toPath()), javaBufSize, Integer.parseInt(type_readSize[1]));
                read = in::read;
                close = in::close;
            }
            break;
            default:
                throw new IllegalArgumentException(
                    "Invalid parameter 'implType': " + implType);
        }

        buf = new byte[readSize];
    }

    @TearDown(Level.Iteration)
    public void tearDown() throws IOException {
        close.close();
    }

    /**
     * 
     * For linux:
     * 
     * Compile the following C program into clear_cache executable:
     * <pre>{@code
     *
     * #include <stdio.h>
     * #include <string.h>
     * #include <errno.h>
     *
     * #define PROC_FILE "/proc/sys/vm/drop_caches"
     *
     * int main(char *argv[], int argc) {
     *   FILE *f;
     *   f = fopen(PROC_FILE, "w");
     *   if (f) {
     *     fprintf(f, "3\n");
     *     fclose(f);
     *     return 0;
     *   } else {
     *     fprintf(stderr, "Can't write to: %s: %s\n", PROC_FILE, strerror(errno));
     *     return 1;
     *   }
     * }
     *
     * }</pre>
     * ... and make it SUID root!
     * 
     * For windows:
     * Use the provided clear_cache.exe
     * (source code and original author:)
     * https://gist.github.com/bitshifter/c87aa396446bbebeab29
     * Then run this program with administrator privileges
     */
    private static void clearCache() throws IOException, InterruptedException {
        // spawn an OS command to clear the FS cache...
    	if(isWindows){
    		new ProcessBuilder("clear_cache.exe").start().waitFor();
    	} else {
    		new ProcessBuilder("clear_cache").start().waitFor();    		
    	}
    }

    @Benchmark
    public int testRead() throws IOException {
        int nread = 0;
        int n;
        while ((n = read.read(buf)) >= 0) {
            nread += n;
            Blackhole.consumeCPU(cpuWork);
        }
        return nread;
    }
}

From brunoaiss at gmail.com  Thu Oct 27 19:08:46 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Thu, 27 Oct 2016 20:08:46 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
Message-ID: <b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>

You are right. Even in windows it does not set the flags for async 
reads. It seems like it is windows itself that does the decision to 
buffer the contents based on its own heuristics.

But... Why? Why won't it be? Why is there no API for it? How am I 
getting 100% HDD use and faster times when I fake work to delay getting 
more data and I only have a fluctuating 60-90% (always going up and 
down) when I use an InputStream?
Is it related to how both classes cache and how frequently and how much 
each one asks for data?

I really would prefer not having to read the source code because it 
takes a real long time T.T.

I end up reinstating... And wondering...

Why doesn't java provide a single-threaded non-block API for file reads 
for all OS that support it? I simply cannot find that information no 
matter how much I search on google, bing, duck duck go... Can any of you 
point me to whomever knows?

On 27/10/2016 14:11, Vitaly Davidovich wrote:
> I don't know about Windows specifically, but generally file systems 
> across major OS's will implement readahead in their IO scheduler when 
> they detect sequential scans.
>
> On Linux, you can also strace your test to confirm which syscalls are 
> emitted (you should be seeing plain read()'s there, with 
> FileInputStream and FileChannel).
>
> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
>
>     Thanks for the heads up.
>
>     I'll try that later. These tests are still useful then. Meanwhile,
>     I'll end up also checking how FileChannel queries the OS on
>     windows. I'm getting 100% HDD reads... Could it be that the OS
>     reads the file ahead on its own?... Anyway, I'll look into it.
>     Thanks for the heads up.
>
>
>     On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>
>>
>>     On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>>     <mailto:brunoaiss at gmail.com>> wrote:
>>
>>         Oh... I see. In that case, it means something is terribly
>>         wrong. It can be my initial tests, though.
>>
>>         I'm testing on both linux and windows and I'm getting
>>         performance gains from using the FileChannel compared to
>>         using FileInputStream... The tests also make sense based on
>>         my predictions O_O...
>>
>>     FileInputStream requires copying native buffers holding the read
>>     data to the java byte[].  If you're using direct ByteBuffer for
>>     FileChannel, that whole memcpy is skipped.  Try comparing
>>     FileChannel with HeapByteBuffer instead.
>>
>>
>>         On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>
>>>
>>>         On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>>         <mailto:brunoaiss at gmail.com>> wrote:
>>>
>>>             Did you read the C code?
>>>
>>>         I looked at the Linux code in the JDK.
>>>
>>>             Have you got any idea how many functions Windows or
>>>             Linux (nearly all flavors) have for the read operation
>>>             towards a file?
>>>
>>>         I do.
>>>
>>>
>>>             I have already done that homework myself. I may not have
>>>             read JVM's source code but I know well that there's
>>>             functions on both Windows and Linux that provide such
>>>             interface I mentioned although they require a slightly
>>>             different treatment (and different constants).
>>>
>>>         You should read the JDK (native) source code instead of
>>>         guessing/assuming.  On Linux, it doesn't use aio facilities
>>>         for files.  The kernel io scheduler may issue readahead
>>>         behind the scenes, but there's no nonblocking file io that's
>>>         at the heart of your premise.
>>>
>>>
>>>
>>>             On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>
>>>
>>>
>>>                 On Wednesday, October 26, 2016, Brunoais
>>>                 <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>                 wrote:
>>>
>>>                     It is actually based on the premise that:
>>>
>>>                     1. The first call to
>>>                 ReadableByteChannel.read(ByteBuffer) sets the OS
>>>                        buffer size to fill in as the same size as
>>>                 ByteBuffer.
>>>
>>>                 Why do you say that? AFAICT, it issues a read
>>>                 syscall and that will block if the data isn't in
>>>                 page cache.
>>>
>>>                     2. The consecutive calls to
>>>                 ReadableByteChannel.read(ByteBuffer)
>>>                     orders
>>>                        the JVM to order the OS to execute memcpy()
>>>                 to copy from its memory
>>>                        to the shared memory created at ByteBuffer
>>>                 instantiation (in
>>>                     java 8)
>>>                        using Unsafe and then for the JVM to update
>>>                 the ByteBuffer fields.
>>>
>>>                 I think subsequent reads just invoke the same read
>>>                 syscall, passing the current file offset maintained
>>>                 by the file channel instance.
>>>
>>>                     3. The call will not block waiting for I/O and
>>>                 it won't take longer
>>>                        than the JNI interface if no new data exists.
>>>                 However, it will
>>>                     block
>>>                        waiting for the OS to execute memcpy() to the
>>>                 shared memory.
>>>
>>>                 So why do you think it won't block?
>>>
>>>
>>>                     Is my premise wrong?
>>>
>>>                     If I read correctly, if I don't use a
>>>                 DirectBuffer, there would be
>>>                     even another intermediate buffer to copy data to
>>>                 before giving it
>>>                     to the "user" which would be useless.
>>>
>>>                 If you use a HeapByteBuffer, then there's an extra
>>>                 copy from the native buffer to the Java buffer.
>>>
>>>
>>>
>>>                     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>
>>>                         I believe I see where you coming from.
>>>                 Please correct me if
>>>                         I'm wrong.
>>>
>>>                         Your implementation is based on the premise
>>>                 that a call to
>>>                 ReadableByteChannel.read()
>>>                         _initiates_ the operation and returns
>>>                 immediately. The OS then
>>>                         continues to fill
>>>                         the buffer while there's a free space in the
>>>                 buffer and the
>>>                         channel hasn't encountered EOF.
>>>
>>>                         Is that right?
>>>
>>>                             On 25 Oct 2016, at 22:16, Brunoais
>>>                 <brunoaiss at gmail.com>
>>>                             wrote:
>>>
>>>                             Thank you for your time. I'll try to
>>>                 explain it. I hope I
>>>                             can clear it up.
>>>                             First of it, I made a meaning mistake
>>>                 between asynchronous
>>>                             and non-blocking. This implementation
>>>                 uses a non-blocking
>>>                             algorithm internally while providing a
>>>                 blocking-like
>>>                             algorithm on the surface. It is
>>>                 single-threaded and not
>>>                             multi-threaded where one thread fetches
>>>                 data and blocks
>>>                             waiting and the other accumulates it and
>>>                 provides to
>>>                             whichever wants it.
>>>
>>>                             Second of it, I had made a mistake of
>>>                 going after
>>>                             BufferedReader instead of going after
>>>                 BufferedInputStream.
>>>                             If you want me to go after
>>>                 BufferedReader it's ok but I
>>>                             only thought that going after
>>>                 BufferedInputStream would be
>>>                             more generically useful than
>>>                 BufferedReaderwhen I started
>>>                             the poc.
>>>
>>>                             On to my code:
>>>                             Short answers:
>>>                                     ? The sleep(int) exists because
>>>                 I don't know how
>>>                             to wait until more data exists in the
>>>                 buffer which is part
>>>                             of read()'s contract.
>>>                                     ? The ByteBuffer gives a buffer
>>>                 that is filled by
>>>                             the OS (what I believe Channels do)
>>>                 instead of getting
>>>                             data only         by demand (what I
>>>                 believe Streams do).
>>>                             Full answers:
>>>                             The blockingFill(boolean) method is a
>>>                 method for a busy
>>>                             wait for a fill which is used
>>>                 exclusively by the read()
>>>                             method. All other methods use the
>>>                 version that does not
>>>                             sleep (fill(boolean)).
>>>                 blockingFill(boolean)'s existance like that is only
>>>                             because the read() method must not
>>>                 return unless either:
>>>
>>>                                     ? The stream ended.
>>>                                     ? The next byte is ready for
>>>                 reading.
>>>                             Additionally, statistically, that while
>>>                 loop will rarely
>>>                             evaluate to true as reads are in chunks
>>>                 so readPos will be
>>>                             behind writePos most of the time.
>>>                             I have no idea if an interrupt will ever
>>>                 happen, to be
>>>                             honest. The main reasons why I'm using a
>>>                 sleep is because
>>>                             I didn't want a hog onto the CPU in a
>>>                 full thread usage
>>>                             busy wait and because I didn't find any
>>>                 way of doing a
>>>                             thread sleep in order to wake up later
>>>                 when the buffer
>>>                             managed by native code has more data.
>>>                             The Non-blocking part is managed by the
>>>                 buffer the OS
>>>                             keeps filling most if not all the time.
>>>                 That buffer is the
>>>                             field
>>>
>>>                             ByteBuffer readBuffer
>>>                             That's the gaining part against the
>>>                 plain old Buffered
>>>                             classes.
>>>
>>>
>>>                             Did that make sense to you? Feel free to
>>>                 ask anything else
>>>                             you need.
>>>
>>>                             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>
>>>                                 I've skimmed through the code and
>>>                 I'm not sure I can
>>>                                 see any asynchronicity
>>>                                 (you were pointing at the lack of it
>>>                 in BufferedReader).
>>>                                 And the mechanics of this is very
>>>                 puzzling to me, to
>>>                                 be honest:
>>>                                      void blockingFill(boolean
>>>                 forced) throws
>>>                                 IOException {
>>>                  fill(forced);
>>>                                          while (readPos == writePos) {
>>>                                              try {
>>>                  Thread.sleep(100);
>>>                                              } catch
>>>                 (InterruptedException e) {
>>>                  // An interrupt may mean more data is
>>>                                 available
>>>                                              }
>>>                  fill(forced);
>>>                                          }
>>>                                      }
>>>                                 I thought you were suggesting that
>>>                 we should utilize
>>>                                 the tools which OS provides
>>>                                 more efficiently. Instead we have
>>>                 something that looks
>>>                                 very similarly to a
>>>                                 "busy loop" and... also who and when
>>>                 is supposed to
>>>                                 interrupt Thread.sleep()?
>>>                                 Sorry, I'm not following. Could you
>>>                 please explain how
>>>                                 this is supposed to work?
>>>
>>>                                     On 24 Oct 2016, at 15:59, Brunoais
>>>                                     <brunoaiss at gmail.com>
>>>                                       wrote:
>>>                                     Attached and sending!
>>>                                     On 24/10/2016 13:48, Pavel Rappo
>>>                 wrote:
>>>
>>>                                         Could you please send a new
>>>                 email on this list
>>>                                         with the source attached as a
>>>                                         text file?
>>>
>>>                                             On 23 Oct 2016, at
>>>                 19:14, Brunoais
>>>                                             <brunoaiss at gmail.com>
>>>                 wrote:
>>>                 Here's my poc/prototype:
>>>
>>>                 http://pastebin.com/WRpYWDJF
>>>
>>>                                             I've implemented the
>>>                 bare minimum of the
>>>                 class that follows the same contract of
>>>                 BufferedReader while signaling all issues
>>>                                             I think it may have or
>>>                 has in comments.
>>>                                             I also wrote some
>>>                 javadoc to help guiding
>>>                 through the class.
>>>                                             I could have used more
>>>                 fields from
>>>                 BufferedReader but the names were so
>>>                 minimalistic that were confusing me. I
>>>                 intent to change them before sending this
>>>                                             to openJDK.
>>>                                             One of the major
>>>                 problems this has is long
>>>                 overflowing. It is major because it is
>>>                 hidden, it will be extremely rare and it
>>>                 takes a really long time to reproduce.
>>>                 There are different ways of dealing with
>>>                                             it. From just
>>>                 documenting to actually
>>>                 making code that works with it.
>>>                                             I built a simple test
>>>                 code for it to have
>>>                                             some ideas about
>>>                 performance and correctness.
>>>
>>>                 http://pastebin.com/eh6LFgwT
>>>
>>>                                             This doesn't do a
>>>                 through test if it is
>>>                 actually working correctly but I see no
>>>                 reason for it not working correctly after
>>>                 fixing the 2 bugs that test found.
>>>                                             I'll also leave here
>>>                 some conclusions
>>>                 about speed and resource consumption I found.
>>>                                             I made tests with
>>>                 default buffer sizes,
>>>                 5000B 15_000B and 500_000B. I noticed
>>>                 that, with my hardware, with the 1 530 000
>>>                                             000B file, I was getting
>>>                 around:
>>>                                             In all buffers and fake
>>>                 work: 10~15s speed
>>>                 improvement ( from 90% HDD speed to 100%
>>>                                             HDD speed)
>>>                                             In all buffers and no
>>>                 fake work: 1~2s
>>>                 speed improvement ( from 90% HDD speed to
>>>                                             100% HDD speed)
>>>                 Changing the buffer size was giving
>>>                 different reading speeds but both were
>>>                 quite equal in how much they would change
>>>                                             when changing the buffer
>>>                 size.
>>>                 Finally, I could always confirm that I/O
>>>                                             was always the slowest
>>>                 thing while this
>>>                                             code was running.
>>>                                             For the ones wondering
>>>                 about the file
>>>                 size; it is both to avoid OS cache and to
>>>                                             make the reading at the
>>>                 main use-case
>>>                 these objects are for (large streams of
>>>                 bytes).
>>>                 @Pavel, are you open for discussion now
>>>                                             ;)? Need anything else?
>>>                                             On 21/10/2016 19:21,
>>>                 Pavel Rappo wrote:
>>>
>>>                 Just to append to my previous email.
>>>                 BufferedReader wraps any Reader out there.
>>>                 Not specifically FileReader. While
>>>                 you're talking about the case of effective
>>>                 reading from a file.
>>>                 I guess there's one existing
>>>                 possibility to provide exactly what
>>>                 you need (as I
>>>                 understand it) under this method:
>>>                 /**
>>>                   * Opens a file for reading,
>>>                 returning a {@code BufferedReader} to
>>>                 read text
>>>                   * from the file in an efficient
>>>                 manner...
>>>                     ...
>>>                   */
>>>                 java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>                 It can return _anything_ as long as it
>>>                 is a BufferedReader. We can do it, but it
>>>                 needs to be investigated not only for
>>>                 your favorite OS but for other OSes as
>>>                 well. Feel free to prototype this and
>>>                 we can discuss it on the list later.
>>>                 Thanks,
>>>                 -Pavel
>>>
>>>                     On 21 Oct 2016, at 18:56, Brunoais
>>>                     <brunoaiss at gmail.com>
>>>                       wrote:
>>>                     Pavel is right.
>>>                     In reality, I was expecting such
>>>                     BufferedReader to use only a
>>>                     single buffer and have that Buffer
>>>                     being filled asynchronously, not
>>>                     in a different Thread.
>>>                     Additionally, I don't have the
>>>                     intention of having a larger
>>>                     buffer than before unless stated
>>>                     through the API (the constructor).
>>>                     In my idea, internally, it is
>>>                     supposed to use
>>>                 java.nio.channels.AsynchronousFileChannel
>>>                     or equivalent.
>>>                     It does not prevent having two
>>>                     buffers and I do not intent to
>>>                     change BufferedReader itself. I'd
>>>                     do an BufferedAsyncReader of sorts
>>>                     (any name suggestion is welcome as
>>>                     I'm an awful namer).
>>>                     On 21/10/2016 18:38, Roger Riggs
>>>                     wrote:
>>>
>>>                         Hi Pavel,
>>>                         I think Brunoais asking for a
>>>                         double buffering scheme in
>>>                         which the implementation of
>>>                         BufferReader fills (a second
>>>                         buffer) in parallel with the
>>>                         application reading from the
>>>                         1st buffer
>>>                         and managing the swaps and
>>>                         async reads transparently.
>>>                         It would not change the API
>>>                         but would change the
>>>                         interactions between the
>>>                         buffered reader
>>>                         and the underlying stream.  It
>>>                         would also increase memory
>>>                         requirements and processing
>>>                         by introducing or using a
>>>                         separate thread and the
>>>                         necessary synchronization.
>>>                         Though I think the formal
>>>                         interface semantics could be
>>>                         maintained, I have doubts
>>>                         about compatibility and its
>>>                         unintended consequences on
>>>                         existing subclasses,
>>>                         applications and libraries.
>>>                         $.02, Roger
>>>                         On 10/21/16 1:22 PM, Pavel
>>>                         Rappo wrote:
>>>
>>>                             Off the top of my head, I
>>>                             would say it's not
>>>                             possible to change the
>>>                             design of an
>>>                             _extensible_ type that has
>>>                             been out there for 20 or
>>>                             so years. All these I/O
>>>                             streams from java.io <http://java.io>
>>>                             <http://java.io> were
>>>                             designed for simple
>>>                             synchronous use case.
>>>                             It's not that their design
>>>                             is flawed in some way,
>>>                             it's that they doesn't seem to
>>>                             suit your needs. Have you
>>>                             considered using
>>>                 java.nio.channels.AsynchronousFileChannel
>>>                             in your applications?
>>>                             -Pavel
>>>
>>>                                 On 21 Oct 2016, at
>>>                                 17:08, Brunoais
>>>                                 <brunoaiss at gmail.com>
>>>                                   wrote:
>>>                                 Any feedback on this?
>>>                                 I'm really interested
>>>                                 in implementing such
>>>                 BufferedReader/BufferedStreamReader
>>>                                 to allow speeding up
>>>                                 my applications
>>>                                 without having to
>>>                                 think in an
>>>                                 asynchronous way or
>>>                                 multi-threading while
>>>                                 programming with it.
>>>                                 That's why I'm asking
>>>                                 this here.
>>>                                 On 13/10/2016 14:45,
>>>                                 Brunoais wrote:
>>>
>>>                                     Hi,
>>>                                     I looked at
>>>                 BufferedReader
>>>                                     source code for
>>>                                     java 9 long with
>>>                                     the source code of
>>>                                     the
>>>                 channels/streams
>>>                                     used. I noticed
>>>                                     that, like in java
>>>                                     7, BufferedReader
>>>                                     does not use an
>>>                                     Async API to load
>>>                                     data from files,
>>>                                     instead, the data
>>>                                     loading is all
>>>                                     done synchronously
>>>                                     even when the OS
>>>                                     allows requesting
>>>                                     a file to be read
>>>                                     and getting a
>>>                                     warning later when
>>>                                     the file is
>>>                                     effectively read.
>>>                                     Why Is
>>>                 BufferedReader not
>>>                                     async while
>>>                                     providing a sync API?
>>>
>>>                 <BufferedNonBlockStream.java><Tests.java>
>>>
>>>
>>>
>>>
>>>
>>>                 -- 
>>>                 Sent from my phone
>>>
>>>
>>>
>>>
>>>         -- 
>>>         Sent from my phone
>>
>>
>
>


From paul.sandoz at oracle.com  Thu Oct 27 19:24:18 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Thu, 27 Oct 2016 12:24:18 -0700
Subject: Review Request: JDK-8168205: Should not default class path to CWD
	if -cp is not specified but -m is specified
In-Reply-To: <653C6BC1-90EE-4CF8-A3D0-CC63D98886B9@oracle.com>
References: <653C6BC1-90EE-4CF8-A3D0-CC63D98886B9@oracle.com>
Message-ID: <2E694DE9-076A-4FB5-B5E5-0B6247AF1906@oracle.com>


> On 26 Oct 2016, at 22:59, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Webrev:
>   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.00/index.html
> 
> If -cp is not specified and -m is not specified, the builtin system
> class loader will default the class path to the current working
> directory.  If -m is specified, no -cp and CLASSPATH environment
> variable is not set, it should mean no class path.  This patch fixes
> the case if -m is specified and the value of java.class.path is empty,
> e.g. via -Djava.class.path option, then no class path should be set.
> 
> This patch also updates the launcher code used for generating launcher
> for JDK tools. As the JDK tool no longer passes any class path, it
> removes APP_CLASSPATH macro.
> 

+1

I would be inclined to separate out the test for whether JAVA_CLASS_PATH property is present from the class loading tests e.g. if you don?t pass any argument to the Main assert the property value is non-null, otherwise assert on loading a resource.

Paul.


From paul.sandoz at oracle.com  Thu Oct 27 19:24:56 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Thu, 27 Oct 2016 12:24:56 -0700
Subject: RFR 8163553 java.lang.LinkageError from test
	java/lang/ThreadGroup/Stop.java
In-Reply-To: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com>
References: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com>
Message-ID: <8C56B19F-22EC-4E1E-AA47-E0D629231B07@oracle.com>

Gentle reminder.

Paul.

> On 18 Oct 2016, at 11:41, Paul Sandoz <Paul.Sandoz at oracle.com> wrote:
> 
> Hi,
> 
> Please review:
> 
>  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8163553-vh-mh-link-errors-not-wrapped/webrev/
> 
> This is the issue that motivated a change in the behaviour of indy wrapping Errors in BootstrapMethodError, JDK-8166974. I plan to push this issue with JDK-8166974 to hs, since they are related in behaviour even though there is no direct dependency between the patches.
> 
> 
> When invoking signature-polymorphic methods a similar but hardcoded dance occurs, with an appeal to Java code, to link the call site.
> 
> - MethodHandle.invoke/invokeExact (and the VH methods) would wrap all Errors in LinkageError. Now they are passed through, thus an Error like ThreadDeath is not wrapped.
> 
> - MethodHandle.invoke/invokeExact/invokeBasic throw Throwable, and in certain cases the Throwable is wrapped in an InternalError. In many other cases Error and RuntimeException are propagated, which i think in general is the right pattern, so i consistently applied that.
> 
> - I updated StringConcatFactory to also pass through Errors and avoid unduly wrapping StringConcatException in another instance of StringConcatException. (LambdaMetafactory and associated classes required no changes.)
> 
> Thanks,
> Paul.


From Alan.Bateman at oracle.com  Thu Oct 27 20:40:47 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 27 Oct 2016 21:40:47 +0100
Subject: Review Request: JDK-8168205: Should not default class path to CWD
	if -cp is not specified but -m is specified
In-Reply-To: <5CDB20F8-0CD9-428E-A757-DADB05F591B6@oracle.com>
References: <0C45950A-566E-4673-91CE-99EC59848F64@oracle.com>
	<80fdbd10-a88e-bf85-ffe9-3af907a2053e@oracle.com>
	<5CDB20F8-0CD9-428E-A757-DADB05F591B6@oracle.com>
Message-ID: <4ee242f0-b2da-c4c2-3e5f-e50da6a1b6ce@oracle.com>

On 27/10/2016 18:02, Mandy Chung wrote:

>> On Oct 27, 2016, at 2:44 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
>>
>> The updated comment in ClassLoaders might be a bit clearer if you drop "if defined?.
>>
> OK, dropped ?if defined?.
>
>> A minor comment on the test is that it could use ProcessTools.executeTestJava to avoid needing JAVA_TOOL.
> ProcessTools::executeTestJava launches the process inherited with CLASSPATH environment variable set.  I can use ProcessTools::createJavaProcessBuilder that calls JDKToolFinder::getJDKTool to find the launcher instead of having the test finding the launcher (no difference anyway).  I?m okay either way and updated the webrev:
>
>    http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.01/
>
Ah, I forgot CLASSPATH was inherited here. In any case, the updated 
webrev looks okay to me.

-Alan

From david.holmes at oracle.com  Thu Oct 27 20:43:33 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 06:43:33 +1000
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
In-Reply-To: <7875836b-0cbe-1218-7bee-270dc2115858@oracle.com>
References: <58115662.6080005@oracle.com>
	<7875836b-0cbe-1218-7bee-270dc2115858@oracle.com>
Message-ID: <8a8219ea-9558-1732-34a5-3f5761686f57@oracle.com>

On 27/10/2016 6:03 PM, Alan Bateman wrote:
> On 27/10/2016 02:20, Kumar Srinivasan wrote:
>
>> Hello,
>>
>> Please review enclosed fix for:
>> https://bugs.openjdk.java.net/browse/JDK-8168010
>>
>> Based on this discussion,
>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html
>>
>> these options are slated for deprecation, only at the documentation
>> level,
>> and will continue to operate with no perceivable behavior change, in
>> JDK9.
> This looks okay to me although I've no doubt that it will go unnoticed
> until the options are actually removed.

Right! The whole point of deprecation is to make it very clear something 
is going away so people stop using it. I thought the consensus from 
previous discussion was to deprecate in 9 and issue a warning. And yes 
that still requires changes to the tests, but so be it. Simply 
documenting deprecation in the help text is not really deprecating it!

David

> -Alan

From david.holmes at oracle.com  Thu Oct 27 20:51:14 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 06:51:14 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
Message-ID: <3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>

On 28/10/2016 3:12 AM, Roger Riggs wrote:
> Hi David,
>
> On 10/27/2016 12:57 PM, David Holmes wrote:
>> Hi Roger,
>>
>> On 28/10/2016 1:44 AM, Roger Riggs wrote:
>>> Please review a test fix for a timeout on a busy system in
>>> Process.waitFor a destroyed process.
>>
>> Won't that now cause the test to hang until timed-out by the harness?
> yes, but an in-app timeout is not much different than the harness timeout
> so I didn't complicate the test. Under normal system loading it should
> be 1-4 seconds.

Okay, but then the logic following this change is redundant:

2407             int exitValue = p.waitFor();  // wait for it to be done
2408
2409             start = System.nanoTime();
2410             p.waitFor(8, TimeUnit.SECONDS);
2411             end = System.nanoTime();
2412
2413             if ((end - start) > TimeUnit.SECONDS.toNanos(7))
2414                 fail("Test failed: waitFor took too long on a dead 
process. (" + (end - start) + "ns)"
2415                 + ", exitValue: " + exitValue);

David

> I contemplated increasing the timeout but given the issue is system loading
> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
> and the harness timeout is 2min.
>
> Roger
>
>>
>> David
>>
>>> Webrev:
>>>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>
>>> Thanks, Roger
>>>
>

From mandy.chung at oracle.com  Thu Oct 27 20:51:26 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 27 Oct 2016 13:51:26 -0700
Subject: Review Request: JDK-8168205: Should not default class path to CWD
	if -cp is not specified but -m is specified
In-Reply-To: <2E694DE9-076A-4FB5-B5E5-0B6247AF1906@oracle.com>
References: <653C6BC1-90EE-4CF8-A3D0-CC63D98886B9@oracle.com>
	<2E694DE9-076A-4FB5-B5E5-0B6247AF1906@oracle.com>
Message-ID: <7EC21B5B-0D1C-4D63-BBE8-3A92124B49D9@oracle.com>


> On Oct 27, 2016, at 12:24 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
> 
> 
>> On 26 Oct 2016, at 22:59, Mandy Chung <mandy.chung at oracle.com> wrote:
>> 
>> Webrev:
>>  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8168205/webrev.00/index.html
>> 
>> If -cp is not specified and -m is not specified, the builtin system
>> class loader will default the class path to the current working
>> directory.  If -m is specified, no -cp and CLASSPATH environment
>> variable is not set, it should mean no class path.  This patch fixes
>> the case if -m is specified and the value of java.class.path is empty,
>> e.g. via -Djava.class.path option, then no class path should be set.
>> 
>> This patch also updates the launcher code used for generating launcher
>> for JDK tools. As the JDK tool no longer passes any class path, it
>> removes APP_CLASSPATH macro.
>> 
> 
> +1
> 
> I would be inclined to separate out the test for whether JAVA_CLASS_PATH property is present from the class loading tests e.g. if you don?t pass any argument to the Main assert the property value is non-null, otherwise assert on loading a resource.

My apology for missing this.  I have just pushed webrev.01 and missed to include you in the reviewer list.

The main purpose of this test is to verify that CWD is not searched when -m is specified.  The check on the non-null value of ?java.class.path? is not strictly needed here.  A slightly better version would be to validate both the expected value of ?java.class.path? and expected CWD to be searched or not.  I can improve this test in another patch I?m preparing.

Mandy

From Roger.Riggs at Oracle.com  Thu Oct 27 20:58:26 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 27 Oct 2016 16:58:26 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
Message-ID: <6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>

Hi David,

On 10/27/2016 4:51 PM, David Holmes wrote:
> On 28/10/2016 3:12 AM, Roger Riggs wrote:
>> Hi David,
>>
>> On 10/27/2016 12:57 PM, David Holmes wrote:
>>> Hi Roger,
>>>
>>> On 28/10/2016 1:44 AM, Roger Riggs wrote:
>>>> Please review a test fix for a timeout on a busy system in
>>>> Process.waitFor a destroyed process.
>>>
>>> Won't that now cause the test to hang until timed-out by the harness?
>> yes, but an in-app timeout is not much different than the harness 
>> timeout
>> so I didn't complicate the test. Under normal system loading it should
>> be 1-4 seconds.
>
> Okay, but then the logic following this change is redundant:
>
> 2407             int exitValue = p.waitFor();  // wait for it to be done
> 2408
> 2409             start = System.nanoTime();
> 2410             p.waitFor(8, TimeUnit.SECONDS);
> 2411             end = System.nanoTime();
> 2412
> 2413             if ((end - start) > TimeUnit.SECONDS.toNanos(7))
> 2414                 fail("Test failed: waitFor took too long on a 
> dead process. (" + (end - start) + "ns)"
> 2415                 + ", exitValue: " + exitValue);
Not entirely, it is intended to check that when waitFor is used with a 
destroyed process
it completes (pretty much) immediately.

Roger

>
> David
>
>> I contemplated increasing the timeout but given the issue is system 
>> loading
>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>> and the harness timeout is 2min.
>>
>> Roger
>>
>>>
>>> David
>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>
>>>> Thanks, Roger
>>>>
>>


From david.holmes at oracle.com  Thu Oct 27 21:09:03 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 07:09:03 +1000
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
Message-ID: <f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>

You might try discussing on net-dev rather than core-libs-dev, to get 
additional historical info related to the io and nio file APIs.

David

On 28/10/2016 5:08 AM, Brunoais wrote:
> You are right. Even in windows it does not set the flags for async
> reads. It seems like it is windows itself that does the decision to
> buffer the contents based on its own heuristics.
>
> But... Why? Why won't it be? Why is there no API for it? How am I
> getting 100% HDD use and faster times when I fake work to delay getting
> more data and I only have a fluctuating 60-90% (always going up and
> down) when I use an InputStream?
> Is it related to how both classes cache and how frequently and how much
> each one asks for data?
>
> I really would prefer not having to read the source code because it
> takes a real long time T.T.
>
> I end up reinstating... And wondering...
>
> Why doesn't java provide a single-threaded non-block API for file reads
> for all OS that support it? I simply cannot find that information no
> matter how much I search on google, bing, duck duck go... Can any of you
> point me to whomever knows?
>
> On 27/10/2016 14:11, Vitaly Davidovich wrote:
>> I don't know about Windows specifically, but generally file systems
>> across major OS's will implement readahead in their IO scheduler when
>> they detect sequential scans.
>>
>> On Linux, you can also strace your test to confirm which syscalls are
>> emitted (you should be seeing plain read()'s there, with
>> FileInputStream and FileChannel).
>>
>> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
>> <mailto:brunoaiss at gmail.com>> wrote:
>>
>>     Thanks for the heads up.
>>
>>     I'll try that later. These tests are still useful then. Meanwhile,
>>     I'll end up also checking how FileChannel queries the OS on
>>     windows. I'm getting 100% HDD reads... Could it be that the OS
>>     reads the file ahead on its own?... Anyway, I'll look into it.
>>     Thanks for the heads up.
>>
>>
>>     On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>>
>>>
>>>     On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>>>     <mailto:brunoaiss at gmail.com>> wrote:
>>>
>>>         Oh... I see. In that case, it means something is terribly
>>>         wrong. It can be my initial tests, though.
>>>
>>>         I'm testing on both linux and windows and I'm getting
>>>         performance gains from using the FileChannel compared to
>>>         using FileInputStream... The tests also make sense based on
>>>         my predictions O_O...
>>>
>>>     FileInputStream requires copying native buffers holding the read
>>>     data to the java byte[].  If you're using direct ByteBuffer for
>>>     FileChannel, that whole memcpy is skipped.  Try comparing
>>>     FileChannel with HeapByteBuffer instead.
>>>
>>>
>>>         On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>         On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>>>         <mailto:brunoaiss at gmail.com>> wrote:
>>>>
>>>>             Did you read the C code?
>>>>
>>>>         I looked at the Linux code in the JDK.
>>>>
>>>>             Have you got any idea how many functions Windows or
>>>>             Linux (nearly all flavors) have for the read operation
>>>>             towards a file?
>>>>
>>>>         I do.
>>>>
>>>>
>>>>             I have already done that homework myself. I may not have
>>>>             read JVM's source code but I know well that there's
>>>>             functions on both Windows and Linux that provide such
>>>>             interface I mentioned although they require a slightly
>>>>             different treatment (and different constants).
>>>>
>>>>         You should read the JDK (native) source code instead of
>>>>         guessing/assuming.  On Linux, it doesn't use aio facilities
>>>>         for files.  The kernel io scheduler may issue readahead
>>>>         behind the scenes, but there's no nonblocking file io that's
>>>>         at the heart of your premise.
>>>>
>>>>
>>>>
>>>>             On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>
>>>>                 On Wednesday, October 26, 2016, Brunoais
>>>>                 <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>                 wrote:
>>>>
>>>>                     It is actually based on the premise that:
>>>>
>>>>                     1. The first call to
>>>>                 ReadableByteChannel.read(ByteBuffer) sets the OS
>>>>                        buffer size to fill in as the same size as
>>>>                 ByteBuffer.
>>>>
>>>>                 Why do you say that? AFAICT, it issues a read
>>>>                 syscall and that will block if the data isn't in
>>>>                 page cache.
>>>>
>>>>                     2. The consecutive calls to
>>>>                 ReadableByteChannel.read(ByteBuffer)
>>>>                     orders
>>>>                        the JVM to order the OS to execute memcpy()
>>>>                 to copy from its memory
>>>>                        to the shared memory created at ByteBuffer
>>>>                 instantiation (in
>>>>                     java 8)
>>>>                        using Unsafe and then for the JVM to update
>>>>                 the ByteBuffer fields.
>>>>
>>>>                 I think subsequent reads just invoke the same read
>>>>                 syscall, passing the current file offset maintained
>>>>                 by the file channel instance.
>>>>
>>>>                     3. The call will not block waiting for I/O and
>>>>                 it won't take longer
>>>>                        than the JNI interface if no new data exists.
>>>>                 However, it will
>>>>                     block
>>>>                        waiting for the OS to execute memcpy() to the
>>>>                 shared memory.
>>>>
>>>>                 So why do you think it won't block?
>>>>
>>>>
>>>>                     Is my premise wrong?
>>>>
>>>>                     If I read correctly, if I don't use a
>>>>                 DirectBuffer, there would be
>>>>                     even another intermediate buffer to copy data to
>>>>                 before giving it
>>>>                     to the "user" which would be useless.
>>>>
>>>>                 If you use a HeapByteBuffer, then there's an extra
>>>>                 copy from the native buffer to the Java buffer.
>>>>
>>>>
>>>>
>>>>                     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>
>>>>                         I believe I see where you coming from.
>>>>                 Please correct me if
>>>>                         I'm wrong.
>>>>
>>>>                         Your implementation is based on the premise
>>>>                 that a call to
>>>>                 ReadableByteChannel.read()
>>>>                         _initiates_ the operation and returns
>>>>                 immediately. The OS then
>>>>                         continues to fill
>>>>                         the buffer while there's a free space in the
>>>>                 buffer and the
>>>>                         channel hasn't encountered EOF.
>>>>
>>>>                         Is that right?
>>>>
>>>>                             On 25 Oct 2016, at 22:16, Brunoais
>>>>                 <brunoaiss at gmail.com>
>>>>                             wrote:
>>>>
>>>>                             Thank you for your time. I'll try to
>>>>                 explain it. I hope I
>>>>                             can clear it up.
>>>>                             First of it, I made a meaning mistake
>>>>                 between asynchronous
>>>>                             and non-blocking. This implementation
>>>>                 uses a non-blocking
>>>>                             algorithm internally while providing a
>>>>                 blocking-like
>>>>                             algorithm on the surface. It is
>>>>                 single-threaded and not
>>>>                             multi-threaded where one thread fetches
>>>>                 data and blocks
>>>>                             waiting and the other accumulates it and
>>>>                 provides to
>>>>                             whichever wants it.
>>>>
>>>>                             Second of it, I had made a mistake of
>>>>                 going after
>>>>                             BufferedReader instead of going after
>>>>                 BufferedInputStream.
>>>>                             If you want me to go after
>>>>                 BufferedReader it's ok but I
>>>>                             only thought that going after
>>>>                 BufferedInputStream would be
>>>>                             more generically useful than
>>>>                 BufferedReaderwhen I started
>>>>                             the poc.
>>>>
>>>>                             On to my code:
>>>>                             Short answers:
>>>>                                     ? The sleep(int) exists because
>>>>                 I don't know how
>>>>                             to wait until more data exists in the
>>>>                 buffer which is part
>>>>                             of read()'s contract.
>>>>                                     ? The ByteBuffer gives a buffer
>>>>                 that is filled by
>>>>                             the OS (what I believe Channels do)
>>>>                 instead of getting
>>>>                             data only         by demand (what I
>>>>                 believe Streams do).
>>>>                             Full answers:
>>>>                             The blockingFill(boolean) method is a
>>>>                 method for a busy
>>>>                             wait for a fill which is used
>>>>                 exclusively by the read()
>>>>                             method. All other methods use the
>>>>                 version that does not
>>>>                             sleep (fill(boolean)).
>>>>                 blockingFill(boolean)'s existance like that is only
>>>>                             because the read() method must not
>>>>                 return unless either:
>>>>
>>>>                                     ? The stream ended.
>>>>                                     ? The next byte is ready for
>>>>                 reading.
>>>>                             Additionally, statistically, that while
>>>>                 loop will rarely
>>>>                             evaluate to true as reads are in chunks
>>>>                 so readPos will be
>>>>                             behind writePos most of the time.
>>>>                             I have no idea if an interrupt will ever
>>>>                 happen, to be
>>>>                             honest. The main reasons why I'm using a
>>>>                 sleep is because
>>>>                             I didn't want a hog onto the CPU in a
>>>>                 full thread usage
>>>>                             busy wait and because I didn't find any
>>>>                 way of doing a
>>>>                             thread sleep in order to wake up later
>>>>                 when the buffer
>>>>                             managed by native code has more data.
>>>>                             The Non-blocking part is managed by the
>>>>                 buffer the OS
>>>>                             keeps filling most if not all the time.
>>>>                 That buffer is the
>>>>                             field
>>>>
>>>>                             ByteBuffer readBuffer
>>>>                             That's the gaining part against the
>>>>                 plain old Buffered
>>>>                             classes.
>>>>
>>>>
>>>>                             Did that make sense to you? Feel free to
>>>>                 ask anything else
>>>>                             you need.
>>>>
>>>>                             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>
>>>>                                 I've skimmed through the code and
>>>>                 I'm not sure I can
>>>>                                 see any asynchronicity
>>>>                                 (you were pointing at the lack of it
>>>>                 in BufferedReader).
>>>>                                 And the mechanics of this is very
>>>>                 puzzling to me, to
>>>>                                 be honest:
>>>>                                      void blockingFill(boolean
>>>>                 forced) throws
>>>>                                 IOException {
>>>>                  fill(forced);
>>>>                                          while (readPos == writePos) {
>>>>                                              try {
>>>>                  Thread.sleep(100);
>>>>                                              } catch
>>>>                 (InterruptedException e) {
>>>>                  // An interrupt may mean more data is
>>>>                                 available
>>>>                                              }
>>>>                  fill(forced);
>>>>                                          }
>>>>                                      }
>>>>                                 I thought you were suggesting that
>>>>                 we should utilize
>>>>                                 the tools which OS provides
>>>>                                 more efficiently. Instead we have
>>>>                 something that looks
>>>>                                 very similarly to a
>>>>                                 "busy loop" and... also who and when
>>>>                 is supposed to
>>>>                                 interrupt Thread.sleep()?
>>>>                                 Sorry, I'm not following. Could you
>>>>                 please explain how
>>>>                                 this is supposed to work?
>>>>
>>>>                                     On 24 Oct 2016, at 15:59, Brunoais
>>>>                                     <brunoaiss at gmail.com>
>>>>                                       wrote:
>>>>                                     Attached and sending!
>>>>                                     On 24/10/2016 13:48, Pavel Rappo
>>>>                 wrote:
>>>>
>>>>                                         Could you please send a new
>>>>                 email on this list
>>>>                                         with the source attached as a
>>>>                                         text file?
>>>>
>>>>                                             On 23 Oct 2016, at
>>>>                 19:14, Brunoais
>>>>                                             <brunoaiss at gmail.com>
>>>>                 wrote:
>>>>                 Here's my poc/prototype:
>>>>
>>>>                 http://pastebin.com/WRpYWDJF
>>>>
>>>>                                             I've implemented the
>>>>                 bare minimum of the
>>>>                 class that follows the same contract of
>>>>                 BufferedReader while signaling all issues
>>>>                                             I think it may have or
>>>>                 has in comments.
>>>>                                             I also wrote some
>>>>                 javadoc to help guiding
>>>>                 through the class.
>>>>                                             I could have used more
>>>>                 fields from
>>>>                 BufferedReader but the names were so
>>>>                 minimalistic that were confusing me. I
>>>>                 intent to change them before sending this
>>>>                                             to openJDK.
>>>>                                             One of the major
>>>>                 problems this has is long
>>>>                 overflowing. It is major because it is
>>>>                 hidden, it will be extremely rare and it
>>>>                 takes a really long time to reproduce.
>>>>                 There are different ways of dealing with
>>>>                                             it. From just
>>>>                 documenting to actually
>>>>                 making code that works with it.
>>>>                                             I built a simple test
>>>>                 code for it to have
>>>>                                             some ideas about
>>>>                 performance and correctness.
>>>>
>>>>                 http://pastebin.com/eh6LFgwT
>>>>
>>>>                                             This doesn't do a
>>>>                 through test if it is
>>>>                 actually working correctly but I see no
>>>>                 reason for it not working correctly after
>>>>                 fixing the 2 bugs that test found.
>>>>                                             I'll also leave here
>>>>                 some conclusions
>>>>                 about speed and resource consumption I found.
>>>>                                             I made tests with
>>>>                 default buffer sizes,
>>>>                 5000B 15_000B and 500_000B. I noticed
>>>>                 that, with my hardware, with the 1 530 000
>>>>                                             000B file, I was getting
>>>>                 around:
>>>>                                             In all buffers and fake
>>>>                 work: 10~15s speed
>>>>                 improvement ( from 90% HDD speed to 100%
>>>>                                             HDD speed)
>>>>                                             In all buffers and no
>>>>                 fake work: 1~2s
>>>>                 speed improvement ( from 90% HDD speed to
>>>>                                             100% HDD speed)
>>>>                 Changing the buffer size was giving
>>>>                 different reading speeds but both were
>>>>                 quite equal in how much they would change
>>>>                                             when changing the buffer
>>>>                 size.
>>>>                 Finally, I could always confirm that I/O
>>>>                                             was always the slowest
>>>>                 thing while this
>>>>                                             code was running.
>>>>                                             For the ones wondering
>>>>                 about the file
>>>>                 size; it is both to avoid OS cache and to
>>>>                                             make the reading at the
>>>>                 main use-case
>>>>                 these objects are for (large streams of
>>>>                 bytes).
>>>>                 @Pavel, are you open for discussion now
>>>>                                             ;)? Need anything else?
>>>>                                             On 21/10/2016 19:21,
>>>>                 Pavel Rappo wrote:
>>>>
>>>>                 Just to append to my previous email.
>>>>                 BufferedReader wraps any Reader out there.
>>>>                 Not specifically FileReader. While
>>>>                 you're talking about the case of effective
>>>>                 reading from a file.
>>>>                 I guess there's one existing
>>>>                 possibility to provide exactly what
>>>>                 you need (as I
>>>>                 understand it) under this method:
>>>>                 /**
>>>>                   * Opens a file for reading,
>>>>                 returning a {@code BufferedReader} to
>>>>                 read text
>>>>                   * from the file in an efficient
>>>>                 manner...
>>>>                     ...
>>>>                   */
>>>>
>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>                 It can return _anything_ as long as it
>>>>                 is a BufferedReader. We can do it, but it
>>>>                 needs to be investigated not only for
>>>>                 your favorite OS but for other OSes as
>>>>                 well. Feel free to prototype this and
>>>>                 we can discuss it on the list later.
>>>>                 Thanks,
>>>>                 -Pavel
>>>>
>>>>                     On 21 Oct 2016, at 18:56, Brunoais
>>>>                     <brunoaiss at gmail.com>
>>>>                       wrote:
>>>>                     Pavel is right.
>>>>                     In reality, I was expecting such
>>>>                     BufferedReader to use only a
>>>>                     single buffer and have that Buffer
>>>>                     being filled asynchronously, not
>>>>                     in a different Thread.
>>>>                     Additionally, I don't have the
>>>>                     intention of having a larger
>>>>                     buffer than before unless stated
>>>>                     through the API (the constructor).
>>>>                     In my idea, internally, it is
>>>>                     supposed to use
>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>                     or equivalent.
>>>>                     It does not prevent having two
>>>>                     buffers and I do not intent to
>>>>                     change BufferedReader itself. I'd
>>>>                     do an BufferedAsyncReader of sorts
>>>>                     (any name suggestion is welcome as
>>>>                     I'm an awful namer).
>>>>                     On 21/10/2016 18:38, Roger Riggs
>>>>                     wrote:
>>>>
>>>>                         Hi Pavel,
>>>>                         I think Brunoais asking for a
>>>>                         double buffering scheme in
>>>>                         which the implementation of
>>>>                         BufferReader fills (a second
>>>>                         buffer) in parallel with the
>>>>                         application reading from the
>>>>                         1st buffer
>>>>                         and managing the swaps and
>>>>                         async reads transparently.
>>>>                         It would not change the API
>>>>                         but would change the
>>>>                         interactions between the
>>>>                         buffered reader
>>>>                         and the underlying stream.  It
>>>>                         would also increase memory
>>>>                         requirements and processing
>>>>                         by introducing or using a
>>>>                         separate thread and the
>>>>                         necessary synchronization.
>>>>                         Though I think the formal
>>>>                         interface semantics could be
>>>>                         maintained, I have doubts
>>>>                         about compatibility and its
>>>>                         unintended consequences on
>>>>                         existing subclasses,
>>>>                         applications and libraries.
>>>>                         $.02, Roger
>>>>                         On 10/21/16 1:22 PM, Pavel
>>>>                         Rappo wrote:
>>>>
>>>>                             Off the top of my head, I
>>>>                             would say it's not
>>>>                             possible to change the
>>>>                             design of an
>>>>                             _extensible_ type that has
>>>>                             been out there for 20 or
>>>>                             so years. All these I/O
>>>>                             streams from java.io <http://java.io>
>>>>                             <http://java.io> were
>>>>                             designed for simple
>>>>                             synchronous use case.
>>>>                             It's not that their design
>>>>                             is flawed in some way,
>>>>                             it's that they doesn't seem to
>>>>                             suit your needs. Have you
>>>>                             considered using
>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>                             in your applications?
>>>>                             -Pavel
>>>>
>>>>                                 On 21 Oct 2016, at
>>>>                                 17:08, Brunoais
>>>>                                 <brunoaiss at gmail.com>
>>>>                                   wrote:
>>>>                                 Any feedback on this?
>>>>                                 I'm really interested
>>>>                                 in implementing such
>>>>                 BufferedReader/BufferedStreamReader
>>>>                                 to allow speeding up
>>>>                                 my applications
>>>>                                 without having to
>>>>                                 think in an
>>>>                                 asynchronous way or
>>>>                                 multi-threading while
>>>>                                 programming with it.
>>>>                                 That's why I'm asking
>>>>                                 this here.
>>>>                                 On 13/10/2016 14:45,
>>>>                                 Brunoais wrote:
>>>>
>>>>                                     Hi,
>>>>                                     I looked at
>>>>                 BufferedReader
>>>>                                     source code for
>>>>                                     java 9 long with
>>>>                                     the source code of
>>>>                                     the
>>>>                 channels/streams
>>>>                                     used. I noticed
>>>>                                     that, like in java
>>>>                                     7, BufferedReader
>>>>                                     does not use an
>>>>                                     Async API to load
>>>>                                     data from files,
>>>>                                     instead, the data
>>>>                                     loading is all
>>>>                                     done synchronously
>>>>                                     even when the OS
>>>>                                     allows requesting
>>>>                                     a file to be read
>>>>                                     and getting a
>>>>                                     warning later when
>>>>                                     the file is
>>>>                                     effectively read.
>>>>                                     Why Is
>>>>                 BufferedReader not
>>>>                                     async while
>>>>                                     providing a sync API?
>>>>
>>>>                 <BufferedNonBlockStream.java><Tests.java>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                 --                 Sent from my phone
>>>>
>>>>
>>>>
>>>>
>>>>         --         Sent from my phone
>>>
>>>
>>
>>
>

From kumar.x.srinivasan at oracle.com  Thu Oct 27 21:08:01 2016
From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan)
Date: Thu, 27 Oct 2016 14:08:01 -0700
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
In-Reply-To: <8a8219ea-9558-1732-34a5-3f5761686f57@oracle.com>
References: <58115662.6080005@oracle.com>
	<7875836b-0cbe-1218-7bee-270dc2115858@oracle.com>
	<8a8219ea-9558-1732-34a5-3f5761686f57@oracle.com>
Message-ID: <58126CB1.5080904@oracle.com>


>>> Hello,
>>>
>>> Please review enclosed fix for:
>>> https://bugs.openjdk.java.net/browse/JDK-8168010
>>>
>>> Based on this discussion,
>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html 
>>>
>>>
>>> these options are slated for deprecation, only at the documentation
>>> level,
>>> and will continue to operate with no perceivable behavior change, in
>>> JDK9.
>> This looks okay to me although I've no doubt that it will go unnoticed
>> until the options are actually removed.
>
> Right! The whole point of deprecation is to make it very clear 
> something is going away so people stop using it. I thought the 
> consensus from previous discussion was to deprecate in 9 and issue a 
> warning. And yes that still requires changes to the tests, but so be 
> it. Simply documenting deprecation in the help text is not really 
> deprecating it!

Issuing a warning is likely to break existing applications and break
tests, which parses the output. By consensus, for JDK9, documenting
seems to be the best approach.

Kumar


>
> David
>
>> -Alan


From rfscholte at apache.org  Thu Oct 27 21:27:27 2016
From: rfscholte at apache.org (Robert Scholte)
Date: Thu, 27 Oct 2016 23:27:27 +0200
Subject: JEP 293: Guidelines for JDK Command-Line Tool Options - @-files
Message-ID: <op.yp0gj1y1kdkhrr@desktop-2khsk44>

Hi,

I'm facing some troubles with the content of the @-files and the  
documentation[1] isn't helping me yet.

First of all it doesn't mention the encoding, I assume it is the OS  
specific encoding.

I'm facing issues with a long --module-path on Windows.
I noticed I can use the "normal" filename (not a URI or URL), but I need  
to escape the \ with an \, resulting in
"E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-command\\target\\classes;E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-weather\\target\\simple-weather-0.8-SNAPSHOT.jar;"  
(and much more entries)

You can use a space or a newline as separator between arguments, but if an  
argument contains spaces it must be surrounded with double quotes.
However, the path to my local (Maven) repository contains a space and  
JLink is complaining it cannot find these dependencies. It seems like I  
need to escape spaces too, though I haven't figured out how to do that.
Changing my local repo to a space-less path works. There should be a way  
to handle spaces, but how?

I'm not sure if there are other specific things to keep in mind when  
writing @-files, but it would be nice if that would be explained it the  
jep too.

thanks,
Robert

[1] http://openjdk.java.net/jeps/293

From vitalyd at gmail.com  Thu Oct 27 21:45:36 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Thu, 27 Oct 2016 17:45:36 -0400
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<1a7237c9-85be-c4ae-b32a-180002288b1c@oracle.com>
	<1d75e4aa-c916-bba4-1e52-313486fb2f36@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
Message-ID: <CAHjP37FgXdkfa=bhig3bZ4LdXnL2tC04mgRW4p9bDrwr7C_2oQ@mail.gmail.com>

On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com> wrote:

> You are right. Even in windows it does not set the flags for async reads.
> It seems like it is windows itself that does the decision to buffer the
> contents based on its own heuristics.
>
You mean nonblocking, not async, right? Two different things.

> But... Why? Why won't it be? Why is there no API for it? How am I getting
> 100% HDD use and faster times when I fake work to delay getting more data
> and I only have a fluctuating 60-90% (always going up and down) when I use
> an InputStream?
> Is it related to how both classes cache and how frequently and how much
> each one asks for data?
>
> I really would prefer not having to read the source code because it takes
> a real long time T.T.
>
> I end up reinstating... And wondering...
>
> Why doesn't java provide a single-threaded non-block API for file reads
> for all OS that support it? I simply cannot find that information no matter
> how much I search on google, bing, duck duck go... Can any of you point me
> to whomever knows?
>
 https://lwn.net/Articles/612483/ for Linux.  Unfortunately, the
nonblocking file io story is complicated and messy.

> On 27/10/2016 14:11, Vitaly Davidovich wrote:
>
> I don't know about Windows specifically, but generally file systems across
> major OS's will implement readahead in their IO scheduler when they detect
> sequential scans.
>
> On Linux, you can also strace your test to confirm which syscalls are
> emitted (you should be seeing plain read()'s there, with FileInputStream
> and FileChannel).
>
> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
> <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>> wrote:
>
>> Thanks for the heads up.
>>
>> I'll try that later. These tests are still useful then. Meanwhile, I'll
>> end up also checking how FileChannel queries the OS on windows. I'm getting
>> 100% HDD reads... Could it be that the OS reads the file ahead on its
>> own?... Anyway, I'll look into it. Thanks for the heads up.
>>
>> On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>
>>
>>
>> On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>> <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>> wrote:
>>
>>> Oh... I see. In that case, it means something is terribly wrong. It can
>>> be my initial tests, though.
>>>
>>> I'm testing on both linux and windows and I'm getting performance gains
>>> from using the FileChannel compared to using FileInputStream... The tests
>>> also make sense based on my predictions O_O...
>>>
>> FileInputStream requires copying native buffers holding the read data to
>> the java byte[].  If you're using direct ByteBuffer for FileChannel, that
>> whole memcpy is skipped.  Try comparing FileChannel with HeapByteBuffer
>> instead.
>>
>>>
>>> On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>
>>>
>>>
>>> On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>> <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>> wrote:
>>>
>>>> Did you read the C code?
>>>
>>> I looked at the Linux code in the JDK.
>>>
>>>> Have you got any idea how many functions Windows or Linux (nearly all
>>>> flavors) have for the read operation towards a file?
>>>
>>> I do.
>>>
>>>>
>>>> I have already done that homework myself. I may not have read JVM's
>>>> source code but I know well that there's functions on both Windows and
>>>> Linux that provide such interface I mentioned although they require a
>>>> slightly different treatment (and different constants).
>>>
>>> You should read the JDK (native) source code instead of
>>> guessing/assuming.  On Linux, it doesn't use aio facilities for files.  The
>>> kernel io scheduler may issue readahead behind the scenes, but there's no
>>> nonblocking file io that's at the heart of your premise.
>>>
>>>>
>>>>
>>>> On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wednesday, October 26, 2016, Brunoais <brunoaiss at gmail.com <mailto:
>>>>> brunoaiss at gmail.com>> wrote:
>>>>>
>>>>>     It is actually based on the premise that:
>>>>>
>>>>>     1. The first call to ReadableByteChannel.read(ByteBuffer) sets
>>>>> the OS
>>>>>        buffer size to fill in as the same size as ByteBuffer.
>>>>>
>>>>> Why do you say that? AFAICT, it issues a read syscall and that will
>>>>> block if the data isn't in page cache.
>>>>>
>>>>>     2. The consecutive calls to ReadableByteChannel.read(ByteBuffer)
>>>>>     orders
>>>>>        the JVM to order the OS to execute memcpy() to copy from its
>>>>> memory
>>>>>        to the shared memory created at ByteBuffer instantiation (in
>>>>>     java 8)
>>>>>        using Unsafe and then for the JVM to update the ByteBuffer
>>>>> fields.
>>>>>
>>>>> I think subsequent reads just invoke the same read syscall, passing
>>>>> the current file offset maintained by the file channel instance.
>>>>>
>>>>>     3. The call will not block waiting for I/O and it won't take longer
>>>>>        than the JNI interface if no new data exists. However, it will
>>>>>     block
>>>>>        waiting for the OS to execute memcpy() to the shared memory.
>>>>>
>>>>> So why do you think it won't block?
>>>>>
>>>>>
>>>>>     Is my premise wrong?
>>>>>
>>>>>     If I read correctly, if I don't use a DirectBuffer, there would be
>>>>>     even another intermediate buffer to copy data to before giving it
>>>>>     to the "user" which would be useless.
>>>>>
>>>>> If you use a HeapByteBuffer, then there's an extra copy from the
>>>>> native buffer to the Java buffer.
>>>>>
>>>>>
>>>>>
>>>>>     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>>
>>>>>         I believe I see where you coming from. Please correct me if
>>>>>         I'm wrong.
>>>>>
>>>>>         Your implementation is based on the premise that a call to
>>>>>         ReadableByteChannel.read()
>>>>>         _initiates_ the operation and returns immediately. The OS then
>>>>>         continues to fill
>>>>>         the buffer while there's a free space in the buffer and the
>>>>>         channel hasn't encountered EOF.
>>>>>
>>>>>         Is that right?
>>>>>
>>>>>             On 25 Oct 2016, at 22:16, Brunoais <brunoaiss at gmail.com>
>>>>>             wrote:
>>>>>
>>>>>             Thank you for your time. I'll try to explain it. I hope I
>>>>>             can clear it up.
>>>>>             First of it, I made a meaning mistake between asynchronous
>>>>>             and non-blocking. This implementation uses a non-blocking
>>>>>             algorithm internally while providing a blocking-like
>>>>>             algorithm on the surface. It is single-threaded and not
>>>>>             multi-threaded where one thread fetches data and blocks
>>>>>             waiting and the other accumulates it and provides to
>>>>>             whichever wants it.
>>>>>
>>>>>             Second of it, I had made a mistake of going after
>>>>>             BufferedReader instead of going after BufferedInputStream.
>>>>>             If you want me to go after BufferedReader it's ok but I
>>>>>             only thought that going after BufferedInputStream would be
>>>>>             more generically useful than BufferedReaderwhen I started
>>>>>             the poc.
>>>>>
>>>>>             On to my code:
>>>>>             Short answers:
>>>>>                     ? The sleep(int) exists because I don't know how
>>>>>             to wait until more data exists in the buffer which is part
>>>>>             of read()'s contract.
>>>>>                     ? The ByteBuffer gives a buffer that is filled by
>>>>>             the OS (what I believe Channels do) instead of getting
>>>>>             data only         by demand (what I believe Streams do).
>>>>>             Full answers:
>>>>>             The blockingFill(boolean) method is a method for a busy
>>>>>             wait for a fill which is used exclusively by the read()
>>>>>             method. All other methods use the version that does not
>>>>>             sleep (fill(boolean)).
>>>>>             blockingFill(boolean)'s existance like that is only
>>>>>             because the read() method must not return unless either:
>>>>>
>>>>>                     ? The stream ended.
>>>>>                     ? The next byte is ready for reading.
>>>>>             Additionally, statistically, that while loop will rarely
>>>>>             evaluate to true as reads are in chunks so readPos will be
>>>>>             behind writePos most of the time.
>>>>>             I have no idea if an interrupt will ever happen, to be
>>>>>             honest. The main reasons why I'm using a sleep is because
>>>>>             I didn't want a hog onto the CPU in a full thread usage
>>>>>             busy wait and because I didn't find any way of doing a
>>>>>             thread sleep in order to wake up later when the buffer
>>>>>             managed by native code has more data.
>>>>>             The Non-blocking part is managed by the buffer the OS
>>>>>             keeps filling most if not all the time. That buffer is the
>>>>>             field
>>>>>
>>>>>             ByteBuffer readBuffer
>>>>>             That's the gaining part against the plain old Buffered
>>>>>             classes.
>>>>>
>>>>>
>>>>>             Did that make sense to you? Feel free to ask anything else
>>>>>             you need.
>>>>>
>>>>>             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>>
>>>>>                 I've skimmed through the code and I'm not sure I can
>>>>>                 see any asynchronicity
>>>>>                 (you were pointing at the lack of it in
>>>>> BufferedReader).
>>>>>                 And the mechanics of this is very puzzling to me, to
>>>>>                 be honest:
>>>>>                      void blockingFill(boolean forced) throws
>>>>>                 IOException {
>>>>>                          fill(forced);
>>>>>                          while (readPos == writePos) {
>>>>>                              try {
>>>>>                                  Thread.sleep(100);
>>>>>                              } catch (InterruptedException e) {
>>>>>                                  // An interrupt may mean more data is
>>>>>                 available
>>>>>                              }
>>>>>                              fill(forced);
>>>>>                          }
>>>>>                      }
>>>>>                 I thought you were suggesting that we should utilize
>>>>>                 the tools which OS provides
>>>>>                 more efficiently. Instead we have something that looks
>>>>>                 very similarly to a
>>>>>                 "busy loop" and... also who and when is supposed to
>>>>>                 interrupt Thread.sleep()?
>>>>>                 Sorry, I'm not following. Could you please explain how
>>>>>                 this is supposed to work?
>>>>>
>>>>>                     On 24 Oct 2016, at 15:59, Brunoais
>>>>>                     <brunoaiss at gmail.com>
>>>>>                       wrote:
>>>>>                     Attached and sending!
>>>>>                     On 24/10/2016 13:48, Pavel Rappo wrote:
>>>>>
>>>>>                         Could you please send a new email on this list
>>>>>                         with the source attached as a
>>>>>                         text file?
>>>>>
>>>>>                             On 23 Oct 2016, at 19:14, Brunoais
>>>>>                             <brunoaiss at gmail.com>
>>>>>                               wrote:
>>>>>                             Here's my poc/prototype:
>>>>>
>>>>>                             http://pastebin.com/WRpYWDJF
>>>>>
>>>>>                             I've implemented the bare minimum of the
>>>>>                             class that follows the same contract of
>>>>>                             BufferedReader while signaling all issues
>>>>>                             I think it may have or has in comments.
>>>>>                             I also wrote some javadoc to help guiding
>>>>>                             through the class.
>>>>>                             I could have used more fields from
>>>>>                             BufferedReader but the names were so
>>>>>                             minimalistic that were confusing me. I
>>>>>                             intent to change them before sending this
>>>>>                             to openJDK.
>>>>>                             One of the major problems this has is long
>>>>>                             overflowing. It is major because it is
>>>>>                             hidden, it will be extremely rare and it
>>>>>                             takes a really long time to reproduce.
>>>>>                             There are different ways of dealing with
>>>>>                             it. From just documenting to actually
>>>>>                             making code that works with it.
>>>>>                             I built a simple test code for it to have
>>>>>                             some ideas about performance and
>>>>> correctness.
>>>>>
>>>>>                             http://pastebin.com/eh6LFgwT
>>>>>
>>>>>                             This doesn't do a through test if it is
>>>>>                             actually working correctly but I see no
>>>>>                             reason for it not working correctly after
>>>>>                             fixing the 2 bugs that test found.
>>>>>                             I'll also leave here some conclusions
>>>>>                             about speed and resource consumption I
>>>>> found.
>>>>>                             I made tests with default buffer sizes,
>>>>>                             5000B 15_000B and 500_000B. I noticed
>>>>>                             that, with my hardware, with the 1 530 000
>>>>>                             000B file, I was getting around:
>>>>>                             In all buffers and fake work: 10~15s speed
>>>>>                             improvement ( from 90% HDD speed to 100%
>>>>>                             HDD speed)
>>>>>                             In all buffers and no fake work: 1~2s
>>>>>                             speed improvement ( from 90% HDD speed to
>>>>>                             100% HDD speed)
>>>>>                             Changing the buffer size was giving
>>>>>                             different reading speeds but both were
>>>>>                             quite equal in how much they would change
>>>>>                             when changing the buffer size.
>>>>>                             Finally, I could always confirm that I/O
>>>>>                             was always the slowest thing while this
>>>>>                             code was running.
>>>>>                             For the ones wondering about the file
>>>>>                             size; it is both to avoid OS cache and to
>>>>>                             make the reading at the main use-case
>>>>>                             these objects are for (large streams of
>>>>>                             bytes).
>>>>>                             @Pavel, are you open for discussion now
>>>>>                             ;)? Need anything else?
>>>>>                             On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>>
>>>>>                                 Just to append to my previous email.
>>>>>                                 BufferedReader wraps any Reader out
>>>>> there.
>>>>>                                 Not specifically FileReader. While
>>>>>                                 you're talking about the case of
>>>>> effective
>>>>>                                 reading from a file.
>>>>>                                 I guess there's one existing
>>>>>                                 possibility to provide exactly what
>>>>>                                 you need (as I
>>>>>                                 understand it) under this method:
>>>>>                                 /**
>>>>>                                   * Opens a file for reading,
>>>>>                                 returning a {@code BufferedReader} to
>>>>>                                 read text
>>>>>                                   * from the file in an efficient
>>>>>                                 manner...
>>>>>                                     ...
>>>>>                                   */
>>>>>                                 java.nio.file.Files#newBuffere
>>>>> dReader(java.nio.file.Path)
>>>>>                                 It can return _anything_ as long as it
>>>>>                                 is a BufferedReader. We can do it, but
>>>>> it
>>>>>                                 needs to be investigated not only for
>>>>>                                 your favorite OS but for other OSes as
>>>>>                                 well. Feel free to prototype this and
>>>>>                                 we can discuss it on the list later.
>>>>>                                 Thanks,
>>>>>                                 -Pavel
>>>>>
>>>>>                                     On 21 Oct 2016, at 18:56, Brunoais
>>>>>                                     <brunoaiss at gmail.com>
>>>>>                                       wrote:
>>>>>                                     Pavel is right.
>>>>>                                     In reality, I was expecting such
>>>>>                                     BufferedReader to use only a
>>>>>                                     single buffer and have that Buffer
>>>>>                                     being filled asynchronously, not
>>>>>                                     in a different Thread.
>>>>>                                     Additionally, I don't have the
>>>>>                                     intention of having a larger
>>>>>                                     buffer than before unless stated
>>>>>                                     through the API (the constructor).
>>>>>                                     In my idea, internally, it is
>>>>>                                     supposed to use
>>>>>                                     java.nio.channels.Asynchronous
>>>>> FileChannel
>>>>>                                     or equivalent.
>>>>>                                     It does not prevent having two
>>>>>                                     buffers and I do not intent to
>>>>>                                     change BufferedReader itself. I'd
>>>>>                                     do an BufferedAsyncReader of sorts
>>>>>                                     (any name suggestion is welcome as
>>>>>                                     I'm an awful namer).
>>>>>                                     On 21/10/2016 18:38, Roger Riggs
>>>>>                                     wrote:
>>>>>
>>>>>                                         Hi Pavel,
>>>>>                                         I think Brunoais asking for a
>>>>>                                         double buffering scheme in
>>>>>                                         which the implementation of
>>>>>                                         BufferReader fills (a second
>>>>>                                         buffer) in parallel with the
>>>>>                                         application reading from the
>>>>>                                         1st buffer
>>>>>                                         and managing the swaps and
>>>>>                                         async reads transparently.
>>>>>                                         It would not change the API
>>>>>                                         but would change the
>>>>>                                         interactions between the
>>>>>                                         buffered reader
>>>>>                                         and the underlying stream.  It
>>>>>                                         would also increase memory
>>>>>                                         requirements and processing
>>>>>                                         by introducing or using a
>>>>>                                         separate thread and the
>>>>>                                         necessary synchronization.
>>>>>                                         Though I think the formal
>>>>>                                         interface semantics could be
>>>>>                                         maintained, I have doubts
>>>>>                                         about compatibility and its
>>>>>                                         unintended consequences on
>>>>>                                         existing subclasses,
>>>>>                                         applications and libraries.
>>>>>                                         $.02, Roger
>>>>>                                         On 10/21/16 1:22 PM, Pavel
>>>>>                                         Rappo wrote:
>>>>>
>>>>>                                             Off the top of my head, I
>>>>>                                             would say it's not
>>>>>                                             possible to change the
>>>>>                                             design of an
>>>>>                                             _extensible_ type that has
>>>>>                                             been out there for 20 or
>>>>>                                             so years. All these I/O
>>>>>                                             streams from java.io
>>>>>                                             <http://java.io> were
>>>>>                                             designed for simple
>>>>>                                             synchronous use case.
>>>>>                                             It's not that their design
>>>>>                                             is flawed in some way,
>>>>>                                             it's that they doesn't
>>>>> seem to
>>>>>                                             suit your needs. Have you
>>>>>                                             considered using
>>>>>
>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>                                             in your applications?
>>>>>                                             -Pavel
>>>>>
>>>>>                                                 On 21 Oct 2016, at
>>>>>                                                 17:08, Brunoais
>>>>>                                                 <brunoaiss at gmail.com>
>>>>>                                                   wrote:
>>>>>                                                 Any feedback on this?
>>>>>                                                 I'm really interested
>>>>>                                                 in implementing such
>>>>>
>>>>> BufferedReader/BufferedStreamReader
>>>>>                                                 to allow speeding up
>>>>>                                                 my applications
>>>>>                                                 without having to
>>>>>                                                 think in an
>>>>>                                                 asynchronous way or
>>>>>                                                 multi-threading while
>>>>>                                                 programming with it.
>>>>>                                                 That's why I'm asking
>>>>>                                                 this here.
>>>>>                                                 On 13/10/2016 14:45,
>>>>>                                                 Brunoais wrote:
>>>>>
>>>>>                                                     Hi,
>>>>>                                                     I looked at
>>>>>                                                     BufferedReader
>>>>>                                                     source code for
>>>>>                                                     java 9 long with
>>>>>                                                     the source code of
>>>>>                                                     the
>>>>>                                                     channels/streams
>>>>>                                                     used. I noticed
>>>>>                                                     that, like in java
>>>>>                                                     7, BufferedReader
>>>>>                                                     does not use an
>>>>>                                                     Async API to load
>>>>>                                                     data from files,
>>>>>                                                     instead, the data
>>>>>                                                     loading is all
>>>>>                                                     done synchronously
>>>>>                                                     even when the OS
>>>>>                                                     allows requesting
>>>>>                                                     a file to be read
>>>>>                                                     and getting a
>>>>>                                                     warning later when
>>>>>                                                     the file is
>>>>>                                                     effectively read.
>>>>>                                                     Why Is
>>>>>                                                     BufferedReader not
>>>>>                                                     async while
>>>>>                                                     providing a sync
>>>>> API?
>>>>>
>>>>>                     <BufferedNonBlockStream.java><Tests.java>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from my phone
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Sent from my phone
>>>
>>>
>>>
>>
>>
>
>

-- 
Sent from my phone

From david.holmes at oracle.com  Thu Oct 27 22:59:23 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 08:59:23 +1000
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
	<f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>
Message-ID: <fad5ab74-f5e5-77d9-1ff1-fe6dd8b67b5f@oracle.com>

Sorry nio-dev ...

On 28/10/2016 7:09 AM, David Holmes wrote:
> You might try discussing on net-dev rather than core-libs-dev, to get
> additional historical info related to the io and nio file APIs.
>
> David
>
> On 28/10/2016 5:08 AM, Brunoais wrote:
>> You are right. Even in windows it does not set the flags for async
>> reads. It seems like it is windows itself that does the decision to
>> buffer the contents based on its own heuristics.
>>
>> But... Why? Why won't it be? Why is there no API for it? How am I
>> getting 100% HDD use and faster times when I fake work to delay getting
>> more data and I only have a fluctuating 60-90% (always going up and
>> down) when I use an InputStream?
>> Is it related to how both classes cache and how frequently and how much
>> each one asks for data?
>>
>> I really would prefer not having to read the source code because it
>> takes a real long time T.T.
>>
>> I end up reinstating... And wondering...
>>
>> Why doesn't java provide a single-threaded non-block API for file reads
>> for all OS that support it? I simply cannot find that information no
>> matter how much I search on google, bing, duck duck go... Can any of you
>> point me to whomever knows?
>>
>> On 27/10/2016 14:11, Vitaly Davidovich wrote:
>>> I don't know about Windows specifically, but generally file systems
>>> across major OS's will implement readahead in their IO scheduler when
>>> they detect sequential scans.
>>>
>>> On Linux, you can also strace your test to confirm which syscalls are
>>> emitted (you should be seeing plain read()'s there, with
>>> FileInputStream and FileChannel).
>>>
>>> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>
>>>     Thanks for the heads up.
>>>
>>>     I'll try that later. These tests are still useful then. Meanwhile,
>>>     I'll end up also checking how FileChannel queries the OS on
>>>     windows. I'm getting 100% HDD reads... Could it be that the OS
>>>     reads the file ahead on its own?... Anyway, I'll look into it.
>>>     Thanks for the heads up.
>>>
>>>
>>>     On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>     On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>>>>     <mailto:brunoaiss at gmail.com>> wrote:
>>>>
>>>>         Oh... I see. In that case, it means something is terribly
>>>>         wrong. It can be my initial tests, though.
>>>>
>>>>         I'm testing on both linux and windows and I'm getting
>>>>         performance gains from using the FileChannel compared to
>>>>         using FileInputStream... The tests also make sense based on
>>>>         my predictions O_O...
>>>>
>>>>     FileInputStream requires copying native buffers holding the read
>>>>     data to the java byte[].  If you're using direct ByteBuffer for
>>>>     FileChannel, that whole memcpy is skipped.  Try comparing
>>>>     FileChannel with HeapByteBuffer instead.
>>>>
>>>>
>>>>         On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>>>
>>>>>
>>>>>         On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>>>>         <mailto:brunoaiss at gmail.com>> wrote:
>>>>>
>>>>>             Did you read the C code?
>>>>>
>>>>>         I looked at the Linux code in the JDK.
>>>>>
>>>>>             Have you got any idea how many functions Windows or
>>>>>             Linux (nearly all flavors) have for the read operation
>>>>>             towards a file?
>>>>>
>>>>>         I do.
>>>>>
>>>>>
>>>>>             I have already done that homework myself. I may not have
>>>>>             read JVM's source code but I know well that there's
>>>>>             functions on both Windows and Linux that provide such
>>>>>             interface I mentioned although they require a slightly
>>>>>             different treatment (and different constants).
>>>>>
>>>>>         You should read the JDK (native) source code instead of
>>>>>         guessing/assuming.  On Linux, it doesn't use aio facilities
>>>>>         for files.  The kernel io scheduler may issue readahead
>>>>>         behind the scenes, but there's no nonblocking file io that's
>>>>>         at the heart of your premise.
>>>>>
>>>>>
>>>>>
>>>>>             On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>>
>>>>>
>>>>>
>>>>>                 On Wednesday, October 26, 2016, Brunoais
>>>>>                 <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>>                 wrote:
>>>>>
>>>>>                     It is actually based on the premise that:
>>>>>
>>>>>                     1. The first call to
>>>>>                 ReadableByteChannel.read(ByteBuffer) sets the OS
>>>>>                        buffer size to fill in as the same size as
>>>>>                 ByteBuffer.
>>>>>
>>>>>                 Why do you say that? AFAICT, it issues a read
>>>>>                 syscall and that will block if the data isn't in
>>>>>                 page cache.
>>>>>
>>>>>                     2. The consecutive calls to
>>>>>                 ReadableByteChannel.read(ByteBuffer)
>>>>>                     orders
>>>>>                        the JVM to order the OS to execute memcpy()
>>>>>                 to copy from its memory
>>>>>                        to the shared memory created at ByteBuffer
>>>>>                 instantiation (in
>>>>>                     java 8)
>>>>>                        using Unsafe and then for the JVM to update
>>>>>                 the ByteBuffer fields.
>>>>>
>>>>>                 I think subsequent reads just invoke the same read
>>>>>                 syscall, passing the current file offset maintained
>>>>>                 by the file channel instance.
>>>>>
>>>>>                     3. The call will not block waiting for I/O and
>>>>>                 it won't take longer
>>>>>                        than the JNI interface if no new data exists.
>>>>>                 However, it will
>>>>>                     block
>>>>>                        waiting for the OS to execute memcpy() to the
>>>>>                 shared memory.
>>>>>
>>>>>                 So why do you think it won't block?
>>>>>
>>>>>
>>>>>                     Is my premise wrong?
>>>>>
>>>>>                     If I read correctly, if I don't use a
>>>>>                 DirectBuffer, there would be
>>>>>                     even another intermediate buffer to copy data to
>>>>>                 before giving it
>>>>>                     to the "user" which would be useless.
>>>>>
>>>>>                 If you use a HeapByteBuffer, then there's an extra
>>>>>                 copy from the native buffer to the Java buffer.
>>>>>
>>>>>
>>>>>
>>>>>                     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>>
>>>>>                         I believe I see where you coming from.
>>>>>                 Please correct me if
>>>>>                         I'm wrong.
>>>>>
>>>>>                         Your implementation is based on the premise
>>>>>                 that a call to
>>>>>                 ReadableByteChannel.read()
>>>>>                         _initiates_ the operation and returns
>>>>>                 immediately. The OS then
>>>>>                         continues to fill
>>>>>                         the buffer while there's a free space in the
>>>>>                 buffer and the
>>>>>                         channel hasn't encountered EOF.
>>>>>
>>>>>                         Is that right?
>>>>>
>>>>>                             On 25 Oct 2016, at 22:16, Brunoais
>>>>>                 <brunoaiss at gmail.com>
>>>>>                             wrote:
>>>>>
>>>>>                             Thank you for your time. I'll try to
>>>>>                 explain it. I hope I
>>>>>                             can clear it up.
>>>>>                             First of it, I made a meaning mistake
>>>>>                 between asynchronous
>>>>>                             and non-blocking. This implementation
>>>>>                 uses a non-blocking
>>>>>                             algorithm internally while providing a
>>>>>                 blocking-like
>>>>>                             algorithm on the surface. It is
>>>>>                 single-threaded and not
>>>>>                             multi-threaded where one thread fetches
>>>>>                 data and blocks
>>>>>                             waiting and the other accumulates it and
>>>>>                 provides to
>>>>>                             whichever wants it.
>>>>>
>>>>>                             Second of it, I had made a mistake of
>>>>>                 going after
>>>>>                             BufferedReader instead of going after
>>>>>                 BufferedInputStream.
>>>>>                             If you want me to go after
>>>>>                 BufferedReader it's ok but I
>>>>>                             only thought that going after
>>>>>                 BufferedInputStream would be
>>>>>                             more generically useful than
>>>>>                 BufferedReaderwhen I started
>>>>>                             the poc.
>>>>>
>>>>>                             On to my code:
>>>>>                             Short answers:
>>>>>                                     ? The sleep(int) exists because
>>>>>                 I don't know how
>>>>>                             to wait until more data exists in the
>>>>>                 buffer which is part
>>>>>                             of read()'s contract.
>>>>>                                     ? The ByteBuffer gives a buffer
>>>>>                 that is filled by
>>>>>                             the OS (what I believe Channels do)
>>>>>                 instead of getting
>>>>>                             data only         by demand (what I
>>>>>                 believe Streams do).
>>>>>                             Full answers:
>>>>>                             The blockingFill(boolean) method is a
>>>>>                 method for a busy
>>>>>                             wait for a fill which is used
>>>>>                 exclusively by the read()
>>>>>                             method. All other methods use the
>>>>>                 version that does not
>>>>>                             sleep (fill(boolean)).
>>>>>                 blockingFill(boolean)'s existance like that is only
>>>>>                             because the read() method must not
>>>>>                 return unless either:
>>>>>
>>>>>                                     ? The stream ended.
>>>>>                                     ? The next byte is ready for
>>>>>                 reading.
>>>>>                             Additionally, statistically, that while
>>>>>                 loop will rarely
>>>>>                             evaluate to true as reads are in chunks
>>>>>                 so readPos will be
>>>>>                             behind writePos most of the time.
>>>>>                             I have no idea if an interrupt will ever
>>>>>                 happen, to be
>>>>>                             honest. The main reasons why I'm using a
>>>>>                 sleep is because
>>>>>                             I didn't want a hog onto the CPU in a
>>>>>                 full thread usage
>>>>>                             busy wait and because I didn't find any
>>>>>                 way of doing a
>>>>>                             thread sleep in order to wake up later
>>>>>                 when the buffer
>>>>>                             managed by native code has more data.
>>>>>                             The Non-blocking part is managed by the
>>>>>                 buffer the OS
>>>>>                             keeps filling most if not all the time.
>>>>>                 That buffer is the
>>>>>                             field
>>>>>
>>>>>                             ByteBuffer readBuffer
>>>>>                             That's the gaining part against the
>>>>>                 plain old Buffered
>>>>>                             classes.
>>>>>
>>>>>
>>>>>                             Did that make sense to you? Feel free to
>>>>>                 ask anything else
>>>>>                             you need.
>>>>>
>>>>>                             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>>
>>>>>                                 I've skimmed through the code and
>>>>>                 I'm not sure I can
>>>>>                                 see any asynchronicity
>>>>>                                 (you were pointing at the lack of it
>>>>>                 in BufferedReader).
>>>>>                                 And the mechanics of this is very
>>>>>                 puzzling to me, to
>>>>>                                 be honest:
>>>>>                                      void blockingFill(boolean
>>>>>                 forced) throws
>>>>>                                 IOException {
>>>>>                  fill(forced);
>>>>>                                          while (readPos == writePos) {
>>>>>                                              try {
>>>>>                  Thread.sleep(100);
>>>>>                                              } catch
>>>>>                 (InterruptedException e) {
>>>>>                  // An interrupt may mean more data is
>>>>>                                 available
>>>>>                                              }
>>>>>                  fill(forced);
>>>>>                                          }
>>>>>                                      }
>>>>>                                 I thought you were suggesting that
>>>>>                 we should utilize
>>>>>                                 the tools which OS provides
>>>>>                                 more efficiently. Instead we have
>>>>>                 something that looks
>>>>>                                 very similarly to a
>>>>>                                 "busy loop" and... also who and when
>>>>>                 is supposed to
>>>>>                                 interrupt Thread.sleep()?
>>>>>                                 Sorry, I'm not following. Could you
>>>>>                 please explain how
>>>>>                                 this is supposed to work?
>>>>>
>>>>>                                     On 24 Oct 2016, at 15:59, Brunoais
>>>>>                                     <brunoaiss at gmail.com>
>>>>>                                       wrote:
>>>>>                                     Attached and sending!
>>>>>                                     On 24/10/2016 13:48, Pavel Rappo
>>>>>                 wrote:
>>>>>
>>>>>                                         Could you please send a new
>>>>>                 email on this list
>>>>>                                         with the source attached as a
>>>>>                                         text file?
>>>>>
>>>>>                                             On 23 Oct 2016, at
>>>>>                 19:14, Brunoais
>>>>>                                             <brunoaiss at gmail.com>
>>>>>                 wrote:
>>>>>                 Here's my poc/prototype:
>>>>>
>>>>>                 http://pastebin.com/WRpYWDJF
>>>>>
>>>>>                                             I've implemented the
>>>>>                 bare minimum of the
>>>>>                 class that follows the same contract of
>>>>>                 BufferedReader while signaling all issues
>>>>>                                             I think it may have or
>>>>>                 has in comments.
>>>>>                                             I also wrote some
>>>>>                 javadoc to help guiding
>>>>>                 through the class.
>>>>>                                             I could have used more
>>>>>                 fields from
>>>>>                 BufferedReader but the names were so
>>>>>                 minimalistic that were confusing me. I
>>>>>                 intent to change them before sending this
>>>>>                                             to openJDK.
>>>>>                                             One of the major
>>>>>                 problems this has is long
>>>>>                 overflowing. It is major because it is
>>>>>                 hidden, it will be extremely rare and it
>>>>>                 takes a really long time to reproduce.
>>>>>                 There are different ways of dealing with
>>>>>                                             it. From just
>>>>>                 documenting to actually
>>>>>                 making code that works with it.
>>>>>                                             I built a simple test
>>>>>                 code for it to have
>>>>>                                             some ideas about
>>>>>                 performance and correctness.
>>>>>
>>>>>                 http://pastebin.com/eh6LFgwT
>>>>>
>>>>>                                             This doesn't do a
>>>>>                 through test if it is
>>>>>                 actually working correctly but I see no
>>>>>                 reason for it not working correctly after
>>>>>                 fixing the 2 bugs that test found.
>>>>>                                             I'll also leave here
>>>>>                 some conclusions
>>>>>                 about speed and resource consumption I found.
>>>>>                                             I made tests with
>>>>>                 default buffer sizes,
>>>>>                 5000B 15_000B and 500_000B. I noticed
>>>>>                 that, with my hardware, with the 1 530 000
>>>>>                                             000B file, I was getting
>>>>>                 around:
>>>>>                                             In all buffers and fake
>>>>>                 work: 10~15s speed
>>>>>                 improvement ( from 90% HDD speed to 100%
>>>>>                                             HDD speed)
>>>>>                                             In all buffers and no
>>>>>                 fake work: 1~2s
>>>>>                 speed improvement ( from 90% HDD speed to
>>>>>                                             100% HDD speed)
>>>>>                 Changing the buffer size was giving
>>>>>                 different reading speeds but both were
>>>>>                 quite equal in how much they would change
>>>>>                                             when changing the buffer
>>>>>                 size.
>>>>>                 Finally, I could always confirm that I/O
>>>>>                                             was always the slowest
>>>>>                 thing while this
>>>>>                                             code was running.
>>>>>                                             For the ones wondering
>>>>>                 about the file
>>>>>                 size; it is both to avoid OS cache and to
>>>>>                                             make the reading at the
>>>>>                 main use-case
>>>>>                 these objects are for (large streams of
>>>>>                 bytes).
>>>>>                 @Pavel, are you open for discussion now
>>>>>                                             ;)? Need anything else?
>>>>>                                             On 21/10/2016 19:21,
>>>>>                 Pavel Rappo wrote:
>>>>>
>>>>>                 Just to append to my previous email.
>>>>>                 BufferedReader wraps any Reader out there.
>>>>>                 Not specifically FileReader. While
>>>>>                 you're talking about the case of effective
>>>>>                 reading from a file.
>>>>>                 I guess there's one existing
>>>>>                 possibility to provide exactly what
>>>>>                 you need (as I
>>>>>                 understand it) under this method:
>>>>>                 /**
>>>>>                   * Opens a file for reading,
>>>>>                 returning a {@code BufferedReader} to
>>>>>                 read text
>>>>>                   * from the file in an efficient
>>>>>                 manner...
>>>>>                     ...
>>>>>                   */
>>>>>
>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>                 It can return _anything_ as long as it
>>>>>                 is a BufferedReader. We can do it, but it
>>>>>                 needs to be investigated not only for
>>>>>                 your favorite OS but for other OSes as
>>>>>                 well. Feel free to prototype this and
>>>>>                 we can discuss it on the list later.
>>>>>                 Thanks,
>>>>>                 -Pavel
>>>>>
>>>>>                     On 21 Oct 2016, at 18:56, Brunoais
>>>>>                     <brunoaiss at gmail.com>
>>>>>                       wrote:
>>>>>                     Pavel is right.
>>>>>                     In reality, I was expecting such
>>>>>                     BufferedReader to use only a
>>>>>                     single buffer and have that Buffer
>>>>>                     being filled asynchronously, not
>>>>>                     in a different Thread.
>>>>>                     Additionally, I don't have the
>>>>>                     intention of having a larger
>>>>>                     buffer than before unless stated
>>>>>                     through the API (the constructor).
>>>>>                     In my idea, internally, it is
>>>>>                     supposed to use
>>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>>                     or equivalent.
>>>>>                     It does not prevent having two
>>>>>                     buffers and I do not intent to
>>>>>                     change BufferedReader itself. I'd
>>>>>                     do an BufferedAsyncReader of sorts
>>>>>                     (any name suggestion is welcome as
>>>>>                     I'm an awful namer).
>>>>>                     On 21/10/2016 18:38, Roger Riggs
>>>>>                     wrote:
>>>>>
>>>>>                         Hi Pavel,
>>>>>                         I think Brunoais asking for a
>>>>>                         double buffering scheme in
>>>>>                         which the implementation of
>>>>>                         BufferReader fills (a second
>>>>>                         buffer) in parallel with the
>>>>>                         application reading from the
>>>>>                         1st buffer
>>>>>                         and managing the swaps and
>>>>>                         async reads transparently.
>>>>>                         It would not change the API
>>>>>                         but would change the
>>>>>                         interactions between the
>>>>>                         buffered reader
>>>>>                         and the underlying stream.  It
>>>>>                         would also increase memory
>>>>>                         requirements and processing
>>>>>                         by introducing or using a
>>>>>                         separate thread and the
>>>>>                         necessary synchronization.
>>>>>                         Though I think the formal
>>>>>                         interface semantics could be
>>>>>                         maintained, I have doubts
>>>>>                         about compatibility and its
>>>>>                         unintended consequences on
>>>>>                         existing subclasses,
>>>>>                         applications and libraries.
>>>>>                         $.02, Roger
>>>>>                         On 10/21/16 1:22 PM, Pavel
>>>>>                         Rappo wrote:
>>>>>
>>>>>                             Off the top of my head, I
>>>>>                             would say it's not
>>>>>                             possible to change the
>>>>>                             design of an
>>>>>                             _extensible_ type that has
>>>>>                             been out there for 20 or
>>>>>                             so years. All these I/O
>>>>>                             streams from java.io <http://java.io>
>>>>>                             <http://java.io> were
>>>>>                             designed for simple
>>>>>                             synchronous use case.
>>>>>                             It's not that their design
>>>>>                             is flawed in some way,
>>>>>                             it's that they doesn't seem to
>>>>>                             suit your needs. Have you
>>>>>                             considered using
>>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>>                             in your applications?
>>>>>                             -Pavel
>>>>>
>>>>>                                 On 21 Oct 2016, at
>>>>>                                 17:08, Brunoais
>>>>>                                 <brunoaiss at gmail.com>
>>>>>                                   wrote:
>>>>>                                 Any feedback on this?
>>>>>                                 I'm really interested
>>>>>                                 in implementing such
>>>>>                 BufferedReader/BufferedStreamReader
>>>>>                                 to allow speeding up
>>>>>                                 my applications
>>>>>                                 without having to
>>>>>                                 think in an
>>>>>                                 asynchronous way or
>>>>>                                 multi-threading while
>>>>>                                 programming with it.
>>>>>                                 That's why I'm asking
>>>>>                                 this here.
>>>>>                                 On 13/10/2016 14:45,
>>>>>                                 Brunoais wrote:
>>>>>
>>>>>                                     Hi,
>>>>>                                     I looked at
>>>>>                 BufferedReader
>>>>>                                     source code for
>>>>>                                     java 9 long with
>>>>>                                     the source code of
>>>>>                                     the
>>>>>                 channels/streams
>>>>>                                     used. I noticed
>>>>>                                     that, like in java
>>>>>                                     7, BufferedReader
>>>>>                                     does not use an
>>>>>                                     Async API to load
>>>>>                                     data from files,
>>>>>                                     instead, the data
>>>>>                                     loading is all
>>>>>                                     done synchronously
>>>>>                                     even when the OS
>>>>>                                     allows requesting
>>>>>                                     a file to be read
>>>>>                                     and getting a
>>>>>                                     warning later when
>>>>>                                     the file is
>>>>>                                     effectively read.
>>>>>                                     Why Is
>>>>>                 BufferedReader not
>>>>>                                     async while
>>>>>                                     providing a sync API?
>>>>>
>>>>>                 <BufferedNonBlockStream.java><Tests.java>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                 --                 Sent from my phone
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>         --         Sent from my phone
>>>>
>>>>
>>>
>>>
>>

From david.holmes at oracle.com  Thu Oct 27 23:13:15 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 09:13:15 +1000
Subject: RFR: 8168640: (fc) Avoiding AtomicBoolean in
	FileInput/-OutputStream improves startup
In-Reply-To: <0b5ad904-822c-1d6c-5e44-e1c103c77964@gmail.com>
References: <17555aa0-c610-aa3c-e80b-ec8cc12a2315@oracle.com>
	<e150dd50-7e12-ab7b-384d-0785bccaf3ab@redhat.com>
	<6e4cd8df-9a66-4bf2-7205-76b9b18cf131@gmail.com>
	<c6562d42-7e18-7c86-287c-0d1cc901517c@oracle.com>
	<0b5ad904-822c-1d6c-5e44-e1c103c77964@gmail.com>
Message-ID: <321a87ec-5457-d477-9d35-95fab12753eb@oracle.com>

On 27/10/2016 3:46 PM, Peter Levart wrote:
> Hi David,
>
>
> On 10/27/2016 04:57 AM, David Holmes wrote:
>> On 25/10/2016 10:36 PM, Peter Levart wrote:
>>> Hi Claes,
>>>
>>>
>>> On 10/25/2016 01:09 PM, Aleksey Shipilev wrote:
>>>> On 10/25/2016 12:51 PM, Claes Redestad wrote:
>>>>> Avoiding AtomicBoolean improves startup and footprint for a sample of
>>>>> small applications:
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~redestad/8168640/webrev.01/
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8168640
>>>> I would generally disagree to avoid Atomics to improve startup. In this
>>>> case, we always lock on close(), even for a short time. I wonder if
>>>> using plain Unsafe.compareAndSwap instead of Atomic would be cleaner,
>>>> instead of introducing performance bugs with synchronized-s.
>>>>
>>>> Thanks,
>>>> -Aleksey
>>>>
>>>>
>>>
>>> In addition, there is no need for this:
>>>
>>>  379                     boolean closed;
>>>  380                     synchronized (closeLock) {
>>>  381                         closed = this.closed;
>>>  382                     }
>>>
>>> A simple:
>>>
>>>         boolean closed = this.closed;
>>>
>>> is equivalent, since this.closed is volatile.
>>
>> Sorry but where exactly is the code referenced above?
>
> It was in the previous webrev in the patched FileInputStream. But the
> webrev has been changed in-place...

Okay - nothing to see then :)

Thanks,
David

> Regards, Peter
>
>>
>> Thanks,
>> David
>>
>>>
>>> But something else bothers me with this code. There is a race. Here are
>>> the relevant parts:
>>>
>>>  316     public void close() throws IOException {
>>>  317         if (closed) {
>>>  318             return;
>>>  319         }
>>>  320         synchronized (closeLock) {
>>>  321             if (closed) {
>>>  322                 return;
>>>  323             }
>>>  324             closed = true;
>>>  325         }
>>>  326
>>>  327         FileChannel fc = channel;
>>>  328         if (fc != null) {
>>>  329            fc.close();
>>>  330         }
>>>  331
>>>  332         fd.closeAll(new Closeable() {
>>>  333             public void close() throws IOException {
>>>  334                close0();
>>>  335            }
>>>  336         });
>>>  337     }
>>>
>>> and:
>>>
>>>  372     public FileChannel getChannel() {
>>>  373         FileChannel fc = this.channel;
>>>  374         if (fc == null) {
>>>  375             synchronized (this) {
>>>  376                 fc = this.channel;
>>>  377                 if (fc == null) {
>>>  378                     this.channel = fc = FileChannelImpl.open(fd,
>>> path, true, false, this);
>>>  379                     boolean closed;
>>>  380                     synchronized (closeLock) {
>>>  381                         closed = this.closed;
>>>  382                     }
>>>  383                     if (closed) {
>>>  384                         try {
>>>  385                             fc.close();
>>>  386                         } catch (IOException ioe) {
>>>  387                             throw new InternalError(ioe); // should
>>> not happen
>>>  388                         }
>>>  389                     }
>>>  390                 }
>>>  391             }
>>>  392         }
>>>  393         return fc;
>>>  394     }
>>>
>>>
>>> Suppose Thread 1 is executing close() up to line 326, Then Thread 2
>>> kicks-in and executes getChannel() for the 1st time on this
>>> FileInputStream. It opens FileChannelImpl and finds closed flag already
>>> set, so it closes the channel. Thread 1 then resumes from line 326 and
>>> finds 'channel' field already set, so it closes the channel once more.
>>>
>>> FileChannelImpl.close() may be idempotent, but why not making sure it is
>>> called just once?
>>>
>>> In order to fix this, you should read the 'channel' field in close()
>>> while holding closeLock and you should publish the 'channel' field in
>>> getChannel() while holding closeLock. Like this:
>>>
>>> diff -r 2e7a303cd1ec
>>> src/java.base/share/classes/java/io/FileInputStream.java
>>> --- a/src/java.base/share/classes/java/io/FileInputStream.java Wed Oct
>>> 19 11:45:43 2016 +0800
>>> +++ b/src/java.base/share/classes/java/io/FileInputStream.java Tue Oct
>>> 25 14:33:05 2016 +0200
>>> @@ -26,7 +26,6 @@
>>>  package java.io;
>>>
>>>  import java.nio.channels.FileChannel;
>>> -import java.util.concurrent.atomic.AtomicBoolean;
>>>  import sun.nio.ch.FileChannelImpl;
>>>
>>>
>>> @@ -60,7 +59,9 @@
>>>
>>>      private volatile FileChannel channel;
>>>
>>> -    private final AtomicBoolean closed = new AtomicBoolean(false);
>>> +    private final Object closeLock = new Object();
>>> +
>>> +    private volatile boolean closed;
>>>
>>>      /**
>>>       * Creates a <code>FileInputStream</code> by
>>> @@ -313,12 +314,18 @@
>>>       * @spec JSR-51
>>>       */
>>>      public void close() throws IOException {
>>> -        if (!closed.compareAndSet(false, true)) {
>>> -            // if compareAndSet() returns false closed was already true
>>> +        if (closed) {
>>>              return;
>>>          }
>>> +        FileChannel fc;
>>> +        synchronized (closeLock) {
>>> +            if (closed) {
>>> +                return;
>>> +            }
>>> +            closed = true;
>>> +            fc = channel;
>>> +        }
>>>
>>> -        FileChannel fc = channel;
>>>          if (fc != null) {
>>>             fc.close();
>>>          }
>>> @@ -369,8 +376,13 @@
>>>              synchronized (this) {
>>>                  fc = this.channel;
>>>                  if (fc == null) {
>>> -                    this.channel = fc = FileChannelImpl.open(fd, path,
>>> true, false, this);
>>> -                    if (closed.get()) {
>>> +                    fc = FileChannelImpl.open(fd, path, true, false,
>>> this);
>>> +                    boolean closed;
>>> +                    synchronized (closeLock) {
>>> +                        closed = this.closed;
>>> +                        this.channel = fc;
>>> +                    }
>>> +                    if (closed) {
>>>                          try {
>>>                              fc.close();
>>>                          } catch (IOException ioe) {
>>>
>>>
>>> Regards, Peter
>>>
>>>
>

From david.holmes at oracle.com  Thu Oct 27 23:17:12 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 09:17:12 +1000
Subject: RFR: 8168010: Deprecate obsolete launcher -d32/-d64 options
In-Reply-To: <58126CB1.5080904@oracle.com>
References: <58115662.6080005@oracle.com>
	<7875836b-0cbe-1218-7bee-270dc2115858@oracle.com>
	<8a8219ea-9558-1732-34a5-3f5761686f57@oracle.com>
	<58126CB1.5080904@oracle.com>
Message-ID: <02fdc4f1-639d-5f06-24f5-1dd71093b55f@oracle.com>

On 28/10/2016 7:08 AM, Kumar Srinivasan wrote:
>
>>>> Hello,
>>>>
>>>> Please review enclosed fix for:
>>>> https://bugs.openjdk.java.net/browse/JDK-8168010
>>>>
>>>> Based on this discussion,
>>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004934.html
>>>>
>>>>
>>>> these options are slated for deprecation, only at the documentation
>>>> level,
>>>> and will continue to operate with no perceivable behavior change, in
>>>> JDK9.
>>> This looks okay to me although I've no doubt that it will go unnoticed
>>> until the options are actually removed.
>>
>> Right! The whole point of deprecation is to make it very clear
>> something is going away so people stop using it. I thought the
>> consensus from previous discussion was to deprecate in 9 and issue a
>> warning. And yes that still requires changes to the tests, but so be
>> it. Simply documenting deprecation in the help text is not really
>> deprecating it!
>
> Issuing a warning is likely to break existing applications and break
> tests, which parses the output. By consensus, for JDK9, documenting
> seems to be the best approach.

No one will read that documentation. When this is actually removed, all 
the breakage you fear now will occur then. I'm not sure who was involved 
in this "consensus" - yes a warning make break some things but that just 
means we have to fix them. This is what we have done with deprecated VM 
options. If you deprecate this such that a warning is produced then you 
add that information to the release notes - which will have higher 
visibility than the help doc.

David

> Kumar
>
>
>>
>> David
>>
>>> -Alan
>

From david.holmes at oracle.com  Thu Oct 27 23:22:52 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 09:22:52 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
Message-ID: <bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>

On 28/10/2016 6:58 AM, Roger Riggs wrote:
> Hi David,
>
> On 10/27/2016 4:51 PM, David Holmes wrote:
>> On 28/10/2016 3:12 AM, Roger Riggs wrote:
>>> Hi David,
>>>
>>> On 10/27/2016 12:57 PM, David Holmes wrote:
>>>> Hi Roger,
>>>>
>>>> On 28/10/2016 1:44 AM, Roger Riggs wrote:
>>>>> Please review a test fix for a timeout on a busy system in
>>>>> Process.waitFor a destroyed process.
>>>>
>>>> Won't that now cause the test to hang until timed-out by the harness?
>>> yes, but an in-app timeout is not much different than the harness
>>> timeout
>>> so I didn't complicate the test. Under normal system loading it should
>>> be 1-4 seconds.
>>
>> Okay, but then the logic following this change is redundant:
>>
>> 2407             int exitValue = p.waitFor();  // wait for it to be done
>> 2408
>> 2409             start = System.nanoTime();
>> 2410             p.waitFor(8, TimeUnit.SECONDS);
>> 2411             end = System.nanoTime();
>> 2412
>> 2413             if ((end - start) > TimeUnit.SECONDS.toNanos(7))
>> 2414                 fail("Test failed: waitFor took too long on a
>> dead process. (" + (end - start) + "ns)"
>> 2415                 + ", exitValue: " + exitValue);
> Not entirely, it is intended to check that when waitFor is used with a
> destroyed process it completes (pretty much) immediately.

But this is the second waitFor call after the process is destroyed. 
Sorry I don't really see the point.

David

>
> Roger
>
>>
>> David
>>
>>> I contemplated increasing the timeout but given the issue is system
>>> loading
>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>>> and the harness timeout is 2min.
>>>
>>> Roger
>>>
>>>>
>>>> David
>>>>
>>>>> Webrev:
>>>>>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>
>>>>> Thanks, Roger
>>>>>
>>>
>

From roger.riggs at oracle.com  Fri Oct 28 00:46:17 2016
From: roger.riggs at oracle.com (Roger Riggs)
Date: Thu, 27 Oct 2016 20:46:17 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
Message-ID: <d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>

Hi David,

On 10/27/16 7:22 PM, David Holmes wrote:
> On 28/10/2016 6:58 AM, Roger Riggs wrote:
>> Hi David,
>>
>> On 10/27/2016 4:51 PM, David Holmes wrote:
>>>
>>>>> Won't that now cause the test to hang until timed-out by the harness?
>>>> yes, but an in-app timeout is not much different than the harness
>>>> timeout
>>>> so I didn't complicate the test. Under normal system loading it should
>>>> be 1-4 seconds.
>>>
>>> Okay, but then the logic following this change is redundant:
>>>
>>> 2407             int exitValue = p.waitFor();  // wait for it to be 
>>> done
>>> 2408
>>> 2409             start = System.nanoTime();
>>> 2410             p.waitFor(8, TimeUnit.SECONDS);
>>> 2411             end = System.nanoTime();
>>> 2412
>>> 2413             if ((end - start) > TimeUnit.SECONDS.toNanos(7))
>>> 2414                 fail("Test failed: waitFor took too long on a
>>> dead process. (" + (end - start) + "ns)"
>>> 2415                 + ", exitValue: " + exitValue);
>> Not entirely, it is intended to check that when waitFor is used with a
>> destroyed process it completes (pretty much) immediately.
>
> But this is the second waitFor call after the process is destroyed. 
> Sorry I don't really see the point.
The tests were added to determine if waitFor(timeout) was handling the 
timeout parameter correctly.
The 2nd test here was to check the datapath when the process already 
been destroyed.
The 'extra' waitFor() ensures the process is gone.
But you are correct, this exercises a path through the waitFor code that 
does not even look
at the timeout value so I'll remove the entire 2nd case.

Webrev updated in place.
   http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

Thanks, Roger
> David
>
>>
>> Roger
>>
>>>
>>> David
>>>
>>>> I contemplated increasing the timeout but given the issue is system
>>>> loading
>>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>>>> and the harness timeout is 2min.
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> David
>>>>>
>>>>>> Webrev:
>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>
>>>>>> Thanks, Roger
>>>>>>
>>>>
>>


From david.holmes at oracle.com  Fri Oct 28 01:00:43 2016
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 28 Oct 2016 11:00:43 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
Message-ID: <18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>

Hi Roger,

On 28/10/2016 10:46 AM, Roger Riggs wrote:
> Hi David,
>
> On 10/27/16 7:22 PM, David Holmes wrote:
>> On 28/10/2016 6:58 AM, Roger Riggs wrote:
>>> Hi David,
>>>
>>> On 10/27/2016 4:51 PM, David Holmes wrote:
>>>>
>>>>>> Won't that now cause the test to hang until timed-out by the harness?
>>>>> yes, but an in-app timeout is not much different than the harness
>>>>> timeout
>>>>> so I didn't complicate the test. Under normal system loading it should
>>>>> be 1-4 seconds.
>>>>
>>>> Okay, but then the logic following this change is redundant:
>>>>
>>>> 2407             int exitValue = p.waitFor();  // wait for it to be
>>>> done
>>>> 2408
>>>> 2409             start = System.nanoTime();
>>>> 2410             p.waitFor(8, TimeUnit.SECONDS);
>>>> 2411             end = System.nanoTime();
>>>> 2412
>>>> 2413             if ((end - start) > TimeUnit.SECONDS.toNanos(7))
>>>> 2414                 fail("Test failed: waitFor took too long on a
>>>> dead process. (" + (end - start) + "ns)"
>>>> 2415                 + ", exitValue: " + exitValue);
>>> Not entirely, it is intended to check that when waitFor is used with a
>>> destroyed process it completes (pretty much) immediately.
>>
>> But this is the second waitFor call after the process is destroyed.
>> Sorry I don't really see the point.
> The tests were added to determine if waitFor(timeout) was handling the
> timeout parameter correctly.
> The 2nd test here was to check the datapath when the process already
> been destroyed.
> The 'extra' waitFor() ensures the process is gone.
> But you are correct, this exercises a path through the waitFor code that
> does not even look
> at the timeout value so I'll remove the entire 2nd case.
>
> Webrev updated in place.
>   http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

Okay, but now you don't need the p.waitFor() after the destroy() at all.

Hmmm ... I guess either we want the original code with a longer timeout 
to accommodate slow/loaded platforms (to test that waitFor(n) after a 
destroy() is timely); or everything after destroy() can be deleted.

Cheers,
David


> Thanks, Roger
>> David
>>
>>>
>>> Roger
>>>
>>>>
>>>> David
>>>>
>>>>> I contemplated increasing the timeout but given the issue is system
>>>>> loading
>>>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>>>>> and the harness timeout is 2min.
>>>>>
>>>>> Roger
>>>>>
>>>>>>
>>>>>> David
>>>>>>
>>>>>>> Webrev:
>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>>
>>>>>>> Thanks, Roger
>>>>>>>
>>>>>
>>>
>

From amy.lu at oracle.com  Fri Oct 28 03:15:10 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Fri, 28 Oct 2016 11:15:10 +0800
Subject: JDK 9 RFR of JDK-8168524: Remove two jdk_nio tests from
	ProblemList: BashStreams and DeleteInterference.java
In-Reply-To: <FF5D6A00-DC71-4D85-AFC7-40A3D02F91C5@oracle.com>
References: <aaac152a-4b2a-4d1b-e05b-046a8b87202d@oracle.com>
	<FF5D6A00-DC71-4D85-AFC7-40A3D02F91C5@oracle.com>
Message-ID: <ed918ec3-c87c-341d-49e6-f4b4356f05a2@oracle.com>

On 10/24/16 10:50 PM, Brian Burkhalter wrote:
> +1 for the DeleteInterference portion.
Thank you Brian.

Still need a reviewer for BashStreams test.

Thanks,
Amy

>
> Brian
>
> On Oct 24, 2016, at 1:08 AM, Amy Lu <amy.lu at oracle.com> wrote:
>
>> Please review the patch to bring back two tests:
>>
>> java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
>>     More debug info has been added to this test (JDK-8162902). It's hard to reproduce the failure. Let's bring back this test. Once test failure happen again, the debug info could help with JDK-8156511 fix.
>>
>> java/nio/charset/coders/BashStreams.java 8149712 generic-all
>>     Mentioned issue seems does not exist anymore, suggest to bring back the test.
>>
>> bug: https://bugs.openjdk.java.net/browse/JDK-8168524
>>
>> Thanks,
>> Amy
>>
>> --- old/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
>> +++ new/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
>> @@ -186,10 +186,6 @@
>> java/nio/file/WatchService/MayFlies.java 7158947 solaris-all Solaris 11
>> java/nio/file/WatchService/LotsOfEvents.java 7158947 solaris-all Solaris 11
>>
>> -java/nio/charset/coders/BashStreams.java 8149712 generic-all
>> -
>> -java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
>> -
>> ############################################################################
>>
>> # jdk_rmi
>>
>>


From henry.jen at oracle.com  Fri Oct 28 05:05:06 2016
From: henry.jen at oracle.com (Henry Jen)
Date: Thu, 27 Oct 2016 22:05:06 -0700
Subject: JEP 293: Guidelines for JDK Command-Line Tool Options -
 @-files
In-Reply-To: <op.yp0gj1y1kdkhrr@desktop-2khsk44>
References: <op.yp0gj1y1kdkhrr@desktop-2khsk44>
Message-ID: <etPan.5812dc82.46191f8f.2ad@oracle.com>

OS-specific encoding, but has to be ASCII friendly, modern system with UTF-8 as system encoding should work just fine.

Space in quote should work just fine, for example, ?c:\\Program Files? should be correct. Can you post messages from JLink? Also if you can verify java is working OK with space, that would be helpful to tell if there is something different in JLink.

Cheers,
Henry

On October 27, 2016 at 2:27:44 PM, Robert Scholte (rfscholte at apache.org) wrote:
> Hi,
>  
> I'm facing some troubles with the content of the @-files and the
> documentation[1] isn't helping me yet.
>  
> First of all it doesn't mention the encoding, I assume it is the OS
> specific encoding.
>  
> I'm facing issues with a long --module-path on Windows.
> I noticed I can use the "normal" filename (not a URI or URL), but I need
> to escape the \ with an \, resulting in
> "E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-command\\target\\classes;E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-weather\\target\\simple-weather-0.8-SNAPSHOT.jar;"  
> (and much more entries)
>  
> You can use a space or a newline as separator between arguments, but if an
> argument contains spaces it must be surrounded with double quotes.
> However, the path to my local (Maven) repository contains a space and
> JLink is complaining it cannot find these dependencies. It seems like I
> need to escape spaces too, though I haven't figured out how to do that.
> Changing my local repo to a space-less path works. There should be a way
> to handle spaces, but how?
>  
> I'm not sure if there are other specific things to keep in mind when
> writing @-files, but it would be nice if that would be explained it the
> jep too.
>  
> thanks,
> Robert
>  
> [1] http://openjdk.java.net/jeps/293
>  


From brunoaiss at gmail.com  Fri Oct 28 06:53:34 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Fri, 28 Oct 2016 07:53:34 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <CAHjP37FgXdkfa=bhig3bZ4LdXnL2tC04mgRW4p9bDrwr7C_2oQ@mail.gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
	<CAHjP37FgXdkfa=bhig3bZ4LdXnL2tC04mgRW4p9bDrwr7C_2oQ@mail.gmail.com>
Message-ID: <86eb18da-c7c8-4f08-4ffb-48ed053e983b@gmail.com>



On 27/10/2016 22:45, Vitaly Davidovich wrote:
>
>
> On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com 
> <mailto:brunoaiss at gmail.com>> wrote:
>
>     You are right. Even in windows it does not set the flags for async
>     reads. It seems like it is windows itself that does the decision
>     to buffer the contents based on its own heuristics.
>
> You mean nonblocking, not async, right? Two different things.
Ups. Mistyped. On windows docs they seem to call it async...
>
>     But... Why? Why won't it be? Why is there no API for it? How am I
>     getting 100% HDD use and faster times when I fake work to delay
>     getting more data and I only have a fluctuating 60-90% (always
>     going up and down) when I use an InputStream?
>     Is it related to how both classes cache and how frequently and how
>     much each one asks for data?
>
>     I really would prefer not having to read the source code because
>     it takes a real long time T.T.
>
>     I end up reinstating... And wondering...
>
>     Why doesn't java provide a single-threaded non-block API for file
>     reads for all OS that support it? I simply cannot find that
>     information no matter how much I search on google, bing, duck duck
>     go... Can any of you point me to whomever knows?
>
> https://lwn.net/Articles/612483/ for Linux.  Unfortunately, the 
> nonblocking file io story is complicated and messy.
In Windows manual and Linux manual, they call asynchonous I/O for what 
is non-blocking synchonous I/O for the program that runs on the OS.
http://man7.org/linux/man-pages/man3/aio_read.3.html
http://man7.org/linux/man-pages/man7/aio.7.html
http://man7.org/linux/man-pages/man7/sigevent.7.html

This does not block, the OS writes directly to the user buffer, does not 
run on a different user thread and uses either signals or a function 
pointer as a callback when the operation is completed. Reading the 
manual, it seems it can even be the own thread. If it is with signals, I 
do know it is completely non-blocking and single-threaded (from the 
"user" thread's perspective). I'd like to see this in java...
I guess I only have the NIO2 for that, then with AsynchronousFileChannel.

>     On 27/10/2016 14:11, Vitaly Davidovich wrote:
>>     I don't know about Windows specifically, but generally file
>>     systems across major OS's will implement readahead in their IO
>>     scheduler when they detect sequential scans.
>>
>>     On Linux, you can also strace your test to confirm which syscalls
>>     are emitted (you should be seeing plain read()'s there, with
>>     FileInputStream and FileChannel).
>>
>>     On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
>>     <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>> wrote:
>>
>>         Thanks for the heads up.
>>
>>         I'll try that later. These tests are still useful then.
>>         Meanwhile, I'll end up also checking how FileChannel queries
>>         the OS on windows. I'm getting 100% HDD reads... Could it be
>>         that the OS reads the file ahead on its own?... Anyway, I'll
>>         look into it. Thanks for the heads up.
>>
>>
>>         On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>>
>>>
>>>         On Thu, Oct 27, 2016 at 8:34 AM, Brunoais
>>>         <brunoaiss at gmail.com
>>>         <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>> wrote:
>>>
>>>             Oh... I see. In that case, it means something is
>>>             terribly wrong. It can be my initial tests, though.
>>>
>>>             I'm testing on both linux and windows and I'm getting
>>>             performance gains from using the FileChannel compared to
>>>             using FileInputStream... The tests also make sense based
>>>             on my predictions O_O...
>>>
>>>         FileInputStream requires copying native buffers holding the
>>>         read data to the java byte[].  If you're using direct
>>>         ByteBuffer for FileChannel, that whole memcpy is skipped. 
>>>         Try comparing FileChannel with HeapByteBuffer instead.
>>>
>>>
>>>             On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>             On Thursday, October 27, 2016, Brunoais
>>>>             <brunoaiss at gmail.com
>>>>             <javascript:_e(%7B%7D,'cvml','brunoaiss at gmail.com');>>
>>>>             wrote:
>>>>
>>>>                 Did you read the C code?
>>>>
>>>>             I looked at the Linux code in the JDK.
>>>>
>>>>                 Have you got any idea how many functions Windows or
>>>>                 Linux (nearly all flavors) have for the read
>>>>                 operation towards a file?
>>>>
>>>>             I do.
>>>>
>>>>
>>>>                 I have already done that homework myself. I may not
>>>>                 have read JVM's source code but I know well that
>>>>                 there's functions on both Windows and Linux that
>>>>                 provide such interface I mentioned although they
>>>>                 require a slightly different treatment (and
>>>>                 different constants).
>>>>
>>>>             You should read the JDK (native) source code instead of
>>>>             guessing/assuming.  On Linux, it doesn't use aio
>>>>             facilities for files.  The kernel io scheduler may
>>>>             issue readahead behind the scenes, but there's no
>>>>             nonblocking file io that's at the heart of your premise.
>>>>
>>>>
>>>>
>>>>                 On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>
>>>>                     On Wednesday, October 26, 2016, Brunoais
>>>>                     <brunoaiss at gmail.com
>>>>                     <mailto:brunoaiss at gmail.com>> wrote:
>>>>
>>>>                         It is actually based on the premise that:
>>>>
>>>>                         1. The first call to
>>>>                     ReadableByteChannel.read(ByteBuffer) sets the OS
>>>>                            buffer size to fill in as the same size
>>>>                     as ByteBuffer.
>>>>
>>>>                     Why do you say that? AFAICT, it issues a read
>>>>                     syscall and that will block if the data isn't
>>>>                     in page cache.
>>>>
>>>>                         2. The consecutive calls to
>>>>                     ReadableByteChannel.read(ByteBuffer)
>>>>                         orders
>>>>                            the JVM to order the OS to execute
>>>>                     memcpy() to copy from its memory
>>>>                            to the shared memory created at
>>>>                     ByteBuffer instantiation (in
>>>>                         java 8)
>>>>                            using Unsafe and then for the JVM to
>>>>                     update the ByteBuffer fields.
>>>>
>>>>                     I think subsequent reads just invoke the same
>>>>                     read syscall, passing the current file offset
>>>>                     maintained by the file channel instance.
>>>>
>>>>                         3. The call will not block waiting for I/O
>>>>                     and it won't take longer
>>>>                            than the JNI interface if no new data
>>>>                     exists. However, it will
>>>>                         block
>>>>                            waiting for the OS to execute memcpy()
>>>>                     to the shared memory.
>>>>
>>>>                     So why do you think it won't block?
>>>>
>>>>
>>>>                         Is my premise wrong?
>>>>
>>>>                         If I read correctly, if I don't use a
>>>>                     DirectBuffer, there would be
>>>>                         even another intermediate buffer to copy
>>>>                     data to before giving it
>>>>                         to the "user" which would be useless.
>>>>
>>>>                     If you use a HeapByteBuffer, then there's an
>>>>                     extra copy from the native buffer to the Java
>>>>                     buffer.
>>>>
>>>>
>>>>
>>>>                         On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>
>>>>                             I believe I see where you coming from.
>>>>                     Please correct me if
>>>>                             I'm wrong.
>>>>
>>>>                             Your implementation is based on the
>>>>                     premise that a call to
>>>>                     ReadableByteChannel.read()
>>>>                             _initiates_ the operation and returns
>>>>                     immediately. The OS then
>>>>                             continues to fill
>>>>                             the buffer while there's a free space
>>>>                     in the buffer and the
>>>>                             channel hasn't encountered EOF.
>>>>
>>>>                             Is that right?
>>>>
>>>>                                 On 25 Oct 2016, at 22:16, Brunoais
>>>>                     <brunoaiss at gmail.com>
>>>>                                 wrote:
>>>>
>>>>                                 Thank you for your time. I'll try
>>>>                     to explain it. I hope I
>>>>                                 can clear it up.
>>>>                                 First of it, I made a meaning
>>>>                     mistake between asynchronous
>>>>                                 and non-blocking. This
>>>>                     implementation uses a non-blocking
>>>>                                 algorithm internally while
>>>>                     providing a blocking-like
>>>>                                 algorithm on the surface. It is
>>>>                     single-threaded and not
>>>>                                 multi-threaded where one thread
>>>>                     fetches data and blocks
>>>>                                 waiting and the other accumulates
>>>>                     it and provides to
>>>>                                 whichever wants it.
>>>>
>>>>                                 Second of it, I had made a mistake
>>>>                     of going after
>>>>                                 BufferedReader instead of going
>>>>                     after BufferedInputStream.
>>>>                                 If you want me to go after
>>>>                     BufferedReader it's ok but I
>>>>                                 only thought that going after
>>>>                     BufferedInputStream would be
>>>>                                 more generically useful than
>>>>                     BufferedReaderwhen I started
>>>>                                 the poc.
>>>>
>>>>                                 On to my code:
>>>>                                 Short answers:
>>>>                                         ? The sleep(int) exists
>>>>                     because I don't know how
>>>>                                 to wait until more data exists in
>>>>                     the buffer which is part
>>>>                                 of read()'s contract.
>>>>                                         ? The ByteBuffer gives a
>>>>                     buffer that is filled by
>>>>                                 the OS (what I believe Channels do)
>>>>                     instead of getting
>>>>                                 data only    by demand (what I
>>>>                     believe Streams do).
>>>>                                 Full answers:
>>>>                                 The blockingFill(boolean) method is
>>>>                     a method for a busy
>>>>                                 wait for a fill which is used
>>>>                     exclusively by the read()
>>>>                                 method. All other methods use the
>>>>                     version that does not
>>>>                                 sleep (fill(boolean)).
>>>>                     blockingFill(boolean)'s existance like that is only
>>>>                                 because the read() method must not
>>>>                     return unless either:
>>>>
>>>>                                         ? The stream ended.
>>>>                                         ? The next byte is ready
>>>>                     for reading.
>>>>                                 Additionally, statistically, that
>>>>                     while loop will rarely
>>>>                                 evaluate to true as reads are in
>>>>                     chunks so readPos will be
>>>>                                 behind writePos most of the time.
>>>>                                 I have no idea if an interrupt will
>>>>                     ever happen, to be
>>>>                                 honest. The main reasons why I'm
>>>>                     using a sleep is because
>>>>                                 I didn't want a hog onto the CPU in
>>>>                     a full thread usage
>>>>                                 busy wait and because I didn't find
>>>>                     any way of doing a
>>>>                                 thread sleep in order to wake up
>>>>                     later when the buffer
>>>>                                 managed by native code has more data.
>>>>                                 The Non-blocking part is managed by
>>>>                     the buffer the OS
>>>>                                 keeps filling most if not all the
>>>>                     time. That buffer is the
>>>>                                 field
>>>>
>>>>                                 ByteBuffer readBuffer
>>>>                                 That's the gaining part against the
>>>>                     plain old Buffered
>>>>                                 classes.
>>>>
>>>>
>>>>                                 Did that make sense to you? Feel
>>>>                     free to ask anything else
>>>>                                 you need.
>>>>
>>>>                                 On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>
>>>>                                     I've skimmed through the code
>>>>                     and I'm not sure I can
>>>>                                     see any asynchronicity
>>>>                                     (you were pointing at the lack
>>>>                     of it in BufferedReader).
>>>>                                     And the mechanics of this is
>>>>                     very puzzling to me, to
>>>>                                     be honest:
>>>>                                          void blockingFill(boolean
>>>>                     forced) throws
>>>>                     IOException {
>>>>                      fill(forced);
>>>>                      while (readPos == writePos) {
>>>>                        try {
>>>>                            Thread.sleep(100);
>>>>                        } catch (InterruptedException e) {
>>>>                            // An interrupt may mean more data is
>>>>                                     available
>>>>                        }
>>>>                        fill(forced);
>>>>                                              }
>>>>                                          }
>>>>                                     I thought you were suggesting
>>>>                     that we should utilize
>>>>                                     the tools which OS provides
>>>>                                     more efficiently. Instead we
>>>>                     have something that looks
>>>>                                     very similarly to a
>>>>                                     "busy loop" and... also who and
>>>>                     when is supposed to
>>>>                                     interrupt Thread.sleep()?
>>>>                                     Sorry, I'm not following. Could
>>>>                     you please explain how
>>>>                                     this is supposed to work?
>>>>
>>>>                                         On 24 Oct 2016, at 15:59,
>>>>                     Brunoais
>>>>                                         <brunoaiss at gmail.com>
>>>>                     wrote:
>>>>                     Attached and sending!
>>>>                                         On 24/10/2016 13:48, Pavel
>>>>                     Rappo wrote:
>>>>
>>>>                     Could you please send a new email on this list
>>>>                     with the source attached as a
>>>>                     text file?
>>>>
>>>>                       On 23 Oct 2016, at 19:14, Brunoais
>>>>                       <brunoaiss at gmail.com>
>>>>                         wrote:
>>>>                       Here's my poc/prototype:
>>>>
>>>>                     http://pastebin.com/WRpYWDJF
>>>>
>>>>                       I've implemented the bare minimum of the
>>>>                       class that follows the same contract of
>>>>                       BufferedReader while signaling all issues
>>>>                       I think it may have or has in comments.
>>>>                       I also wrote some javadoc to help guiding
>>>>                       through the class.
>>>>                       I could have used more fields from
>>>>                       BufferedReader but the names were so
>>>>                       minimalistic that were confusing me. I
>>>>                       intent to change them before sending this
>>>>                       to openJDK.
>>>>                       One of the major problems this has is long
>>>>                       overflowing. It is major because it is
>>>>                       hidden, it will be extremely rare and it
>>>>                       takes a really long time to reproduce.
>>>>                       There are different ways of dealing with
>>>>                       it. From just documenting to actually
>>>>                       making code that works with it.
>>>>                       I built a simple test code for it to have
>>>>                       some ideas about performance and correctness.
>>>>
>>>>                     http://pastebin.com/eh6LFgwT
>>>>
>>>>                       This doesn't do a through test if it is
>>>>                       actually working correctly but I see no
>>>>                       reason for it not working correctly after
>>>>                       fixing the 2 bugs that test found.
>>>>                       I'll also leave here some conclusions
>>>>                       about speed and resource consumption I found.
>>>>                       I made tests with default buffer sizes,
>>>>                       5000B 15_000B and 500_000B. I noticed
>>>>                       that, with my hardware, with the 1 530 000
>>>>                       000B file, I was getting around:
>>>>                       In all buffers and fake work: 10~15s speed
>>>>                       improvement ( from 90% HDD speed to 100%
>>>>                       HDD speed)
>>>>                       In all buffers and no fake work: 1~2s
>>>>                       speed improvement ( from 90% HDD speed to
>>>>                       100% HDD speed)
>>>>                       Changing the buffer size was giving
>>>>                       different reading speeds but both were
>>>>                       quite equal in how much they would change
>>>>                       when changing the buffer size.
>>>>                       Finally, I could always confirm that I/O
>>>>                       was always the slowest thing while this
>>>>                       code was running.
>>>>                       For the ones wondering about the file
>>>>                       size; it is both to avoid OS cache and to
>>>>                       make the reading at the main use-case
>>>>                       these objects are for (large streams of
>>>>                       bytes).
>>>>                       @Pavel, are you open for discussion now
>>>>                       ;)? Need anything else?
>>>>                       On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>
>>>>                           Just to append to my previous email.
>>>>                           BufferedReader wraps any Reader out there.
>>>>                           Not specifically FileReader. While
>>>>                           you're talking about the case of effective
>>>>                           reading from a file.
>>>>                           I guess there's one existing
>>>>                           possibility to provide exactly what
>>>>                           you need (as I
>>>>                           understand it) under this method:
>>>>                           /**
>>>>                             * Opens a file for reading,
>>>>                           returning a {@code BufferedReader} to
>>>>                           read text
>>>>                             * from the file in an efficient
>>>>                           manner...
>>>>                               ...
>>>>                             */
>>>>                     java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>                           It can return _anything_ as long as it
>>>>                           is a BufferedReader. We can do it, but it
>>>>                           needs to be investigated not only for
>>>>                           your favorite OS but for other OSes as
>>>>                           well. Feel free to prototype this and
>>>>                           we can discuss it on the list later.
>>>>                           Thanks,
>>>>                           -Pavel
>>>>
>>>>                               On 21 Oct 2016, at 18:56, Brunoais
>>>>                               <brunoaiss at gmail.com>
>>>>                                 wrote:
>>>>                               Pavel is right.
>>>>                               In reality, I was expecting such
>>>>                               BufferedReader to use only a
>>>>                               single buffer and have that Buffer
>>>>                               being filled asynchronously, not
>>>>                               in a different Thread.
>>>>                               Additionally, I don't have the
>>>>                               intention of having a larger
>>>>                               buffer than before unless stated
>>>>                               through the API (the constructor).
>>>>                               In my idea, internally, it is
>>>>                               supposed to use
>>>>                     java.nio.channels.AsynchronousFileChannel
>>>>                               or equivalent.
>>>>                               It does not prevent having two
>>>>                               buffers and I do not intent to
>>>>                               change BufferedReader itself. I'd
>>>>                               do an BufferedAsyncReader of sorts
>>>>                               (any name suggestion is welcome as
>>>>                               I'm an awful namer).
>>>>                               On 21/10/2016 18:38, Roger Riggs
>>>>                               wrote:
>>>>
>>>>                                   Hi Pavel,
>>>>                                   I think Brunoais asking for a
>>>>                                   double buffering scheme in
>>>>                                   which the implementation of
>>>>                                   BufferReader fills (a second
>>>>                                   buffer) in parallel with the
>>>>                                   application reading from the
>>>>                                   1st buffer
>>>>                                   and managing the swaps and
>>>>                                   async reads transparently.
>>>>                                   It would not change the API
>>>>                                   but would change the
>>>>                                   interactions between the
>>>>                                   buffered reader
>>>>                                   and the underlying stream.  It
>>>>                                   would also increase memory
>>>>                                   requirements and processing
>>>>                                   by introducing or using a
>>>>                                   separate thread and the
>>>>                                   necessary synchronization.
>>>>                                   Though I think the formal
>>>>                                   interface semantics could be
>>>>                                   maintained, I have doubts
>>>>                                   about compatibility and its
>>>>                                   unintended consequences on
>>>>                                   existing subclasses,
>>>>                                   applications and libraries.
>>>>                                   $.02, Roger
>>>>                                   On 10/21/16 1:22 PM, Pavel
>>>>                                   Rappo wrote:
>>>>
>>>>                                       Off the top of my head, I
>>>>                                       would say it's not
>>>>                                       possible to change the
>>>>                                       design of an
>>>>                     _extensible_ type that has
>>>>                                       been out there for 20 or
>>>>                                       so years. All these I/O
>>>>                                       streams from java.io
>>>>                     <http://java.io>
>>>>                                       <http://java.io> were
>>>>                                       designed for simple
>>>>                     synchronous use case.
>>>>                                       It's not that their design
>>>>                                       is flawed in some way,
>>>>                                       it's that they doesn't seem to
>>>>                                       suit your needs. Have you
>>>>                     considered using
>>>>                     java.nio.channels.AsynchronousFileChannel
>>>>                                       in your applications?
>>>>                                       -Pavel
>>>>
>>>>                                           On 21 Oct 2016, at
>>>>                     17:08, Brunoais
>>>>                                           <brunoaiss at gmail.com>
>>>>                     wrote:
>>>>                                           Any feedback on this?
>>>>                                           I'm really interested
>>>>                                           in implementing such
>>>>                     BufferedReader/BufferedStreamReader
>>>>                                           to allow speeding up
>>>>                                           my applications
>>>>                     without having to
>>>>                     think in an
>>>>                     asynchronous way or
>>>>                     multi-threading while
>>>>                     programming with it.
>>>>                     That's why I'm asking
>>>>                                           this here.
>>>>                                           On 13/10/2016 14:45,
>>>>                     Brunoais wrote:
>>>>
>>>>                     Hi,
>>>>                     I looked at
>>>>                     BufferedReader
>>>>                     source code for
>>>>                     java 9 long with
>>>>                     the source code of
>>>>                     the
>>>>                     channels/streams
>>>>                     used. I noticed
>>>>                     that, like in java
>>>>                     7, BufferedReader
>>>>                     does not use an
>>>>                     Async API to load
>>>>                     data from files,
>>>>                     instead, the data
>>>>                     loading is all
>>>>                     done synchronously
>>>>                     even when the OS
>>>>                     allows requesting
>>>>                     a file to be read
>>>>                     and getting a
>>>>                     warning later when
>>>>                     the file is
>>>>                     effectively read.
>>>>                     Why Is
>>>>                     BufferedReader not
>>>>                     async while
>>>>                     providing a sync API?
>>>>
>>>>                     <BufferedNonBlockStream.java><Tests.java>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                     -- 
>>>>                     Sent from my phone
>>>>
>>>>
>>>>
>>>>
>>>>             -- 
>>>>             Sent from my phone
>>>
>>>
>>
>>
>
>
>
> -- 
> Sent from my phone


From anubhav.meena at oracle.com  Fri Oct 28 07:12:53 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Fri, 28 Oct 2016 12:42:53 +0530
Subject: RFR: JDK-8167618: DateTimeFormatter.format() uses exceptions for flow
	control.
Message-ID: <93729659-EFD9-4834-BB99-5638659E336D@oracle.com>

Hi all,

Please review.
Bug Id :  https://bugs.openjdk.java.net/browse/JDK-8167618 <https://bugs.openjdk.java.net/browse/JDK-8163330>
Issue:  DateTimeFormatter.format() uses exceptions for flow control.
Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8163330/webrev.02/>
-- 
Thanks and Regards,
Anubhav


From anubhav.meena at oracle.com  Fri Oct 28 07:18:32 2016
From: anubhav.meena at oracle.com (Anubhav Meena)
Date: Fri, 28 Oct 2016 12:48:32 +0530
Subject: RFR: JDK-8167618: DateTimeFormatter.format() uses exceptions for flow
	control. 
Message-ID: <FD29C42C-F30F-42C8-877B-719E127B91F0@oracle.com>

Hi all,

Please review. Please ignore last mail as links not working properly there.

Bug Id : https://bugs.openjdk.java.net/browse/JDK-8167618 <https://bugs.openjdk.java.net/browse/JDK-8167618>
Issue:  DateTimeFormatter.format() uses exceptions for flow control.
Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/>
-- 
Thanks and Regards,
Anubhav


From brunoaiss at gmail.com  Fri Oct 28 08:16:46 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Fri, 28 Oct 2016 09:16:46 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<F93A5BF4-9C4C-4AD6-8027-F1A79BEE613D@oracle.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
	<f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>
Message-ID: <dc5b8f4a-720d-abf0-6aec-af04847a4c03@gmail.com>

I'll try going back to a previous version I worked on which used the 
java7's AsynchronousFileChannel and work from there. My small research 
shows it can also work with AsynchronousFileChannel mostly without changes.

For now, 1 question:
Is Thread.sleep() a possible way of dealing the block requirements of 
read()? Do I need to use LockSupport.park() or something like that?

I'll call back here when it is done.


On 27/10/2016 22:09, David Holmes wrote:
> You might try discussing on net-dev rather than core-libs-dev, to get 
> additional historical info related to the io and nio file APIs.
>
> David
>
> On 28/10/2016 5:08 AM, Brunoais wrote:
>> You are right. Even in windows it does not set the flags for async
>> reads. It seems like it is windows itself that does the decision to
>> buffer the contents based on its own heuristics.
>>
>> But... Why? Why won't it be? Why is there no API for it? How am I
>> getting 100% HDD use and faster times when I fake work to delay getting
>> more data and I only have a fluctuating 60-90% (always going up and
>> down) when I use an InputStream?
>> Is it related to how both classes cache and how frequently and how much
>> each one asks for data?
>>
>> I really would prefer not having to read the source code because it
>> takes a real long time T.T.
>>
>> I end up reinstating... And wondering...
>>
>> Why doesn't java provide a single-threaded non-block API for file reads
>> for all OS that support it? I simply cannot find that information no
>> matter how much I search on google, bing, duck duck go... Can any of you
>> point me to whomever knows?
>>
>> On 27/10/2016 14:11, Vitaly Davidovich wrote:
>>> I don't know about Windows specifically, but generally file systems
>>> across major OS's will implement readahead in their IO scheduler when
>>> they detect sequential scans.
>>>
>>> On Linux, you can also strace your test to confirm which syscalls are
>>> emitted (you should be seeing plain read()'s there, with
>>> FileInputStream and FileChannel).
>>>
>>> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>
>>>     Thanks for the heads up.
>>>
>>>     I'll try that later. These tests are still useful then. Meanwhile,
>>>     I'll end up also checking how FileChannel queries the OS on
>>>     windows. I'm getting 100% HDD reads... Could it be that the OS
>>>     reads the file ahead on its own?... Anyway, I'll look into it.
>>>     Thanks for the heads up.
>>>
>>>
>>>     On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>>>
>>>>
>>>>     On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>>>>     <mailto:brunoaiss at gmail.com>> wrote:
>>>>
>>>>         Oh... I see. In that case, it means something is terribly
>>>>         wrong. It can be my initial tests, though.
>>>>
>>>>         I'm testing on both linux and windows and I'm getting
>>>>         performance gains from using the FileChannel compared to
>>>>         using FileInputStream... The tests also make sense based on
>>>>         my predictions O_O...
>>>>
>>>>     FileInputStream requires copying native buffers holding the read
>>>>     data to the java byte[].  If you're using direct ByteBuffer for
>>>>     FileChannel, that whole memcpy is skipped.  Try comparing
>>>>     FileChannel with HeapByteBuffer instead.
>>>>
>>>>
>>>>         On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>>>
>>>>>
>>>>>         On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>>>>         <mailto:brunoaiss at gmail.com>> wrote:
>>>>>
>>>>>             Did you read the C code?
>>>>>
>>>>>         I looked at the Linux code in the JDK.
>>>>>
>>>>>             Have you got any idea how many functions Windows or
>>>>>             Linux (nearly all flavors) have for the read operation
>>>>>             towards a file?
>>>>>
>>>>>         I do.
>>>>>
>>>>>
>>>>>             I have already done that homework myself. I may not have
>>>>>             read JVM's source code but I know well that there's
>>>>>             functions on both Windows and Linux that provide such
>>>>>             interface I mentioned although they require a slightly
>>>>>             different treatment (and different constants).
>>>>>
>>>>>         You should read the JDK (native) source code instead of
>>>>>         guessing/assuming.  On Linux, it doesn't use aio facilities
>>>>>         for files.  The kernel io scheduler may issue readahead
>>>>>         behind the scenes, but there's no nonblocking file io that's
>>>>>         at the heart of your premise.
>>>>>
>>>>>
>>>>>
>>>>>             On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>>
>>>>>
>>>>>
>>>>>                 On Wednesday, October 26, 2016, Brunoais
>>>>>                 <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>>                 wrote:
>>>>>
>>>>>                     It is actually based on the premise that:
>>>>>
>>>>>                     1. The first call to
>>>>>                 ReadableByteChannel.read(ByteBuffer) sets the OS
>>>>>                        buffer size to fill in as the same size as
>>>>>                 ByteBuffer.
>>>>>
>>>>>                 Why do you say that? AFAICT, it issues a read
>>>>>                 syscall and that will block if the data isn't in
>>>>>                 page cache.
>>>>>
>>>>>                     2. The consecutive calls to
>>>>>                 ReadableByteChannel.read(ByteBuffer)
>>>>>                     orders
>>>>>                        the JVM to order the OS to execute memcpy()
>>>>>                 to copy from its memory
>>>>>                        to the shared memory created at ByteBuffer
>>>>>                 instantiation (in
>>>>>                     java 8)
>>>>>                        using Unsafe and then for the JVM to update
>>>>>                 the ByteBuffer fields.
>>>>>
>>>>>                 I think subsequent reads just invoke the same read
>>>>>                 syscall, passing the current file offset maintained
>>>>>                 by the file channel instance.
>>>>>
>>>>>                     3. The call will not block waiting for I/O and
>>>>>                 it won't take longer
>>>>>                        than the JNI interface if no new data exists.
>>>>>                 However, it will
>>>>>                     block
>>>>>                        waiting for the OS to execute memcpy() to the
>>>>>                 shared memory.
>>>>>
>>>>>                 So why do you think it won't block?
>>>>>
>>>>>
>>>>>                     Is my premise wrong?
>>>>>
>>>>>                     If I read correctly, if I don't use a
>>>>>                 DirectBuffer, there would be
>>>>>                     even another intermediate buffer to copy data to
>>>>>                 before giving it
>>>>>                     to the "user" which would be useless.
>>>>>
>>>>>                 If you use a HeapByteBuffer, then there's an extra
>>>>>                 copy from the native buffer to the Java buffer.
>>>>>
>>>>>
>>>>>
>>>>>                     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>>
>>>>>                         I believe I see where you coming from.
>>>>>                 Please correct me if
>>>>>                         I'm wrong.
>>>>>
>>>>>                         Your implementation is based on the premise
>>>>>                 that a call to
>>>>>                 ReadableByteChannel.read()
>>>>>                         _initiates_ the operation and returns
>>>>>                 immediately. The OS then
>>>>>                         continues to fill
>>>>>                         the buffer while there's a free space in the
>>>>>                 buffer and the
>>>>>                         channel hasn't encountered EOF.
>>>>>
>>>>>                         Is that right?
>>>>>
>>>>>                             On 25 Oct 2016, at 22:16, Brunoais
>>>>>                 <brunoaiss at gmail.com>
>>>>>                             wrote:
>>>>>
>>>>>                             Thank you for your time. I'll try to
>>>>>                 explain it. I hope I
>>>>>                             can clear it up.
>>>>>                             First of it, I made a meaning mistake
>>>>>                 between asynchronous
>>>>>                             and non-blocking. This implementation
>>>>>                 uses a non-blocking
>>>>>                             algorithm internally while providing a
>>>>>                 blocking-like
>>>>>                             algorithm on the surface. It is
>>>>>                 single-threaded and not
>>>>>                             multi-threaded where one thread fetches
>>>>>                 data and blocks
>>>>>                             waiting and the other accumulates it and
>>>>>                 provides to
>>>>>                             whichever wants it.
>>>>>
>>>>>                             Second of it, I had made a mistake of
>>>>>                 going after
>>>>>                             BufferedReader instead of going after
>>>>>                 BufferedInputStream.
>>>>>                             If you want me to go after
>>>>>                 BufferedReader it's ok but I
>>>>>                             only thought that going after
>>>>>                 BufferedInputStream would be
>>>>>                             more generically useful than
>>>>>                 BufferedReaderwhen I started
>>>>>                             the poc.
>>>>>
>>>>>                             On to my code:
>>>>>                             Short answers:
>>>>>                                     ? The sleep(int) exists because
>>>>>                 I don't know how
>>>>>                             to wait until more data exists in the
>>>>>                 buffer which is part
>>>>>                             of read()'s contract.
>>>>>                                     ? The ByteBuffer gives a buffer
>>>>>                 that is filled by
>>>>>                             the OS (what I believe Channels do)
>>>>>                 instead of getting
>>>>>                             data only         by demand (what I
>>>>>                 believe Streams do).
>>>>>                             Full answers:
>>>>>                             The blockingFill(boolean) method is a
>>>>>                 method for a busy
>>>>>                             wait for a fill which is used
>>>>>                 exclusively by the read()
>>>>>                             method. All other methods use the
>>>>>                 version that does not
>>>>>                             sleep (fill(boolean)).
>>>>>                 blockingFill(boolean)'s existance like that is only
>>>>>                             because the read() method must not
>>>>>                 return unless either:
>>>>>
>>>>>                                     ? The stream ended.
>>>>>                                     ? The next byte is ready for
>>>>>                 reading.
>>>>>                             Additionally, statistically, that while
>>>>>                 loop will rarely
>>>>>                             evaluate to true as reads are in chunks
>>>>>                 so readPos will be
>>>>>                             behind writePos most of the time.
>>>>>                             I have no idea if an interrupt will ever
>>>>>                 happen, to be
>>>>>                             honest. The main reasons why I'm using a
>>>>>                 sleep is because
>>>>>                             I didn't want a hog onto the CPU in a
>>>>>                 full thread usage
>>>>>                             busy wait and because I didn't find any
>>>>>                 way of doing a
>>>>>                             thread sleep in order to wake up later
>>>>>                 when the buffer
>>>>>                             managed by native code has more data.
>>>>>                             The Non-blocking part is managed by the
>>>>>                 buffer the OS
>>>>>                             keeps filling most if not all the time.
>>>>>                 That buffer is the
>>>>>                             field
>>>>>
>>>>>                             ByteBuffer readBuffer
>>>>>                             That's the gaining part against the
>>>>>                 plain old Buffered
>>>>>                             classes.
>>>>>
>>>>>
>>>>>                             Did that make sense to you? Feel free to
>>>>>                 ask anything else
>>>>>                             you need.
>>>>>
>>>>>                             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>>
>>>>>                                 I've skimmed through the code and
>>>>>                 I'm not sure I can
>>>>>                                 see any asynchronicity
>>>>>                                 (you were pointing at the lack of it
>>>>>                 in BufferedReader).
>>>>>                                 And the mechanics of this is very
>>>>>                 puzzling to me, to
>>>>>                                 be honest:
>>>>>                                      void blockingFill(boolean
>>>>>                 forced) throws
>>>>>                                 IOException {
>>>>>                  fill(forced);
>>>>>                                          while (readPos == 
>>>>> writePos) {
>>>>>                                              try {
>>>>>                  Thread.sleep(100);
>>>>>                                              } catch
>>>>>                 (InterruptedException e) {
>>>>>                  // An interrupt may mean more data is
>>>>>                                 available
>>>>>                                              }
>>>>>                  fill(forced);
>>>>>                                          }
>>>>>                                      }
>>>>>                                 I thought you were suggesting that
>>>>>                 we should utilize
>>>>>                                 the tools which OS provides
>>>>>                                 more efficiently. Instead we have
>>>>>                 something that looks
>>>>>                                 very similarly to a
>>>>>                                 "busy loop" and... also who and when
>>>>>                 is supposed to
>>>>>                                 interrupt Thread.sleep()?
>>>>>                                 Sorry, I'm not following. Could you
>>>>>                 please explain how
>>>>>                                 this is supposed to work?
>>>>>
>>>>>                                     On 24 Oct 2016, at 15:59, 
>>>>> Brunoais
>>>>> <brunoaiss at gmail.com>
>>>>>                                       wrote:
>>>>>                                     Attached and sending!
>>>>>                                     On 24/10/2016 13:48, Pavel Rappo
>>>>>                 wrote:
>>>>>
>>>>>                                         Could you please send a new
>>>>>                 email on this list
>>>>>                                         with the source attached as a
>>>>>                                         text file?
>>>>>
>>>>>                                             On 23 Oct 2016, at
>>>>>                 19:14, Brunoais
>>>>> <brunoaiss at gmail.com>
>>>>>                 wrote:
>>>>>                 Here's my poc/prototype:
>>>>>
>>>>>                 http://pastebin.com/WRpYWDJF
>>>>>
>>>>>                                             I've implemented the
>>>>>                 bare minimum of the
>>>>>                 class that follows the same contract of
>>>>>                 BufferedReader while signaling all issues
>>>>>                                             I think it may have or
>>>>>                 has in comments.
>>>>>                                             I also wrote some
>>>>>                 javadoc to help guiding
>>>>>                 through the class.
>>>>>                                             I could have used more
>>>>>                 fields from
>>>>>                 BufferedReader but the names were so
>>>>>                 minimalistic that were confusing me. I
>>>>>                 intent to change them before sending this
>>>>>                                             to openJDK.
>>>>>                                             One of the major
>>>>>                 problems this has is long
>>>>>                 overflowing. It is major because it is
>>>>>                 hidden, it will be extremely rare and it
>>>>>                 takes a really long time to reproduce.
>>>>>                 There are different ways of dealing with
>>>>>                                             it. From just
>>>>>                 documenting to actually
>>>>>                 making code that works with it.
>>>>>                                             I built a simple test
>>>>>                 code for it to have
>>>>>                                             some ideas about
>>>>>                 performance and correctness.
>>>>>
>>>>>                 http://pastebin.com/eh6LFgwT
>>>>>
>>>>>                                             This doesn't do a
>>>>>                 through test if it is
>>>>>                 actually working correctly but I see no
>>>>>                 reason for it not working correctly after
>>>>>                 fixing the 2 bugs that test found.
>>>>>                                             I'll also leave here
>>>>>                 some conclusions
>>>>>                 about speed and resource consumption I found.
>>>>>                                             I made tests with
>>>>>                 default buffer sizes,
>>>>>                 5000B 15_000B and 500_000B. I noticed
>>>>>                 that, with my hardware, with the 1 530 000
>>>>>                                             000B file, I was getting
>>>>>                 around:
>>>>>                                             In all buffers and fake
>>>>>                 work: 10~15s speed
>>>>>                 improvement ( from 90% HDD speed to 100%
>>>>>                                             HDD speed)
>>>>>                                             In all buffers and no
>>>>>                 fake work: 1~2s
>>>>>                 speed improvement ( from 90% HDD speed to
>>>>>                                             100% HDD speed)
>>>>>                 Changing the buffer size was giving
>>>>>                 different reading speeds but both were
>>>>>                 quite equal in how much they would change
>>>>>                                             when changing the buffer
>>>>>                 size.
>>>>>                 Finally, I could always confirm that I/O
>>>>>                                             was always the slowest
>>>>>                 thing while this
>>>>>                                             code was running.
>>>>>                                             For the ones wondering
>>>>>                 about the file
>>>>>                 size; it is both to avoid OS cache and to
>>>>>                                             make the reading at the
>>>>>                 main use-case
>>>>>                 these objects are for (large streams of
>>>>>                 bytes).
>>>>>                 @Pavel, are you open for discussion now
>>>>>                                             ;)? Need anything else?
>>>>>                                             On 21/10/2016 19:21,
>>>>>                 Pavel Rappo wrote:
>>>>>
>>>>>                 Just to append to my previous email.
>>>>>                 BufferedReader wraps any Reader out there.
>>>>>                 Not specifically FileReader. While
>>>>>                 you're talking about the case of effective
>>>>>                 reading from a file.
>>>>>                 I guess there's one existing
>>>>>                 possibility to provide exactly what
>>>>>                 you need (as I
>>>>>                 understand it) under this method:
>>>>>                 /**
>>>>>                   * Opens a file for reading,
>>>>>                 returning a {@code BufferedReader} to
>>>>>                 read text
>>>>>                   * from the file in an efficient
>>>>>                 manner...
>>>>>                     ...
>>>>>                   */
>>>>>
>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>                 It can return _anything_ as long as it
>>>>>                 is a BufferedReader. We can do it, but it
>>>>>                 needs to be investigated not only for
>>>>>                 your favorite OS but for other OSes as
>>>>>                 well. Feel free to prototype this and
>>>>>                 we can discuss it on the list later.
>>>>>                 Thanks,
>>>>>                 -Pavel
>>>>>
>>>>>                     On 21 Oct 2016, at 18:56, Brunoais
>>>>>                     <brunoaiss at gmail.com>
>>>>>                       wrote:
>>>>>                     Pavel is right.
>>>>>                     In reality, I was expecting such
>>>>>                     BufferedReader to use only a
>>>>>                     single buffer and have that Buffer
>>>>>                     being filled asynchronously, not
>>>>>                     in a different Thread.
>>>>>                     Additionally, I don't have the
>>>>>                     intention of having a larger
>>>>>                     buffer than before unless stated
>>>>>                     through the API (the constructor).
>>>>>                     In my idea, internally, it is
>>>>>                     supposed to use
>>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>>                     or equivalent.
>>>>>                     It does not prevent having two
>>>>>                     buffers and I do not intent to
>>>>>                     change BufferedReader itself. I'd
>>>>>                     do an BufferedAsyncReader of sorts
>>>>>                     (any name suggestion is welcome as
>>>>>                     I'm an awful namer).
>>>>>                     On 21/10/2016 18:38, Roger Riggs
>>>>>                     wrote:
>>>>>
>>>>>                         Hi Pavel,
>>>>>                         I think Brunoais asking for a
>>>>>                         double buffering scheme in
>>>>>                         which the implementation of
>>>>>                         BufferReader fills (a second
>>>>>                         buffer) in parallel with the
>>>>>                         application reading from the
>>>>>                         1st buffer
>>>>>                         and managing the swaps and
>>>>>                         async reads transparently.
>>>>>                         It would not change the API
>>>>>                         but would change the
>>>>>                         interactions between the
>>>>>                         buffered reader
>>>>>                         and the underlying stream.  It
>>>>>                         would also increase memory
>>>>>                         requirements and processing
>>>>>                         by introducing or using a
>>>>>                         separate thread and the
>>>>>                         necessary synchronization.
>>>>>                         Though I think the formal
>>>>>                         interface semantics could be
>>>>>                         maintained, I have doubts
>>>>>                         about compatibility and its
>>>>>                         unintended consequences on
>>>>>                         existing subclasses,
>>>>>                         applications and libraries.
>>>>>                         $.02, Roger
>>>>>                         On 10/21/16 1:22 PM, Pavel
>>>>>                         Rappo wrote:
>>>>>
>>>>>                             Off the top of my head, I
>>>>>                             would say it's not
>>>>>                             possible to change the
>>>>>                             design of an
>>>>>                             _extensible_ type that has
>>>>>                             been out there for 20 or
>>>>>                             so years. All these I/O
>>>>>                             streams from java.io <http://java.io>
>>>>>                             <http://java.io> were
>>>>>                             designed for simple
>>>>>                             synchronous use case.
>>>>>                             It's not that their design
>>>>>                             is flawed in some way,
>>>>>                             it's that they doesn't seem to
>>>>>                             suit your needs. Have you
>>>>>                             considered using
>>>>>                 java.nio.channels.AsynchronousFileChannel
>>>>>                             in your applications?
>>>>>                             -Pavel
>>>>>
>>>>>                                 On 21 Oct 2016, at
>>>>>                                 17:08, Brunoais
>>>>> <brunoaiss at gmail.com>
>>>>>                                   wrote:
>>>>>                                 Any feedback on this?
>>>>>                                 I'm really interested
>>>>>                                 in implementing such
>>>>>                 BufferedReader/BufferedStreamReader
>>>>>                                 to allow speeding up
>>>>>                                 my applications
>>>>>                                 without having to
>>>>>                                 think in an
>>>>>                                 asynchronous way or
>>>>>                                 multi-threading while
>>>>>                                 programming with it.
>>>>>                                 That's why I'm asking
>>>>>                                 this here.
>>>>>                                 On 13/10/2016 14:45,
>>>>>                                 Brunoais wrote:
>>>>>
>>>>>                                     Hi,
>>>>>                                     I looked at
>>>>>                 BufferedReader
>>>>>                                     source code for
>>>>>                                     java 9 long with
>>>>>                                     the source code of
>>>>>                                     the
>>>>>                 channels/streams
>>>>>                                     used. I noticed
>>>>>                                     that, like in java
>>>>>                                     7, BufferedReader
>>>>>                                     does not use an
>>>>>                                     Async API to load
>>>>>                                     data from files,
>>>>>                                     instead, the data
>>>>>                                     loading is all
>>>>>                                     done synchronously
>>>>>                                     even when the OS
>>>>>                                     allows requesting
>>>>>                                     a file to be read
>>>>>                                     and getting a
>>>>>                                     warning later when
>>>>>                                     the file is
>>>>>                                     effectively read.
>>>>>                                     Why Is
>>>>>                 BufferedReader not
>>>>>                                     async while
>>>>>                                     providing a sync API?
>>>>>
>>>>> <BufferedNonBlockStream.java><Tests.java>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                 --                 Sent from my phone
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>         --         Sent from my phone
>>>>
>>>>
>>>
>>>
>>
>


From daniel.fuchs at oracle.com  Fri Oct 28 10:03:47 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Fri, 28 Oct 2016 11:03:47 +0100
Subject: Review Request: JDK-8167057 jdeps to list the modules and
	internal APIs to help find @modules for tests
In-Reply-To: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
References: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
Message-ID: <09cd4286-abbe-3411-f392-051a12cd75de@oracle.com>

Hi Mandy,

Looks good to me in general, but I feel like the new
option --list-reduced-deps should be better documented:

jdeps.properties:

  152 main.opt.list-deps=\
  153 \  --list-deps\n\
  154 \  --list-reduced-deps           Lists the dependences and use of 
JDK internal\n\
  155 \                                APIs. --list-reduced-deps lists 
the dependences\n\
  156 \                                after transition reduction.

#1 - is it 'transition reduction' or 'transitive reduction'?
      (at two places in this file: line 156 & line 135)

#2 - could the description be made a little more verbose? something
      like:

--list-reduced-deps                   lists the dependences
                                       after transitive reduction.
                                       Transitive reduction is obtained
                                       by removing the dependencies which
                                       are already transitively exported
                                       by another module in the
                                       dependency graph. For instance,
                                       if both java.sql and java.logging
                                       are included in the dependency
                                       graph, then java.logging will be
                                       removed because it is already
                                       transitively exported by
                                       java.sql, and therefore
                                       requiring java.sql should be
                                       enough.


best regards,

-- daniel


On 19/10/16 23:19, Mandy Chung wrote:
> Webrev at:
>   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167057/webrev.00/
>
> This patch enhances jdeps to print the dependences in the format : $MODULE[/$PACKAGE].
>
> This is intended for analyzing the regression tests we develop and add make it easy to add the proper @modules.
>
> Mandy
>


From daniel.fuchs at oracle.com  Fri Oct 28 11:51:45 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Fri, 28 Oct 2016 12:51:45 +0100
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
Message-ID: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>

Hi,

Please find below a trivial  patch for:

8152515: (logging) LogManager.resetLogger should ignore LinkageError
https://bugs.openjdk.java.net/browse/JDK-8152515


Patch:
http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/

The issue might occur at shutdown, when a handler that makes uses
of some APIs provided by an OSGI bundle which was already closed
by the shutdown process is in turn closed by the LogManager.Cleaner
thread. In that case some subclasses of LinkageError may be thrown,
interrupting the reset process and preventing other handlers from
being closed properly.

The patch proposes to trivially ignore LinkageError at shutdown while
the LogManager.Cleaner thread is running.

best regards,

-- daniel

From Roger.Riggs at Oracle.com  Fri Oct 28 13:50:14 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 09:50:14 -0400
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
Message-ID: <15b839a1-a3e5-56f8-5bb2-2d43e13486a6@Oracle.com>

Hi Daniel,

Looks fine.

Roger


On 10/28/2016 7:51 AM, Daniel Fuchs wrote:
> Hi,
>
> Please find below a trivial  patch for:
>
> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
> https://bugs.openjdk.java.net/browse/JDK-8152515
>
>
> Patch:
> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>
> The issue might occur at shutdown, when a handler that makes uses
> of some APIs provided by an OSGI bundle which was already closed
> by the shutdown process is in turn closed by the LogManager.Cleaner
> thread. In that case some subclasses of LinkageError may be thrown,
> interrupting the reset process and preventing other handlers from
> being closed properly.
>
> The patch proposes to trivially ignore LinkageError at shutdown while
> the LogManager.Cleaner thread is running.
>
> best regards,
>
> -- daniel


From Roger.Riggs at Oracle.com  Fri Oct 28 14:10:33 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 10:10:33 -0400
Subject: JDK 9 RFR of JDK-8168524: Remove two jdk_nio tests from
	ProblemList: BashStreams and DeleteInterference.java
In-Reply-To: <ed918ec3-c87c-341d-49e6-f4b4356f05a2@oracle.com>
References: <aaac152a-4b2a-4d1b-e05b-046a8b87202d@oracle.com>
	<FF5D6A00-DC71-4D85-AFC7-40A3D02F91C5@oracle.com>
	<ed918ec3-c87c-341d-49e6-f4b4356f05a2@oracle.com>
Message-ID: <84c14185-0e86-d8e4-6d44-75e23219549a@Oracle.com>

+1
(for BashStreams test)

Since this is restoring tests to gather more data.

Roger


On 10/27/2016 11:15 PM, Amy Lu wrote:
> On 10/24/16 10:50 PM, Brian Burkhalter wrote:
>> +1 for the DeleteInterference portion.
> Thank you Brian.
>
> Still need a reviewer for BashStreams test.
>
> Thanks,
> Amy
>
>>
>> Brian
>>
>> On Oct 24, 2016, at 1:08 AM, Amy Lu <amy.lu at oracle.com> wrote:
>>
>>> Please review the patch to bring back two tests:
>>>
>>> java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
>>>     More debug info has been added to this test (JDK-8162902). It's 
>>> hard to reproduce the failure. Let's bring back this test. Once test 
>>> failure happen again, the debug info could help with JDK-8156511 fix.
>>>
>>> java/nio/charset/coders/BashStreams.java 8149712 generic-all
>>>     Mentioned issue seems does not exist anymore, suggest to bring 
>>> back the test.
>>>
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8168524
>>>
>>> Thanks,
>>> Amy
>>>
>>> --- old/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
>>> +++ new/test/ProblemList.txt    2016-10-24 15:50:14.000000000 +0800
>>> @@ -186,10 +186,6 @@
>>> java/nio/file/WatchService/MayFlies.java 7158947 solaris-all Solaris 11
>>> java/nio/file/WatchService/LotsOfEvents.java 7158947 solaris-all 
>>> Solaris 11
>>>
>>> -java/nio/charset/coders/BashStreams.java 8149712 generic-all
>>> -
>>> -java/nio/file/WatchService/DeleteInterference.java 8156511 linux-all
>>> -
>>> ############################################################################ 
>>>
>>>
>>> # jdk_rmi
>>>
>>>
>


From daniel.fuchs at oracle.com  Fri Oct 28 14:44:03 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Fri, 28 Oct 2016 15:44:03 +0100
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <15b839a1-a3e5-56f8-5bb2-2d43e13486a6@Oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<15b839a1-a3e5-56f8-5bb2-2d43e13486a6@Oracle.com>
Message-ID: <4d9c8b60-930d-8232-f3a6-eceae318ba04@oracle.com>

On 28/10/16 14:50, Roger Riggs wrote:
> Hi Daniel,
>
> Looks fine.
>
> Roger

Thanks Roger.

I have turned the reproducer attached to JDK-8152515 into a
jtreg test. The test creates some logger and registers a handler
that throws LinkageError when closed.

The test passes as expected with the fix.
Without the fix however, it makes jtreg fail in timeout
(which is a bit strange):

----------System.out:(2/193)----------
[LinkageErrorTest$TestHandler at 3a7f70{closed=false}, 
LinkageErrorTest$TestHandler at 25da7d46{closed=false}, 
LinkageErrorTest$TestHandler at 6c57d74f{closed=true}]
Timeout signalled after 120 seconds
----------System.err:(9/621)----------
STATUS:Passed.
java.lang.LinkageError
	at LinkageErrorTest$TestHandler.close(LinkageErrorTest.java:63)
	at 
java.util.logging.LogManager.closeHandlers(java.logging at 9-internal/LogManager.java:1449)
	at 
java.util.logging.LogManager.resetLogger(java.logging at 9-internal/LogManager.java:1459)
	at 
java.util.logging.LogManager.resetLoggerContext(java.logging at 9-internal/LogManager.java:1439)
	at 
java.util.logging.LogManager.reset(java.logging at 9-internal/LogManager.java:1424)
	at 
java.util.logging.LogManager$Cleaner.run(java.logging at 9-internal/LogManager.java:284)
STATUS:Failed.`main' threw exception: java.lang.LinkageError
----------rerun:(21/1847)*----------

I am not sure whether I should push this test or not...

What do you think? Is that behavior going to stay stable?
I mean - can we rely on the fact that jtreg will always
fail this test if the fix is not present?

Here is the webrev with the test:
http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.01

-- daniel

>
>
> On 10/28/2016 7:51 AM, Daniel Fuchs wrote:
>> Hi,
>>
>> Please find below a trivial  patch for:
>>
>> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
>> https://bugs.openjdk.java.net/browse/JDK-8152515
>>
>>
>> Patch:
>> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>>
>> The issue might occur at shutdown, when a handler that makes uses
>> of some APIs provided by an OSGI bundle which was already closed
>> by the shutdown process is in turn closed by the LogManager.Cleaner
>> thread. In that case some subclasses of LinkageError may be thrown,
>> interrupting the reset process and preventing other handlers from
>> being closed properly.
>>
>> The patch proposes to trivially ignore LinkageError at shutdown while
>> the LogManager.Cleaner thread is running.
>>
>> best regards,
>>
>> -- daniel
>


From mandy.chung at oracle.com  Fri Oct 28 15:30:16 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 28 Oct 2016 08:30:16 -0700
Subject: Review Request: JDK-8167057 jdeps to list the modules and
	internal APIs to help find @modules for tests
In-Reply-To: <09cd4286-abbe-3411-f392-051a12cd75de@oracle.com>
References: <AE4EC68D-AE7A-426E-B76B-53F67ACF2DE8@oracle.com>
	<09cd4286-abbe-3411-f392-051a12cd75de@oracle.com>
Message-ID: <E7D77AD6-D071-4692-9475-DBC4ECD14EDB@oracle.com>


> On Oct 28, 2016, at 3:03 AM, Daniel Fuchs <daniel.fuchs at oracle.com> wrote:
> 
> Hi Mandy,
> 
> Looks good to me in general, but I feel like the new
> option --list-reduced-deps should be better documented:
> 

I agree and this details and example can be covered in the man page.

> jdeps.properties:
> 
> 152 main.opt.list-deps=\
> 153 \  --list-deps\n\
> 154 \  --list-reduced-deps           Lists the dependences and use of JDK internal\n\
> 155 \                                APIs. --list-reduced-deps lists the dependences\n\
> 156 \                                after transition reduction.
> 
> #1 - is it 'transition reduction' or 'transitive reduction'?
>     (at two places in this file: line 156 & line 135)
> 

Good catch.  typo should be ?transitive? and will fix it.
  
> #2 - could the description be made a little more verbose? something
>     like:
> 
> --list-reduced-deps                   lists the dependences
>                                      after transitive reduction.
>                                      Transitive reduction is obtained
>                                      by removing the dependencies which
>                                      are already transitively exported
>                                      by another module in the
>                                      dependency graph. For instance,
>                                      if both java.sql and java.logging
>                                      are included in the dependency
>                                      graph, then java.logging will be
>                                      removed because it is already
>                                      transitively exported by
>                                      java.sql, and therefore
>                                      requiring java.sql should be
>                                      enough.

Good idea.  I will expand the description.

Mandy

> 
> 
> best regards,
> 
> -- daniel
> 
> 
> On 19/10/16 23:19, Mandy Chung wrote:
>> Webrev at:
>>  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8167057/webrev.00/
>> 
>> This patch enhances jdeps to print the dependences in the format : $MODULE[/$PACKAGE].
>> 
>> This is intended for analyzing the regression tests we develop and add make it easy to add the proper @modules.
>> 
>> Mandy
>> 
> 


From jason_mehrens at hotmail.com  Fri Oct 28 15:36:25 2016
From: jason_mehrens at hotmail.com (Jason Mehrens)
Date: Fri, 28 Oct 2016 15:36:25 +0000
Subject: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
Message-ID: <DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>

Daniel,

Looks good to me.

Thanks for fixing this!

Jason

________________________________________
From: Daniel Fuchs <daniel.fuchs at oracle.com>
Sent: Friday, October 28, 2016 6:51 AM
To: core-libs-dev
Cc: Jason Mehrens
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore LinkageError

Hi,

Please find below a trivial  patch for:

8152515: (logging) LogManager.resetLogger should ignore LinkageError
https://bugs.openjdk.java.net/browse/JDK-8152515


Patch:
http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/

The issue might occur at shutdown, when a handler that makes uses
of some APIs provided by an OSGI bundle which was already closed
by the shutdown process is in turn closed by the LogManager.Cleaner
thread. In that case some subclasses of LinkageError may be thrown,
interrupting the reset process and preventing other handlers from
being closed properly.

The patch proposes to trivially ignore LinkageError at shutdown while
the LogManager.Cleaner thread is running.

best regards,

-- daniel

From nadeesh.tv at oracle.com  Fri Oct 28 15:58:10 2016
From: nadeesh.tv at oracle.com (nadeesh tv)
Date: Fri, 28 Oct 2016 08:58:10 -0700
Subject: RFR: JDK-8167618: DateTimeFormatter.format() uses exceptions
	for flow control.
In-Reply-To: <FD29C42C-F30F-42C8-877B-719E127B91F0@oracle.com>
References: <FD29C42C-F30F-42C8-877B-719E127B91F0@oracle.com>
Message-ID: <58137592.1000207@oracle.com>


Hi Anubhav,


-     * @throws DateTimeException if the field is not available and the section is not optional


I think you should not remove the DTException since still there is a 
chance of throwing DTE
Regards,
Nadeesh
On 10/28/2016 12:18 AM, Anubhav Meena wrote:
> Hi all,
>
> Please review. Please ignore last mail as links not working properly there.
>
> Bug Id : https://bugs.openjdk.java.net/browse/JDK-8167618 <https://bugs.openjdk.java.net/browse/JDK-8167618>
> Issue:  DateTimeFormatter.format() uses exceptions for flow control.
> Webrev: http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/ <http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/>

-- 
Thanks and Regards,
Nadeesh TV


From jbluettduncan at gmail.com  Fri Oct 28 16:00:34 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Fri, 28 Oct 2016 17:00:34 +0100
Subject: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>
Message-ID: <CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>

I've an awful suspicion that the `catch (LinkageError e)` block is
unreachable, as the `catch (Exception e)` block would run first, being
located above the other block in the source code.

Is my suspicion correct?

Kind regards,
Jonathan

On 28 October 2016 at 16:36, Jason Mehrens <jason_mehrens at hotmail.com>
wrote:

> Daniel,
>
> Looks good to me.
>
> Thanks for fixing this!
>
> Jason
>
> ________________________________________
> From: Daniel Fuchs <daniel.fuchs at oracle.com>
> Sent: Friday, October 28, 2016 6:51 AM
> To: core-libs-dev
> Cc: Jason Mehrens
> Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
> LinkageError
>
> Hi,
>
> Please find below a trivial  patch for:
>
> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
> https://bugs.openjdk.java.net/browse/JDK-8152515
>
>
> Patch:
> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>
> The issue might occur at shutdown, when a handler that makes uses
> of some APIs provided by an OSGI bundle which was already closed
> by the shutdown process is in turn closed by the LogManager.Cleaner
> thread. In that case some subclasses of LinkageError may be thrown,
> interrupting the reset process and preventing other handlers from
> being closed properly.
>
> The patch proposes to trivially ignore LinkageError at shutdown while
> the LogManager.Cleaner thread is running.
>
> best regards,
>
> -- daniel
>

From Roger.Riggs at Oracle.com  Fri Oct 28 16:15:31 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 12:15:31 -0400
Subject: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>
	<CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>
Message-ID: <e783a94e-16f5-50d0-4aa2-db484ed930c6@Oracle.com>

Hi Jonathan,

There is no issue in this case.
LinkageError does not extend Exception so they are disjoint at the catch 
clauses.
And the compiler produces an error if a catch clause hides another 
exception to keep
the mistake from being hidden.

$.02, Roger


On 10/28/2016 12:00 PM, Jonathan Bluett-Duncan wrote:
> I've an awful suspicion that the `catch (LinkageError e)` block is
> unreachable, as the `catch (Exception e)` block would run first, being
> located above the other block in the source code.
>
> Is my suspicion correct?
>
> Kind regards,
> Jonathan
>
> On 28 October 2016 at 16:36, Jason Mehrens <jason_mehrens at hotmail.com>
> wrote:
>
>> Daniel,
>>
>> Looks good to me.
>>
>> Thanks for fixing this!
>>
>> Jason
>>
>> ________________________________________
>> From: Daniel Fuchs <daniel.fuchs at oracle.com>
>> Sent: Friday, October 28, 2016 6:51 AM
>> To: core-libs-dev
>> Cc: Jason Mehrens
>> Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
>> LinkageError
>>
>> Hi,
>>
>> Please find below a trivial  patch for:
>>
>> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
>> https://bugs.openjdk.java.net/browse/JDK-8152515
>>
>>
>> Patch:
>> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>>
>> The issue might occur at shutdown, when a handler that makes uses
>> of some APIs provided by an OSGI bundle which was already closed
>> by the shutdown process is in turn closed by the LogManager.Cleaner
>> thread. In that case some subclasses of LinkageError may be thrown,
>> interrupting the reset process and preventing other handlers from
>> being closed properly.
>>
>> The patch proposes to trivially ignore LinkageError at shutdown while
>> the LogManager.Cleaner thread is running.
>>
>> best regards,
>>
>> -- daniel
>>


From daniel.fuchs at oracle.com  Fri Oct 28 16:15:27 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Fri, 28 Oct 2016 17:15:27 +0100
Subject: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>
	<CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>
Message-ID: <2dcc0942-0278-467e-deaa-8e2c86de0c57@oracle.com>

Hi Jonathan,

On 28/10/16 17:00, Jonathan Bluett-Duncan wrote:
> I've an awful suspicion that the `catch (LinkageError e)` block is
> unreachable, as the `catch (Exception e)` block would run first, being
> located above the other block in the source code.
>
> Is my suspicion correct?

Not really. As its name indicate, LinkageError is an Error, not a
subclass of Exception.

best regards,

-- daniel

>
> Kind regards,
> Jonathan
>
> On 28 October 2016 at 16:36, Jason Mehrens <jason_mehrens at hotmail.com
> <mailto:jason_mehrens at hotmail.com>> wrote:
>
>     Daniel,
>
>     Looks good to me.
>
>     Thanks for fixing this!
>
>     Jason
>
>     ________________________________________
>     From: Daniel Fuchs <daniel.fuchs at oracle.com
>     <mailto:daniel.fuchs at oracle.com>>
>     Sent: Friday, October 28, 2016 6:51 AM
>     To: core-libs-dev
>     Cc: Jason Mehrens
>     Subject: RFR: 8152515: (logging) LogManager.resetLogger should
>     ignore LinkageError
>
>     Hi,
>
>     Please find below a trivial  patch for:
>
>     8152515: (logging) LogManager.resetLogger should ignore LinkageError
>     https://bugs.openjdk.java.net/browse/JDK-8152515
>     <https://bugs.openjdk.java.net/browse/JDK-8152515>
>
>
>     Patch:
>     http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>     <http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/>
>
>     The issue might occur at shutdown, when a handler that makes uses
>     of some APIs provided by an OSGI bundle which was already closed
>     by the shutdown process is in turn closed by the LogManager.Cleaner
>     thread. In that case some subclasses of LinkageError may be thrown,
>     interrupting the reset process and preventing other handlers from
>     being closed properly.
>
>     The patch proposes to trivially ignore LinkageError at shutdown while
>     the LogManager.Cleaner thread is running.
>
>     best regards,
>
>     -- daniel
>
>


From jbluettduncan at gmail.com  Fri Oct 28 16:21:30 2016
From: jbluettduncan at gmail.com (Jonathan Bluett-Duncan)
Date: Fri, 28 Oct 2016 17:21:30 +0100
Subject: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <e783a94e-16f5-50d0-4aa2-db484ed930c6@Oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<DM2PR07MB3049254F9EFDDE704BEB6F483AD0@DM2PR07MB304.namprd07.prod.outlook.com>
	<CAPMys84EoGUbp5Fh0cEB_t6-frDmnLxBsU_us5xF588hnVr=3Q@mail.gmail.com>
	<e783a94e-16f5-50d0-4aa2-db484ed930c6@Oracle.com>
Message-ID: <CAPMys87+kTXUOqc0ti4s1cik-FL6uQ-NJkzaVXuw0P55g0Y-gQ@mail.gmail.com>

Oh, I see! Thanks for pointing out my misconception for me. :)

In that case, this fix looks fine to me as a non-reviewer.

Kind regards,
Jonathan

On 28 October 2016 at 17:15, Roger Riggs <Roger.Riggs at oracle.com> wrote:

> Hi Jonathan,
>
> There is no issue in this case.
> LinkageError does not extend Exception so they are disjoint at the catch
> clauses.
> And the compiler produces an error if a catch clause hides another
> exception to keep
> the mistake from being hidden.
>
> $.02, Roger
>
>
>
> On 10/28/2016 12:00 PM, Jonathan Bluett-Duncan wrote:
>
>> I've an awful suspicion that the `catch (LinkageError e)` block is
>> unreachable, as the `catch (Exception e)` block would run first, being
>> located above the other block in the source code.
>>
>> Is my suspicion correct?
>>
>> Kind regards,
>> Jonathan
>>
>> On 28 October 2016 at 16:36, Jason Mehrens <jason_mehrens at hotmail.com>
>> wrote:
>>
>> Daniel,
>>>
>>> Looks good to me.
>>>
>>> Thanks for fixing this!
>>>
>>> Jason
>>>
>>> ________________________________________
>>> From: Daniel Fuchs <daniel.fuchs at oracle.com>
>>> Sent: Friday, October 28, 2016 6:51 AM
>>> To: core-libs-dev
>>> Cc: Jason Mehrens
>>> Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
>>> LinkageError
>>>
>>> Hi,
>>>
>>> Please find below a trivial  patch for:
>>>
>>> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
>>> https://bugs.openjdk.java.net/browse/JDK-8152515
>>>
>>>
>>> Patch:
>>> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>>>
>>> The issue might occur at shutdown, when a handler that makes uses
>>> of some APIs provided by an OSGI bundle which was already closed
>>> by the shutdown process is in turn closed by the LogManager.Cleaner
>>> thread. In that case some subclasses of LinkageError may be thrown,
>>> interrupting the reset process and preventing other handlers from
>>> being closed properly.
>>>
>>> The patch proposes to trivially ignore LinkageError at shutdown while
>>> the LogManager.Cleaner thread is running.
>>>
>>> best regards,
>>>
>>> -- daniel
>>>
>>>
>

From Roger.Riggs at Oracle.com  Fri Oct 28 16:25:47 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 12:25:47 -0400
Subject: RFR: JDK-8167618: DateTimeFormatter.format() uses exceptions for
	flow control.
In-Reply-To: <58137592.1000207@oracle.com>
References: <FD29C42C-F30F-42C8-877B-719E127B91F0@oracle.com>
	<58137592.1000207@oracle.com>
Message-ID: <13098760-c98f-9da5-d04c-1fd6724eb521@Oracle.com>

+1,

Roger


On 10/28/2016 11:58 AM, nadeesh tv wrote:
>
> Hi Anubhav,
>
>
> -     * @throws DateTimeException if the field is not available and 
> the section is not optional
>
>
> I think you should not remove the DTException since still there is a 
> chance of throwing DTE
> Regards,
> Nadeesh
> On 10/28/2016 12:18 AM, Anubhav Meena wrote:
>> Hi all,
>>
>> Please review. Please ignore last mail as links not working properly 
>> there.
>>
>> Bug Id : https://bugs.openjdk.java.net/browse/JDK-8167618 
>> <https://bugs.openjdk.java.net/browse/JDK-8167618>
>> Issue:  DateTimeFormatter.format() uses exceptions for flow control.
>> Webrev: 
>> http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/ 
>> <http://cr.openjdk.java.net/~rchamyal/anmeena/8167618/webrev.00/>
>


From Roger.Riggs at Oracle.com  Fri Oct 28 16:43:37 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 12:43:37 -0400
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <4d9c8b60-930d-8232-f3a6-eceae318ba04@oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
	<15b839a1-a3e5-56f8-5bb2-2d43e13486a6@Oracle.com>
	<4d9c8b60-930d-8232-f3a6-eceae318ba04@oracle.com>
Message-ID: <790664e7-0cb8-e57e-625a-d4cc63a5bfee@Oracle.com>

Hi Daniel,

The test is fine.
Without the fix, jtreg is handling the uncaught LinkageError exception
and has called System.exit which will deadlock since it is already in 
System.exit.

Roger


On 10/28/2016 10:44 AM, Daniel Fuchs wrote:
> On 28/10/16 14:50, Roger Riggs wrote:
>> Hi Daniel,
>>
>> Looks fine.
>>
>> Roger
>
> Thanks Roger.
>
> I have turned the reproducer attached to JDK-8152515 into a
> jtreg test. The test creates some logger and registers a handler
> that throws LinkageError when closed.
>
> The test passes as expected with the fix.
> Without the fix however, it makes jtreg fail in timeout
> (which is a bit strange):
>
> ----------System.out:(2/193)----------
> [LinkageErrorTest$TestHandler at 3a7f70{closed=false}, 
> LinkageErrorTest$TestHandler at 25da7d46{closed=false}, 
> LinkageErrorTest$TestHandler at 6c57d74f{closed=true}]
> Timeout signalled after 120 seconds
> ----------System.err:(9/621)----------
> STATUS:Passed.
> java.lang.LinkageError
>     at LinkageErrorTest$TestHandler.close(LinkageErrorTest.java:63)
>     at 
> java.util.logging.LogManager.closeHandlers(java.logging at 9-internal/LogManager.java:1449)
>     at 
> java.util.logging.LogManager.resetLogger(java.logging at 9-internal/LogManager.java:1459)
>     at 
> java.util.logging.LogManager.resetLoggerContext(java.logging at 9-internal/LogManager.java:1439)
>     at 
> java.util.logging.LogManager.reset(java.logging at 9-internal/LogManager.java:1424)
>     at 
> java.util.logging.LogManager$Cleaner.run(java.logging at 9-internal/LogManager.java:284)
> STATUS:Failed.`main' threw exception: java.lang.LinkageError
> ----------rerun:(21/1847)*----------
>
> I am not sure whether I should push this test or not...
>
> What do you think? Is that behavior going to stay stable?
> I mean - can we rely on the fact that jtreg will always
> fail this test if the fix is not present?
>
> Here is the webrev with the test:
> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.01
>
> -- daniel
>
>>
>>
>> On 10/28/2016 7:51 AM, Daniel Fuchs wrote:
>>> Hi,
>>>
>>> Please find below a trivial  patch for:
>>>
>>> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
>>> https://bugs.openjdk.java.net/browse/JDK-8152515
>>>
>>>
>>> Patch:
>>> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>>>
>>> The issue might occur at shutdown, when a handler that makes uses
>>> of some APIs provided by an OSGI bundle which was already closed
>>> by the shutdown process is in turn closed by the LogManager.Cleaner
>>> thread. In that case some subclasses of LinkageError may be thrown,
>>> interrupting the reset process and preventing other handlers from
>>> being closed properly.
>>>
>>> The patch proposes to trivially ignore LinkageError at shutdown while
>>> the LogManager.Cleaner thread is running.
>>>
>>> best regards,
>>>
>>> -- daniel
>>
>


From Roger.Riggs at Oracle.com  Fri Oct 28 17:48:27 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 13:48:27 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
Message-ID: <892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>

Hi David,

On 10/27/2016 9:00 PM, David Holmes wrote:
>>> But this is the second waitFor call after the process is destroyed.
>>> Sorry I don't really see the point.
>> The tests were added to determine if waitFor(timeout) was handling the
>> timeout parameter correctly.
>> The 2nd test here was to check the datapath when the process already
>> been destroyed.
>> The 'extra' waitFor() ensures the process is gone.
>> But you are correct, this exercises a path through the waitFor code that
>> does not even look
>> at the timeout value so I'll remove the entire 2nd case.
>>
>> Webrev updated in place.
>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>
> Okay, but now you don't need the p.waitFor() after the destroy() at all.
>
> Hmmm ... I guess either we want the original code with a longer 
> timeout to accommodate slow/loaded platforms (to test that waitFor(n) 
> after a destroy() is timely); or everything after destroy() can be 
> deleted.
Here's another approach to test whether waitFor(timeout) is behaving as 
expected.
The test should be checking that waitFor does not wait longer than 
requested.

So shorten the timeout to a very short time (10 milliseconds) and check that
waitFor returns in no more than that time(plus epsilon).  It is similar 
to the first test
in that section but the process is exiting.

    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

Thanks, Roger


>
> Cheers,
> David
>
>
>> Thanks, Roger
>>> David
>>>
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> David
>>>>>
>>>>>> I contemplated increasing the timeout but given the issue is system
>>>>>> loading
>>>>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>>>>>> and the harness timeout is 2min.
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>> Webrev:
>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>>>
>>>>>>>> Thanks, Roger
>>>>>>>>
>>>>>>
>>>>
>>


From david.holmes at oracle.com  Fri Oct 28 19:14:36 2016
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 29 Oct 2016 05:14:36 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
	<892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
Message-ID: <fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>

On 29/10/2016 3:48 AM, Roger Riggs wrote:
> Hi David,
>
> On 10/27/2016 9:00 PM, David Holmes wrote:
>>>> But this is the second waitFor call after the process is destroyed.
>>>> Sorry I don't really see the point.
>>> The tests were added to determine if waitFor(timeout) was handling the
>>> timeout parameter correctly.
>>> The 2nd test here was to check the datapath when the process already
>>> been destroyed.
>>> The 'extra' waitFor() ensures the process is gone.
>>> But you are correct, this exercises a path through the waitFor code that
>>> does not even look
>>> at the timeout value so I'll remove the entire 2nd case.
>>>
>>> Webrev updated in place.
>>>   http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>
>> Okay, but now you don't need the p.waitFor() after the destroy() at all.
>>
>> Hmmm ... I guess either we want the original code with a longer
>> timeout to accommodate slow/loaded platforms (to test that waitFor(n)
>> after a destroy() is timely); or everything after destroy() can be
>> deleted.
> Here's another approach to test whether waitFor(timeout) is behaving as
> expected.
> The test should be checking that waitFor does not wait longer than
> requested.
>
> So shorten the timeout to a very short time (10 milliseconds) and check that
> waitFor returns in no more than that time(plus epsilon).  It is similar
> to the first test
> in that section but the process is exiting.
>
>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

But the first test is checking that we wait at-least as long as the 
timeout ie no early return when we know the process is still alive.

Sorry but this isn't making sense to me. The original problem is that 
waitFor(8) has timed out and we proceeded to call exitValue() while the 
process is still alive. If the process had in fact terminated then we 
would have hit the (end-start) check and failed due to a timeout. So 
what exactly are we trying to fix here? Do we simply prefer to fail due 
to timeout rather than have exitValue() throw IllegalStateException? If 
so then check for waitFor(8) timing out, but leave the timeout value at 
8 seconds - just avoid the call to exitValue() if a time-out occurred. 
Otherwise if we want to avoid timing out we either bump the timeout to 
some value > 8 to accommodate the load problem, or we don't even bother 
doing waitFor after the destroy().

I'm really unclear what this is trying to test, and what a failure would 
indicate.

Thanks,
David

> Thanks, Roger
>
>
>>
>> Cheers,
>> David
>>
>>
>>> Thanks, Roger
>>>> David
>>>>
>>>>>
>>>>> Roger
>>>>>
>>>>>>
>>>>>> David
>>>>>>
>>>>>>> I contemplated increasing the timeout but given the issue is system
>>>>>>> loading
>>>>>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2 min.
>>>>>>> and the harness timeout is 2min.
>>>>>>>
>>>>>>> Roger
>>>>>>>
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>> Webrev:
>>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>>>>
>>>>>>>>> Thanks, Roger
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>

From rfscholte at apache.org  Fri Oct 28 19:15:13 2016
From: rfscholte at apache.org (Robert Scholte)
Date: Fri, 28 Oct 2016 21:15:13 +0200
Subject: JEP 293: Guidelines for JDK Command-Line Tool Options - @-files
In-Reply-To: <etPan.5812dc82.46191f8f.2ad@oracle.com>
References: <op.yp0gj1y1kdkhrr@desktop-2khsk44>
	<etPan.5812dc82.46191f8f.2ad@oracle.com>
Message-ID: <op.yp143ngxkdkhrr@desktop-2khsk44.dynamic.ziggo.nl>

On Fri, 28 Oct 2016 07:05:06 +0200, Henry Jen <henry.jen at oracle.com> wrote:

> OS-specific encoding, but has to be ASCII friendly, modern system with  
> UTF-8 as system encoding should work just fine.

Hmm, this will probably work for 99%, but I'm sure that we will have Maven  
users which use special characters on their system and they will hit this  
problem and put it on our mailinglist. I would really like a complete  
answer which, for instance, makes it possible to refer to any file on the  
system. Even if the answer is: use  ISO 8859-1 and unicode escaping I  
would be happy.

btw, it seems like my modern Windows 10 still uses Cp1252

>
> Space in quote should work just fine, for example, ?c:\\Program Files?  
> should be correct. Can you post messages from JLink? Also if you can  
> verify java is working OK with space, that would be helpful to tell if  
> there is something different in JLink.

I got this message:
Error: Module velocity not found, required by simple.command

Just to be sure it is not my system I cleared my local repository.

Now I'm getting new messages, often one of these:
Error: module-info.class not found for hibernate.jpa module
or
Error: module-info.class not found for spring.context module
or
Error: module-info.class not found for spring.orm module

So let's forget about the space, might be a corrupted file in my  
repository, can't explain the exception though.

Now I'm blocked by these errors. Yes, it is true that these modules don't  
have a module-info.class, but that shouldn't be a problem, right?  
Compiling the project with my module-info files went fine...

thanks,
Robert

>
> Cheers,
> Henry
>
> On October 27, 2016 at 2:27:44 PM, Robert Scholte (rfscholte at apache.org)  
> wrote:
>> Hi,
>>
>> I'm facing some troubles with the content of the @-files and the
>> documentation[1] isn't helping me yet.
>>
>> First of all it doesn't mention the encoding, I assume it is the OS
>> specific encoding.
>>
>> I'm facing issues with a long --module-path on Windows.
>> I noticed I can use the "normal" filename (not a URI or URL), but I need
>> to escape the \ with an \, resulting in
>> "E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-command\\target\\classes;E:\\java-workspace\\sandbox\\mvnexbook-examples-1.0\\ch-multi-spring\\simple-parent\\simple-weather\\target\\simple-weather-0.8-SNAPSHOT.jar;"
>> (and much more entries)
>>
>> You can use a space or a newline as separator between arguments, but if  
>> an
>> argument contains spaces it must be surrounded with double quotes.
>> However, the path to my local (Maven) repository contains a space and
>> JLink is complaining it cannot find these dependencies. It seems like I
>> need to escape spaces too, though I haven't figured out how to do that.
>> Changing my local repo to a space-less path works. There should be a way
>> to handle spaces, but how?
>>
>> I'm not sure if there are other specific things to keep in mind when
>> writing @-files, but it would be nice if that would be explained it the
>> jep too.
>>
>> thanks,
>> Robert
>>
>> [1] http://openjdk.java.net/jeps/293
>>

From david.holmes at oracle.com  Fri Oct 28 19:32:14 2016
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 29 Oct 2016 05:32:14 +1000
Subject: RFR: 8152515: (logging) LogManager.resetLogger should ignore
	LinkageError
In-Reply-To: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
References: <850f971b-bffc-10d4-3097-979beb302351@oracle.com>
Message-ID: <415297a7-4d20-6680-b36e-df6a5041713e@oracle.com>

Hi Daniel,

I've read the bug report on this and this issue "smells" to me. 
LinkageError should not be a special case IMHO. The existing code was 
trying to go the "ignore everything but 'serious errors' " route - but 
without really considering what constitutes a "serious error". I think 
the fact a LinkageError can turn up here reflects a bug in the code that 
throws it - or possibly in something in between.

There should be a very clear exception handling policy for this code, 
and not special cases IMHO. Having that policy depend on whether 
shutdown is in progress is okay to me as we know that things can behave 
unexpectedly in that case. So I would advocate for a more general

+             } catch (Throwable e) {
+                 // Ignore all exceptions while shutting down
+                 if (globalHandlersState != STATE_SHUTDOWN) {
+                     throw e;
+                 }

Cheers,
David

On 28/10/2016 9:51 PM, Daniel Fuchs wrote:
> Hi,
>
> Please find below a trivial  patch for:
>
> 8152515: (logging) LogManager.resetLogger should ignore LinkageError
> https://bugs.openjdk.java.net/browse/JDK-8152515
>
>
> Patch:
> http://cr.openjdk.java.net/~dfuchs/webrev_8152515/webrev.00/
>
> The issue might occur at shutdown, when a handler that makes uses
> of some APIs provided by an OSGI bundle which was already closed
> by the shutdown process is in turn closed by the LogManager.Cleaner
> thread. In that case some subclasses of LinkageError may be thrown,
> interrupting the reset process and preventing other handlers from
> being closed properly.
>
> The patch proposes to trivially ignore LinkageError at shutdown while
> the LogManager.Cleaner thread is running.
>
> best regards,
>
> -- daniel

From Roger.Riggs at Oracle.com  Fri Oct 28 19:39:16 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 15:39:16 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
	<892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
	<fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>
Message-ID: <ea64cdcd-ae80-9071-28d0-3ddd889065ee@Oracle.com>

Hi,

On 10/28/2016 3:14 PM, David Holmes wrote:
> On 29/10/2016 3:48 AM, Roger Riggs wrote:
>> Hi David,
>>
>> On 10/27/2016 9:00 PM, David Holmes wrote:
>>>>> But this is the second waitFor call after the process is destroyed.
>>>>> Sorry I don't really see the point.
>>>> The tests were added to determine if waitFor(timeout) was handling the
>>>> timeout parameter correctly.
>>>> The 2nd test here was to check the datapath when the process already
>>>> been destroyed.
>>>> The 'extra' waitFor() ensures the process is gone.
>>>> But you are correct, this exercises a path through the waitFor code 
>>>> that
>>>> does not even look
>>>> at the timeout value so I'll remove the entire 2nd case.
>>>>
>>>> Webrev updated in place.
>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>
>>> Okay, but now you don't need the p.waitFor() after the destroy() at 
>>> all.
>>>
>>> Hmmm ... I guess either we want the original code with a longer
>>> timeout to accommodate slow/loaded platforms (to test that waitFor(n)
>>> after a destroy() is timely); or everything after destroy() can be
>>> deleted.
>> Here's another approach to test whether waitFor(timeout) is behaving as
>> expected.
>> The test should be checking that waitFor does not wait longer than
>> requested.
>>
>> So shorten the timeout to a very short time (10 milliseconds) and 
>> check that
>> waitFor returns in no more than that time(plus epsilon).  It is similar
>> to the first test
>> in that section but the process is exiting.
>>
>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>
> But the first test is checking that we wait at-least as long as the 
> timeout ie no early return when we know the process is still alive.
>
> Sorry but this isn't making sense to me. The original problem is that 
> waitFor(8) has timed out and we proceeded to call exitValue() while 
> the process is still alive. If the process had in fact terminated then 
> we would have hit the (end-start) check and failed due to a timeout. 
> So what exactly are we trying to fix here? Do we simply prefer to fail 
> due to timeout rather than have exitValue() throw IllegalStateException?
calling exitValue was just additional debugging info; but the test would 
have failed the timeout condition later.
So the test is still broken, because it is not checking the correct 
timeout behavior of waitFor(timeout) and instead
is checking how long it takes the OS to destroy.  That's not interesting.
> If so then check for waitFor(8) timing out, but leave the timeout 
> value at 8 seconds - just avoid the call to exitValue() if a time-out 
> occurred. Otherwise if we want to avoid timing out we either bump the 
> timeout to some value > 8 to accommodate the load problem, or we don't 
> even bother doing waitFor after the destroy().
Lengthening the time isn't interesting either because it is only testing 
that it returns
when the process has exited, not whether the timeout is hit at the right 
time.

So I guess, its time to remove the whole test case on a destroyed process.

Roger

>
> I'm really unclear what this is trying to test, and what a failure 
> would indicate.
>
> Thanks,
> David
>
>> Thanks, Roger
>>
>>
>>>
>>> Cheers,
>>> David
>>>
>>>
>>>> Thanks, Roger
>>>>> David
>>>>>
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>> I contemplated increasing the timeout but given the issue is 
>>>>>>>> system
>>>>>>>> loading
>>>>>>>> I didn't have a good idea what to raise it to. 30sec, 1 min, 2 
>>>>>>>> min.
>>>>>>>> and the harness timeout is 2min.
>>>>>>>>
>>>>>>>> Roger
>>>>>>>>
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>> Webrev:
>>>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>>>>>
>>>>>>>>>> Thanks, Roger
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>


From Alan.Bateman at oracle.com  Fri Oct 28 20:34:42 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Fri, 28 Oct 2016 21:34:42 +0100
Subject: JEP 293: Guidelines for JDK Command-Line Tool Options - @-files
In-Reply-To: <op.yp143ngxkdkhrr@desktop-2khsk44.dynamic.ziggo.nl>
References: <op.yp0gj1y1kdkhrr@desktop-2khsk44>
	<etPan.5812dc82.46191f8f.2ad@oracle.com>
	<op.yp143ngxkdkhrr@desktop-2khsk44.dynamic.ziggo.nl>
Message-ID: <a40f5941-1f6a-8c0e-4e99-e8e6c124cdb1@oracle.com>



On 28/10/2016 20:15, Robert Scholte wrote:
> :
>
> Now I'm getting new messages, often one of these:
> Error: module-info.class not found for hibernate.jpa module
> or
> Error: module-info.class not found for spring.context module
> or
> Error: module-info.class not found for spring.orm module
>
> So let's forget about the space, might be a corrupted file in my 
> repository, can't explain the exception though.
>
> Now I'm blocked by these errors. Yes, it is true that these modules 
> don't have a module-info.class, but that shouldn't be a problem, 
> right? Compiling the project with my module-info files went fine...
If there isn't a module-info.class in the root directory of these JAR 
files then they are considered to be automatic modules. Automatic 
modules can't be linked into a runtime image. I realize this isn't the 
main point of your thread but just mentioning it that you will always 
get an error from link when there is a dependency on an automatic module.

-Alan

From david.holmes at oracle.com  Fri Oct 28 20:59:03 2016
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 29 Oct 2016 06:59:03 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <ea64cdcd-ae80-9071-28d0-3ddd889065ee@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
	<892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
	<fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>
	<ea64cdcd-ae80-9071-28d0-3ddd889065ee@Oracle.com>
Message-ID: <06ac3a6b-5b7e-8648-424d-81b78d93f186@oracle.com>

On 29/10/2016 5:39 AM, Roger Riggs wrote:
> Hi,
>
> On 10/28/2016 3:14 PM, David Holmes wrote:
>> On 29/10/2016 3:48 AM, Roger Riggs wrote:
>>> Hi David,
>>>
>>> On 10/27/2016 9:00 PM, David Holmes wrote:
>>>>>> But this is the second waitFor call after the process is destroyed.
>>>>>> Sorry I don't really see the point.
>>>>> The tests were added to determine if waitFor(timeout) was handling the
>>>>> timeout parameter correctly.
>>>>> The 2nd test here was to check the datapath when the process already
>>>>> been destroyed.
>>>>> The 'extra' waitFor() ensures the process is gone.
>>>>> But you are correct, this exercises a path through the waitFor code
>>>>> that
>>>>> does not even look
>>>>> at the timeout value so I'll remove the entire 2nd case.
>>>>>
>>>>> Webrev updated in place.
>>>>>   http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>
>>>> Okay, but now you don't need the p.waitFor() after the destroy() at
>>>> all.
>>>>
>>>> Hmmm ... I guess either we want the original code with a longer
>>>> timeout to accommodate slow/loaded platforms (to test that waitFor(n)
>>>> after a destroy() is timely); or everything after destroy() can be
>>>> deleted.
>>> Here's another approach to test whether waitFor(timeout) is behaving as
>>> expected.
>>> The test should be checking that waitFor does not wait longer than
>>> requested.
>>>
>>> So shorten the timeout to a very short time (10 milliseconds) and
>>> check that
>>> waitFor returns in no more than that time(plus epsilon).  It is similar
>>> to the first test
>>> in that section but the process is exiting.
>>>
>>>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>
>> But the first test is checking that we wait at-least as long as the
>> timeout ie no early return when we know the process is still alive.
>>
>> Sorry but this isn't making sense to me. The original problem is that
>> waitFor(8) has timed out and we proceeded to call exitValue() while
>> the process is still alive. If the process had in fact terminated then
>> we would have hit the (end-start) check and failed due to a timeout.
>> So what exactly are we trying to fix here? Do we simply prefer to fail
>> due to timeout rather than have exitValue() throw IllegalStateException?
> calling exitValue was just additional debugging info; but the test would
> have failed the timeout condition later.
> So the test is still broken, because it is not checking the correct
> timeout behavior of waitFor(timeout) and instead
> is checking how long it takes the OS to destroy.  That's not interesting.
>> If so then check for waitFor(8) timing out, but leave the timeout
>> value at 8 seconds - just avoid the call to exitValue() if a time-out
>> occurred. Otherwise if we want to avoid timing out we either bump the
>> timeout to some value > 8 to accommodate the load problem, or we don't
>> even bother doing waitFor after the destroy().
> Lengthening the time isn't interesting either because it is only testing
> that it returns
> when the process has exited, not whether the timeout is hit at the right
> time.

There is no real way to test timeouts. You can check you don't get early 
returns but you can't check that you return in a timely fashion. At 
least without knowing the execution environment.

Does waitFor implement it's own timed-waiting mechanism, or does it just 
delegate to an existing API (like wait() or park()) ? If the latter then 
that is where timeout testing would exist.

The earlier part of the test already verifies both that the timeout 
mechanism functions, and that it doesn't return early.

> So I guess, its time to remove the whole test case on a destroyed process.

Given destroy() is not guaranteed to work either ... testing waitFor in 
that context will always be somewhat probabilistic I think - and timeout 
testing just makes that more so.

Cheers,
David

> Roger
>
>>
>> I'm really unclear what this is trying to test, and what a failure
>> would indicate.
>>
>> Thanks,
>> David
>>
>>> Thanks, Roger
>>>
>>>
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>>
>>>>> Thanks, Roger
>>>>>> David
>>>>>>
>>>>>>>
>>>>>>> Roger
>>>>>>>
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>> I contemplated increasing the timeout but given the issue is
>>>>>>>>> system
>>>>>>>>> loading
>>>>>>>>> I didn't have a good idea what to raise it to.  30sec, 1 min, 2
>>>>>>>>> min.
>>>>>>>>> and the harness timeout is 2min.
>>>>>>>>>
>>>>>>>>> Roger
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>> Webrev:
>>>>>>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Roger
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>

From Roger.Riggs at Oracle.com  Fri Oct 28 21:16:38 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 28 Oct 2016 17:16:38 -0400
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <06ac3a6b-5b7e-8648-424d-81b78d93f186@oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
	<892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
	<fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>
	<ea64cdcd-ae80-9071-28d0-3ddd889065ee@Oracle.com>
	<06ac3a6b-5b7e-8648-424d-81b78d93f186@oracle.com>
Message-ID: <aa70acd5-6e58-8414-1a4a-022ac2a1e693@Oracle.com>

Hi,


On 10/28/2016 4:59 PM, David Holmes wrote:

>>>>>>> But this is the second waitFor call after the process is destroyed.
>>>>>>> Sorry I don't really see the point.
>>>>>> The tests were added to determine if waitFor(timeout) was 
>>>>>> handling the
>>>>>> timeout parameter correctly.
>>>>>> The 2nd test here was to check the datapath when the process already
>>>>>> been destroyed.
>>>>>> The 'extra' waitFor() ensures the process is gone.
>>>>>> But you are correct, this exercises a path through the waitFor code
>>>>>> that
>>>>>> does not even look
>>>>>> at the timeout value so I'll remove the entire 2nd case.
>>>>>>
>>>>>> Webrev updated in place.
>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>
>>>>> Okay, but now you don't need the p.waitFor() after the destroy() at
>>>>> all.
>>>>>
>>>>> Hmmm ... I guess either we want the original code with a longer
>>>>> timeout to accommodate slow/loaded platforms (to test that waitFor(n)
>>>>> after a destroy() is timely); or everything after destroy() can be
>>>>> deleted.
>>>> Here's another approach to test whether waitFor(timeout) is 
>>>> behaving as
>>>> expected.
>>>> The test should be checking that waitFor does not wait longer than
>>>> requested.
>>>>
>>>> So shorten the timeout to a very short time (10 milliseconds) and
>>>> check that
>>>> waitFor returns in no more than that time(plus epsilon).  It is 
>>>> similar
>>>> to the first test
>>>> in that section but the process is exiting.
>>>>
>>>> http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>
>>> But the first test is checking that we wait at-least as long as the
>>> timeout ie no early return when we know the process is still alive.
>>>
>>> Sorry but this isn't making sense to me. The original problem is that
>>> waitFor(8) has timed out and we proceeded to call exitValue() while
>>> the process is still alive. If the process had in fact terminated then
>>> we would have hit the (end-start) check and failed due to a timeout.
>>> So what exactly are we trying to fix here? Do we simply prefer to fail
>>> due to timeout rather than have exitValue() throw 
>>> IllegalStateException?
>> calling exitValue was just additional debugging info; but the test would
>> have failed the timeout condition later.
>> So the test is still broken, because it is not checking the correct
>> timeout behavior of waitFor(timeout) and instead
>> is checking how long it takes the OS to destroy.  That's not 
>> interesting.
>>> If so then check for waitFor(8) timing out, but leave the timeout
>>> value at 8 seconds - just avoid the call to exitValue() if a time-out
>>> occurred. Otherwise if we want to avoid timing out we either bump the
>>> timeout to some value > 8 to accommodate the load problem, or we don't
>>> even bother doing waitFor after the destroy().
>> Lengthening the time isn't interesting either because it is only testing
>> that it returns
>> when the process has exited, not whether the timeout is hit at the right
>> time.
>
> There is no real way to test timeouts. You can check you don't get 
> early returns but you can't check that you return in a timely fashion. 
> At least without knowing the execution environment.
The spec for waitFor is " until the
      * process represented by this Process object has
      * terminated, or the specified waiting time elapses."
>
> Does waitFor implement it's own timed-waiting mechanism, or does it 
> just delegate to an existing API (like wait() or park()) ? If the 
> latter then that is where timeout testing would exist.
Thread.sleep() on Linux with interrupt() from a monitoring thread.

>
> The earlier part of the test already verifies both that the timeout 
> mechanism functions, and that it doesn't return early.
>
>> So I guess, its time to remove the whole test case on a destroyed 
>> process.
>
> Given destroy() is not guaranteed to work either ... testing waitFor 
> in that context will always be somewhat probabilistic I think - and 
> timeout testing just makes that more so.
ok, removed the test case.

     http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/

Thanks, Roger

>
> Cheers,
> David
>
>> R


From david.holmes at oracle.com  Fri Oct 28 21:33:38 2016
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 29 Oct 2016 07:33:38 +1000
Subject: RFR 9: 8168517 : java/lang/ProcessBuilder/Basic.java failed
In-Reply-To: <aa70acd5-6e58-8414-1a4a-022ac2a1e693@Oracle.com>
References: <eb7c50ba-2911-f855-3d55-b797aee31c08@Oracle.com>
	<6dd7c8a5-e62d-19b1-6118-fefcde853e29@oracle.com>
	<1166accb-71ba-85da-6898-13c20a1077f5@Oracle.com>
	<3529c173-32e9-500e-f8f5-531caeea4d82@oracle.com>
	<6370a4ec-75dd-2e90-62f7-8ed4635f63f9@Oracle.com>
	<bb7540f9-df1c-6765-12f4-249573fa6231@oracle.com>
	<d1f21035-c9be-7666-59ac-69a5df2e52fb@oracle.com>
	<18c54710-b03c-cf97-384d-d2708213cd9a@oracle.com>
	<892e3e15-8d7d-141b-47e1-8c66d01e278f@Oracle.com>
	<fce3453d-ec5f-96c3-681d-7e6679c33723@oracle.com>
	<ea64cdcd-ae80-9071-28d0-3ddd889065ee@Oracle.com>
	<06ac3a6b-5b7e-8648-424d-81b78d93f186@oracle.com>
	<aa70acd5-6e58-8414-1a4a-022ac2a1e693@Oracle.com>
Message-ID: <fddfb4ef-4672-e7c3-ef01-7af66dc4d7d1@oracle.com>

Hi Roger,

Okay we are back to where we were a couple of emails ago. :) Removing 
everything after the p.destroy() seems fine to me.

Thanks,
David

On 29/10/2016 7:16 AM, Roger Riggs wrote:
> Hi,
>
>
> On 10/28/2016 4:59 PM, David Holmes wrote:
>
>>>>>>>> But this is the second waitFor call after the process is destroyed.
>>>>>>>> Sorry I don't really see the point.
>>>>>>> The tests were added to determine if waitFor(timeout) was
>>>>>>> handling the
>>>>>>> timeout parameter correctly.
>>>>>>> The 2nd test here was to check the datapath when the process already
>>>>>>> been destroyed.
>>>>>>> The 'extra' waitFor() ensures the process is gone.
>>>>>>> But you are correct, this exercises a path through the waitFor code
>>>>>>> that
>>>>>>> does not even look
>>>>>>> at the timeout value so I'll remove the entire 2nd case.
>>>>>>>
>>>>>>> Webrev updated in place.
>>>>>>>   http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>>>
>>>>>> Okay, but now you don't need the p.waitFor() after the destroy() at
>>>>>> all.
>>>>>>
>>>>>> Hmmm ... I guess either we want the original code with a longer
>>>>>> timeout to accommodate slow/loaded platforms (to test that waitFor(n)
>>>>>> after a destroy() is timely); or everything after destroy() can be
>>>>>> deleted.
>>>>> Here's another approach to test whether waitFor(timeout) is
>>>>> behaving as
>>>>> expected.
>>>>> The test should be checking that waitFor does not wait longer than
>>>>> requested.
>>>>>
>>>>> So shorten the timeout to a very short time (10 milliseconds) and
>>>>> check that
>>>>> waitFor returns in no more than that time(plus epsilon).  It is
>>>>> similar
>>>>> to the first test
>>>>> in that section but the process is exiting.
>>>>>
>>>>>    http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>>>>
>>>> But the first test is checking that we wait at-least as long as the
>>>> timeout ie no early return when we know the process is still alive.
>>>>
>>>> Sorry but this isn't making sense to me. The original problem is that
>>>> waitFor(8) has timed out and we proceeded to call exitValue() while
>>>> the process is still alive. If the process had in fact terminated then
>>>> we would have hit the (end-start) check and failed due to a timeout.
>>>> So what exactly are we trying to fix here? Do we simply prefer to fail
>>>> due to timeout rather than have exitValue() throw
>>>> IllegalStateException?
>>> calling exitValue was just additional debugging info; but the test would
>>> have failed the timeout condition later.
>>> So the test is still broken, because it is not checking the correct
>>> timeout behavior of waitFor(timeout) and instead
>>> is checking how long it takes the OS to destroy.  That's not
>>> interesting.
>>>> If so then check for waitFor(8) timing out, but leave the timeout
>>>> value at 8 seconds - just avoid the call to exitValue() if a time-out
>>>> occurred. Otherwise if we want to avoid timing out we either bump the
>>>> timeout to some value > 8 to accommodate the load problem, or we don't
>>>> even bother doing waitFor after the destroy().
>>> Lengthening the time isn't interesting either because it is only testing
>>> that it returns
>>> when the process has exited, not whether the timeout is hit at the right
>>> time.
>>
>> There is no real way to test timeouts. You can check you don't get
>> early returns but you can't check that you return in a timely fashion.
>> At least without knowing the execution environment.
> The spec for waitFor is " until the
>      * process represented by this Process object has
>      * terminated, or the specified waiting time elapses."
>>
>> Does waitFor implement it's own timed-waiting mechanism, or does it
>> just delegate to an existing API (like wait() or park()) ? If the
>> latter then that is where timeout testing would exist.
> Thread.sleep() on Linux with interrupt() from a monitoring thread.
>
>>
>> The earlier part of the test already verifies both that the timeout
>> mechanism functions, and that it doesn't return early.
>>
>>> So I guess, its time to remove the whole test case on a destroyed
>>> process.
>>
>> Given destroy() is not guaranteed to work either ... testing waitFor
>> in that context will always be somewhat probabilistic I think - and
>> timeout testing just makes that more so.
> ok, removed the test case.
>
>     http://cr.openjdk.java.net/~rriggs/webrev-basic-destroy-8168517/
>
> Thanks, Roger
>
>>
>> Cheers,
>> David
>>
>>> R
>

From lance.andersen at oracle.com  Fri Oct 28 23:29:26 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Fri, 28 Oct 2016 19:29:26 -0400
Subject: RFR (JAXP) 8069098 : StAX produces the wrong event stream
In-Reply-To: <5810F5FE.4040908@oracle.com>
References: <5810F5FE.4040908@oracle.com>
Message-ID: <66846685-7BD8-416A-B9AF-3C5A42B468DB@oracle.com>

+1
> On Oct 26, 2016, at 2:29 PM, Joe Wang <huizhe.wang at oracle.com> wrote:
> 
> Hi,
> 
> Please review an update to the Javadoc of the StAX API from the JSR 173 specification [1], and fix to the implementation to comply with the specification that the reader's initial event shall be START_DOCUMENT.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8069098
> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8069098/webrev/
> 
> 
> [1] https://www.jcp.org/en/jsr/detail?id=173
> 
> Thanks,
> Joe
> 
> 

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From brunoaiss at gmail.com  Sat Oct 29 18:10:29 2016
From: brunoaiss at gmail.com (Brunoais)
Date: Sat, 29 Oct 2016 19:10:29 +0100
Subject: Request/discussion: BufferedReader reading using async API while
	providing sync API
In-Reply-To: <dc5b8f4a-720d-abf0-6aec-af04847a4c03@gmail.com>
References: <077bf6e8-14ae-5588-b1a4-03fb4333c15b@gmail.com>
	<00d24fd4-3c8f-9e9a-2419-91c7c37c91e5@gmail.com>
	<8AB1350F-F3EE-4756-923C-4178263BEBF5@oracle.com>
	<48cd729d-7922-3de5-3983-cc7486b5a9d3@gmail.com>
	<CCE28203-5E35-437A-87EA-998209C02BDA@oracle.com>
	<4bf67d65-7b2d-59a7-cd4a-19d84a062799@gmail.com>
	<A495710C-9F61-4045-AFF9-88512510F008@oracle.com>
	<10318b41-0383-d68a-3145-bdc57ae78450@gmail.com>
	<CAHjP37HPE3c2OZsGv+7seJb2XouQOU7ZsmRJ+3yNxHY2bQuhZg@mail.gmail.com>
	<c4812ac5-2b85-cb47-f2e6-d809bcdade4a@gmail.com>
	<CAHjP37EFcxxHS3gy6o7t9fzFEtfR2Q_OwFnv5cwGLYKXFsmR7A@mail.gmail.com>
	<7f977d1d-3aa7-ec2e-6df5-51e8ac6473e5@gmail.com>
	<CAHjP37FL2wwNudUVDneqveS0Y_jiiOL82yUALG+w6Y=u_deoUw@mail.gmail.com>
	<a7b3d1ec-b2af-2d92-d400-d6ec18d95d45@gmail.com>
	<CAHjP37Hjzb5ydvoYnYnUe_m1NUXwLFceK1vMzZgkj2bSpoTvCA@mail.gmail.com>
	<b357da6f-a505-fdad-a0c9-8a8914023f66@gmail.com>
	<f6253ff3-e154-73db-cc67-8c4bc802a5f9@oracle.com>
	<dc5b8f4a-720d-abf0-6aec-af04847a4c03@gmail.com>
Message-ID: <6fe864bb-bc46-59fd-a7db-c9328e9ea71f@gmail.com>

Here's my try on going async.

On my tests on my local windows 10 machine (I don't have access to the 
linux one without a VM ATM) , now with 1GB file, I noticed:

~2s of speed improvement from BufferedInputStream to 
BufferedNonBlockStream when BufferedNonBlockStream is at its most 
advantageous state (with CPU work between reads). Otherwise, it has 
~0.3s speed improvement; maybe less.

~0.8s of speed improvement from BufferedNonBlockStream to 
BufferedAsyncStream when BufferedNonBlockStream/BufferedAsyncStream is 
at its most advantageous state. Otherwise, ~0 speed improvement.

I noticed: Until you process as fast as you read, both 
BufferedNonBlockStream and BufferedAsyncStream gain advantage towards 
BufferedInputStream. From the point as you take longer to process than 
to I/O, BufferedNonBlockStream and BufferedAsyncStream tend to keep the 
advantage.

If the file is in cache, BufferedInputStream takes ~1.6-1.9s to read the 
file, BufferedNonBlockStream takes a steady ~2.05-2.1s to read the file 
and BufferedAsyncStream ~1.2s.

BufferedNonBlockStream and BufferedAsyncStream win more when given more 
buffer memory than BufferedInputStream but only until a certain limit. 
On my hardware, the best speed I got was with 655360B for the new ones. 
Any more than that was not producing any visible results. I guess it is 
due to the speed data was processing for the test.

On 28/10/2016 09:16, Brunoais wrote:
> I'll try going back to a previous version I worked on which used the 
> java7's AsynchronousFileChannel and work from there. My small research 
> shows it can also work with AsynchronousFileChannel mostly without 
> changes.
>
> For now, 1 question:
> Is Thread.sleep() a possible way of dealing the block requirements of 
> read()? Do I need to use LockSupport.park() or something like that?
>
> I'll call back here when it is done.
>
>
> On 27/10/2016 22:09, David Holmes wrote:
>> You might try discussing on net-dev rather than core-libs-dev, to get 
>> additional historical info related to the io and nio file APIs.
>>
>> David
>>
>> On 28/10/2016 5:08 AM, Brunoais wrote:
>>> You are right. Even in windows it does not set the flags for async
>>> reads. It seems like it is windows itself that does the decision to
>>> buffer the contents based on its own heuristics.
>>>
>>> But... Why? Why won't it be? Why is there no API for it? How am I
>>> getting 100% HDD use and faster times when I fake work to delay getting
>>> more data and I only have a fluctuating 60-90% (always going up and
>>> down) when I use an InputStream?
>>> Is it related to how both classes cache and how frequently and how much
>>> each one asks for data?
>>>
>>> I really would prefer not having to read the source code because it
>>> takes a real long time T.T.
>>>
>>> I end up reinstating... And wondering...
>>>
>>> Why doesn't java provide a single-threaded non-block API for file reads
>>> for all OS that support it? I simply cannot find that information no
>>> matter how much I search on google, bing, duck duck go... Can any of 
>>> you
>>> point me to whomever knows?
>>>
>>> On 27/10/2016 14:11, Vitaly Davidovich wrote:
>>>> I don't know about Windows specifically, but generally file systems
>>>> across major OS's will implement readahead in their IO scheduler when
>>>> they detect sequential scans.
>>>>
>>>> On Linux, you can also strace your test to confirm which syscalls are
>>>> emitted (you should be seeing plain read()'s there, with
>>>> FileInputStream and FileChannel).
>>>>
>>>> On Thu, Oct 27, 2016 at 9:06 AM, Brunoais <brunoaiss at gmail.com
>>>> <mailto:brunoaiss at gmail.com>> wrote:
>>>>
>>>>     Thanks for the heads up.
>>>>
>>>>     I'll try that later. These tests are still useful then. Meanwhile,
>>>>     I'll end up also checking how FileChannel queries the OS on
>>>>     windows. I'm getting 100% HDD reads... Could it be that the OS
>>>>     reads the file ahead on its own?... Anyway, I'll look into it.
>>>>     Thanks for the heads up.
>>>>
>>>>
>>>>     On 27/10/2016 13:53, Vitaly Davidovich wrote:
>>>>>
>>>>>
>>>>>     On Thu, Oct 27, 2016 at 8:34 AM, Brunoais <brunoaiss at gmail.com
>>>>>     <mailto:brunoaiss at gmail.com>> wrote:
>>>>>
>>>>>         Oh... I see. In that case, it means something is terribly
>>>>>         wrong. It can be my initial tests, though.
>>>>>
>>>>>         I'm testing on both linux and windows and I'm getting
>>>>>         performance gains from using the FileChannel compared to
>>>>>         using FileInputStream... The tests also make sense based on
>>>>>         my predictions O_O...
>>>>>
>>>>>     FileInputStream requires copying native buffers holding the read
>>>>>     data to the java byte[].  If you're using direct ByteBuffer for
>>>>>     FileChannel, that whole memcpy is skipped.  Try comparing
>>>>>     FileChannel with HeapByteBuffer instead.
>>>>>
>>>>>
>>>>>         On 27/10/2016 11:47, Vitaly Davidovich wrote:
>>>>>>
>>>>>>
>>>>>>         On Thursday, October 27, 2016, Brunoais <brunoaiss at gmail.com
>>>>>>         <mailto:brunoaiss at gmail.com>> wrote:
>>>>>>
>>>>>>             Did you read the C code?
>>>>>>
>>>>>>         I looked at the Linux code in the JDK.
>>>>>>
>>>>>>             Have you got any idea how many functions Windows or
>>>>>>             Linux (nearly all flavors) have for the read operation
>>>>>>             towards a file?
>>>>>>
>>>>>>         I do.
>>>>>>
>>>>>>
>>>>>>             I have already done that homework myself. I may not have
>>>>>>             read JVM's source code but I know well that there's
>>>>>>             functions on both Windows and Linux that provide such
>>>>>>             interface I mentioned although they require a slightly
>>>>>>             different treatment (and different constants).
>>>>>>
>>>>>>         You should read the JDK (native) source code instead of
>>>>>>         guessing/assuming.  On Linux, it doesn't use aio facilities
>>>>>>         for files.  The kernel io scheduler may issue readahead
>>>>>>         behind the scenes, but there's no nonblocking file io that's
>>>>>>         at the heart of your premise.
>>>>>>
>>>>>>
>>>>>>
>>>>>>             On 27/10/2016 00:06, Vitaly Davidovich wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>                 On Wednesday, October 26, 2016, Brunoais
>>>>>>                 <brunoaiss at gmail.com <mailto:brunoaiss at gmail.com>>
>>>>>>                 wrote:
>>>>>>
>>>>>>                     It is actually based on the premise that:
>>>>>>
>>>>>>                     1. The first call to
>>>>>>                 ReadableByteChannel.read(ByteBuffer) sets the OS
>>>>>>                        buffer size to fill in as the same size as
>>>>>>                 ByteBuffer.
>>>>>>
>>>>>>                 Why do you say that? AFAICT, it issues a read
>>>>>>                 syscall and that will block if the data isn't in
>>>>>>                 page cache.
>>>>>>
>>>>>>                     2. The consecutive calls to
>>>>>>                 ReadableByteChannel.read(ByteBuffer)
>>>>>>                     orders
>>>>>>                        the JVM to order the OS to execute memcpy()
>>>>>>                 to copy from its memory
>>>>>>                        to the shared memory created at ByteBuffer
>>>>>>                 instantiation (in
>>>>>>                     java 8)
>>>>>>                        using Unsafe and then for the JVM to update
>>>>>>                 the ByteBuffer fields.
>>>>>>
>>>>>>                 I think subsequent reads just invoke the same read
>>>>>>                 syscall, passing the current file offset maintained
>>>>>>                 by the file channel instance.
>>>>>>
>>>>>>                     3. The call will not block waiting for I/O and
>>>>>>                 it won't take longer
>>>>>>                        than the JNI interface if no new data exists.
>>>>>>                 However, it will
>>>>>>                     block
>>>>>>                        waiting for the OS to execute memcpy() to the
>>>>>>                 shared memory.
>>>>>>
>>>>>>                 So why do you think it won't block?
>>>>>>
>>>>>>
>>>>>>                     Is my premise wrong?
>>>>>>
>>>>>>                     If I read correctly, if I don't use a
>>>>>>                 DirectBuffer, there would be
>>>>>>                     even another intermediate buffer to copy data to
>>>>>>                 before giving it
>>>>>>                     to the "user" which would be useless.
>>>>>>
>>>>>>                 If you use a HeapByteBuffer, then there's an extra
>>>>>>                 copy from the native buffer to the Java buffer.
>>>>>>
>>>>>>
>>>>>>
>>>>>>                     On 26/10/2016 11:57, Pavel Rappo wrote:
>>>>>>
>>>>>>                         I believe I see where you coming from.
>>>>>>                 Please correct me if
>>>>>>                         I'm wrong.
>>>>>>
>>>>>>                         Your implementation is based on the premise
>>>>>>                 that a call to
>>>>>>                 ReadableByteChannel.read()
>>>>>>                         _initiates_ the operation and returns
>>>>>>                 immediately. The OS then
>>>>>>                         continues to fill
>>>>>>                         the buffer while there's a free space in the
>>>>>>                 buffer and the
>>>>>>                         channel hasn't encountered EOF.
>>>>>>
>>>>>>                         Is that right?
>>>>>>
>>>>>>                             On 25 Oct 2016, at 22:16, Brunoais
>>>>>>                 <brunoaiss at gmail.com>
>>>>>>                             wrote:
>>>>>>
>>>>>>                             Thank you for your time. I'll try to
>>>>>>                 explain it. I hope I
>>>>>>                             can clear it up.
>>>>>>                             First of it, I made a meaning mistake
>>>>>>                 between asynchronous
>>>>>>                             and non-blocking. This implementation
>>>>>>                 uses a non-blocking
>>>>>>                             algorithm internally while providing a
>>>>>>                 blocking-like
>>>>>>                             algorithm on the surface. It is
>>>>>>                 single-threaded and not
>>>>>>                             multi-threaded where one thread fetches
>>>>>>                 data and blocks
>>>>>>                             waiting and the other accumulates it and
>>>>>>                 provides to
>>>>>>                             whichever wants it.
>>>>>>
>>>>>>                             Second of it, I had made a mistake of
>>>>>>                 going after
>>>>>>                             BufferedReader instead of going after
>>>>>>                 BufferedInputStream.
>>>>>>                             If you want me to go after
>>>>>>                 BufferedReader it's ok but I
>>>>>>                             only thought that going after
>>>>>>                 BufferedInputStream would be
>>>>>>                             more generically useful than
>>>>>>                 BufferedReaderwhen I started
>>>>>>                             the poc.
>>>>>>
>>>>>>                             On to my code:
>>>>>>                             Short answers:
>>>>>>                                     ? The sleep(int) exists because
>>>>>>                 I don't know how
>>>>>>                             to wait until more data exists in the
>>>>>>                 buffer which is part
>>>>>>                             of read()'s contract.
>>>>>>                                     ? The ByteBuffer gives a buffer
>>>>>>                 that is filled by
>>>>>>                             the OS (what I believe Channels do)
>>>>>>                 instead of getting
>>>>>>                             data only         by demand (what I
>>>>>>                 believe Streams do).
>>>>>>                             Full answers:
>>>>>>                             The blockingFill(boolean) method is a
>>>>>>                 method for a busy
>>>>>>                             wait for a fill which is used
>>>>>>                 exclusively by the read()
>>>>>>                             method. All other methods use the
>>>>>>                 version that does not
>>>>>>                             sleep (fill(boolean)).
>>>>>>                 blockingFill(boolean)'s existance like that is only
>>>>>>                             because the read() method must not
>>>>>>                 return unless either:
>>>>>>
>>>>>>                                     ? The stream ended.
>>>>>>                                     ? The next byte is ready for
>>>>>>                 reading.
>>>>>>                             Additionally, statistically, that while
>>>>>>                 loop will rarely
>>>>>>                             evaluate to true as reads are in chunks
>>>>>>                 so readPos will be
>>>>>>                             behind writePos most of the time.
>>>>>>                             I have no idea if an interrupt will ever
>>>>>>                 happen, to be
>>>>>>                             honest. The main reasons why I'm using a
>>>>>>                 sleep is because
>>>>>>                             I didn't want a hog onto the CPU in a
>>>>>>                 full thread usage
>>>>>>                             busy wait and because I didn't find any
>>>>>>                 way of doing a
>>>>>>                             thread sleep in order to wake up later
>>>>>>                 when the buffer
>>>>>>                             managed by native code has more data.
>>>>>>                             The Non-blocking part is managed by the
>>>>>>                 buffer the OS
>>>>>>                             keeps filling most if not all the time.
>>>>>>                 That buffer is the
>>>>>>                             field
>>>>>>
>>>>>>                             ByteBuffer readBuffer
>>>>>>                             That's the gaining part against the
>>>>>>                 plain old Buffered
>>>>>>                             classes.
>>>>>>
>>>>>>
>>>>>>                             Did that make sense to you? Feel free to
>>>>>>                 ask anything else
>>>>>>                             you need.
>>>>>>
>>>>>>                             On 25/10/2016 20:52, Pavel Rappo wrote:
>>>>>>
>>>>>>                                 I've skimmed through the code and
>>>>>>                 I'm not sure I can
>>>>>>                                 see any asynchronicity
>>>>>>                                 (you were pointing at the lack of it
>>>>>>                 in BufferedReader).
>>>>>>                                 And the mechanics of this is very
>>>>>>                 puzzling to me, to
>>>>>>                                 be honest:
>>>>>>                                      void blockingFill(boolean
>>>>>>                 forced) throws
>>>>>>                                 IOException {
>>>>>>                  fill(forced);
>>>>>>                                          while (readPos == 
>>>>>> writePos) {
>>>>>>                                              try {
>>>>>>                  Thread.sleep(100);
>>>>>>                                              } catch
>>>>>>                 (InterruptedException e) {
>>>>>>                  // An interrupt may mean more data is
>>>>>>                                 available
>>>>>>                                              }
>>>>>>                  fill(forced);
>>>>>>                                          }
>>>>>>                                      }
>>>>>>                                 I thought you were suggesting that
>>>>>>                 we should utilize
>>>>>>                                 the tools which OS provides
>>>>>>                                 more efficiently. Instead we have
>>>>>>                 something that looks
>>>>>>                                 very similarly to a
>>>>>>                                 "busy loop" and... also who and when
>>>>>>                 is supposed to
>>>>>>                                 interrupt Thread.sleep()?
>>>>>>                                 Sorry, I'm not following. Could you
>>>>>>                 please explain how
>>>>>>                                 this is supposed to work?
>>>>>>
>>>>>>                                     On 24 Oct 2016, at 15:59, 
>>>>>> Brunoais
>>>>>> <brunoaiss at gmail.com>
>>>>>>                                       wrote:
>>>>>>                                     Attached and sending!
>>>>>>                                     On 24/10/2016 13:48, Pavel Rappo
>>>>>>                 wrote:
>>>>>>
>>>>>>                                         Could you please send a new
>>>>>>                 email on this list
>>>>>>                                         with the source attached 
>>>>>> as a
>>>>>>                                         text file?
>>>>>>
>>>>>>                                             On 23 Oct 2016, at
>>>>>>                 19:14, Brunoais
>>>>>> <brunoaiss at gmail.com>
>>>>>>                 wrote:
>>>>>>                 Here's my poc/prototype:
>>>>>>
>>>>>>                 http://pastebin.com/WRpYWDJF
>>>>>>
>>>>>>                                             I've implemented the
>>>>>>                 bare minimum of the
>>>>>>                 class that follows the same contract of
>>>>>>                 BufferedReader while signaling all issues
>>>>>>                                             I think it may have or
>>>>>>                 has in comments.
>>>>>>                                             I also wrote some
>>>>>>                 javadoc to help guiding
>>>>>>                 through the class.
>>>>>>                                             I could have used more
>>>>>>                 fields from
>>>>>>                 BufferedReader but the names were so
>>>>>>                 minimalistic that were confusing me. I
>>>>>>                 intent to change them before sending this
>>>>>>                                             to openJDK.
>>>>>>                                             One of the major
>>>>>>                 problems this has is long
>>>>>>                 overflowing. It is major because it is
>>>>>>                 hidden, it will be extremely rare and it
>>>>>>                 takes a really long time to reproduce.
>>>>>>                 There are different ways of dealing with
>>>>>>                                             it. From just
>>>>>>                 documenting to actually
>>>>>>                 making code that works with it.
>>>>>>                                             I built a simple test
>>>>>>                 code for it to have
>>>>>>                                             some ideas about
>>>>>>                 performance and correctness.
>>>>>>
>>>>>>                 http://pastebin.com/eh6LFgwT
>>>>>>
>>>>>>                                             This doesn't do a
>>>>>>                 through test if it is
>>>>>>                 actually working correctly but I see no
>>>>>>                 reason for it not working correctly after
>>>>>>                 fixing the 2 bugs that test found.
>>>>>>                                             I'll also leave here
>>>>>>                 some conclusions
>>>>>>                 about speed and resource consumption I found.
>>>>>>                                             I made tests with
>>>>>>                 default buffer sizes,
>>>>>>                 5000B 15_000B and 500_000B. I noticed
>>>>>>                 that, with my hardware, with the 1 530 000
>>>>>>                                             000B file, I was getting
>>>>>>                 around:
>>>>>>                                             In all buffers and fake
>>>>>>                 work: 10~15s speed
>>>>>>                 improvement ( from 90% HDD speed to 100%
>>>>>>                                             HDD speed)
>>>>>>                                             In all buffers and no
>>>>>>                 fake work: 1~2s
>>>>>>                 speed improvement ( from 90% HDD speed to
>>>>>>                                             100% HDD speed)
>>>>>>                 Changing the buffer size was giving
>>>>>>                 different reading speeds but both were
>>>>>>                 quite equal in how much they would change
>>>>>>                                             when changing the buffer
>>>>>>                 size.
>>>>>>                 Finally, I could always confirm that I/O
>>>>>>                                             was always the slowest
>>>>>>                 thing while this
>>>>>>                                             code was running.
>>>>>>                                             For the ones wondering
>>>>>>                 about the file
>>>>>>                 size; it is both to avoid OS cache and to
>>>>>>                                             make the reading at the
>>>>>>                 main use-case
>>>>>>                 these objects are for (large streams of
>>>>>>                 bytes).
>>>>>>                 @Pavel, are you open for discussion now
>>>>>>                                             ;)? Need anything else?
>>>>>>                                             On 21/10/2016 19:21,
>>>>>>                 Pavel Rappo wrote:
>>>>>>
>>>>>>                 Just to append to my previous email.
>>>>>>                 BufferedReader wraps any Reader out there.
>>>>>>                 Not specifically FileReader. While
>>>>>>                 you're talking about the case of effective
>>>>>>                 reading from a file.
>>>>>>                 I guess there's one existing
>>>>>>                 possibility to provide exactly what
>>>>>>                 you need (as I
>>>>>>                 understand it) under this method:
>>>>>>                 /**
>>>>>>                   * Opens a file for reading,
>>>>>>                 returning a {@code BufferedReader} to
>>>>>>                 read text
>>>>>>                   * from the file in an efficient
>>>>>>                 manner...
>>>>>>                     ...
>>>>>>                   */
>>>>>>
>>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>>                 It can return _anything_ as long as it
>>>>>>                 is a BufferedReader. We can do it, but it
>>>>>>                 needs to be investigated not only for
>>>>>>                 your favorite OS but for other OSes as
>>>>>>                 well. Feel free to prototype this and
>>>>>>                 we can discuss it on the list later.
>>>>>>                 Thanks,
>>>>>>                 -Pavel
>>>>>>
>>>>>>                     On 21 Oct 2016, at 18:56, Brunoais
>>>>>>                     <brunoaiss at gmail.com>
>>>>>>                       wrote:
>>>>>>                     Pavel is right.
>>>>>>                     In reality, I was expecting such
>>>>>>                     BufferedReader to use only a
>>>>>>                     single buffer and have that Buffer
>>>>>>                     being filled asynchronously, not
>>>>>>                     in a different Thread.
>>>>>>                     Additionally, I don't have the
>>>>>>                     intention of having a larger
>>>>>>                     buffer than before unless stated
>>>>>>                     through the API (the constructor).
>>>>>>                     In my idea, internally, it is
>>>>>>                     supposed to use
>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>>                     or equivalent.
>>>>>>                     It does not prevent having two
>>>>>>                     buffers and I do not intent to
>>>>>>                     change BufferedReader itself. I'd
>>>>>>                     do an BufferedAsyncReader of sorts
>>>>>>                     (any name suggestion is welcome as
>>>>>>                     I'm an awful namer).
>>>>>>                     On 21/10/2016 18:38, Roger Riggs
>>>>>>                     wrote:
>>>>>>
>>>>>>                         Hi Pavel,
>>>>>>                         I think Brunoais asking for a
>>>>>>                         double buffering scheme in
>>>>>>                         which the implementation of
>>>>>>                         BufferReader fills (a second
>>>>>>                         buffer) in parallel with the
>>>>>>                         application reading from the
>>>>>>                         1st buffer
>>>>>>                         and managing the swaps and
>>>>>>                         async reads transparently.
>>>>>>                         It would not change the API
>>>>>>                         but would change the
>>>>>>                         interactions between the
>>>>>>                         buffered reader
>>>>>>                         and the underlying stream.  It
>>>>>>                         would also increase memory
>>>>>>                         requirements and processing
>>>>>>                         by introducing or using a
>>>>>>                         separate thread and the
>>>>>>                         necessary synchronization.
>>>>>>                         Though I think the formal
>>>>>>                         interface semantics could be
>>>>>>                         maintained, I have doubts
>>>>>>                         about compatibility and its
>>>>>>                         unintended consequences on
>>>>>>                         existing subclasses,
>>>>>>                         applications and libraries.
>>>>>>                         $.02, Roger
>>>>>>                         On 10/21/16 1:22 PM, Pavel
>>>>>>                         Rappo wrote:
>>>>>>
>>>>>>                             Off the top of my head, I
>>>>>>                             would say it's not
>>>>>>                             possible to change the
>>>>>>                             design of an
>>>>>>                             _extensible_ type that has
>>>>>>                             been out there for 20 or
>>>>>>                             so years. All these I/O
>>>>>>                             streams from java.io <http://java.io>
>>>>>>                             <http://java.io> were
>>>>>>                             designed for simple
>>>>>>                             synchronous use case.
>>>>>>                             It's not that their design
>>>>>>                             is flawed in some way,
>>>>>>                             it's that they doesn't seem to
>>>>>>                             suit your needs. Have you
>>>>>>                             considered using
>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>>                             in your applications?
>>>>>>                             -Pavel
>>>>>>
>>>>>>                                 On 21 Oct 2016, at
>>>>>>                                 17:08, Brunoais
>>>>>> <brunoaiss at gmail.com>
>>>>>>                                   wrote:
>>>>>>                                 Any feedback on this?
>>>>>>                                 I'm really interested
>>>>>>                                 in implementing such
>>>>>>                 BufferedReader/BufferedStreamReader
>>>>>>                                 to allow speeding up
>>>>>>                                 my applications
>>>>>>                                 without having to
>>>>>>                                 think in an
>>>>>>                                 asynchronous way or
>>>>>>                                 multi-threading while
>>>>>>                                 programming with it.
>>>>>>                                 That's why I'm asking
>>>>>>                                 this here.
>>>>>>                                 On 13/10/2016 14:45,
>>>>>>                                 Brunoais wrote:
>>>>>>
>>>>>>                                     Hi,
>>>>>>                                     I looked at
>>>>>>                 BufferedReader
>>>>>>                                     source code for
>>>>>>                                     java 9 long with
>>>>>>                                     the source code of
>>>>>>                                     the
>>>>>>                 channels/streams
>>>>>>                                     used. I noticed
>>>>>>                                     that, like in java
>>>>>>                                     7, BufferedReader
>>>>>>                                     does not use an
>>>>>>                                     Async API to load
>>>>>>                                     data from files,
>>>>>>                                     instead, the data
>>>>>>                                     loading is all
>>>>>>                                     done synchronously
>>>>>>                                     even when the OS
>>>>>>                                     allows requesting
>>>>>>                                     a file to be read
>>>>>>                                     and getting a
>>>>>>                                     warning later when
>>>>>>                                     the file is
>>>>>>                                     effectively read.
>>>>>>                                     Why Is
>>>>>>                 BufferedReader not
>>>>>>                                     async while
>>>>>>                                     providing a sync API?
>>>>>>
>>>>>> <BufferedNonBlockStream.java><Tests.java>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>                 --                 Sent from my phone
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>         --         Sent from my phone
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
package org.sample.BufferedNon_BlockIO;

import java.io.BufferedInputStream;
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousByteChannel;
import java.nio.channels.AsynchronousFileChannel;
import java.nio.channels.CompletionHandler;
import java.util.concurrent.locks.LockSupport;

public class BufferedAsyncStream extends BufferedInputStream {

	private static final int DEFAULT_BUFFER_SIZE = 8_192;
	
	/**
     * The maximum size of array to allocate.
     * Some VMs reserve some header words in an array.
     * Attempts to allocate larger arrays may result in
     * OutOfMemoryError: Requested array size exceeds VM limit
     */
    private static int MAX_BUFFER_SIZE = Integer.MAX_VALUE - 8;

	private static final int MIN_READ_BUFFER_SIZE = 10;
	/**
     * The maximum size of the read buffer if set automatically.
     * Unless explicitly told, there's no need to starve the VM out of memory
     */
	private static final int MAX_READ_BUFFER_SIZE = 1_000_000;
	
	/**
     * The virtual index of the next position where data will be read.
     * Every operation requires executing a mathematical mod ({@code %}) operation on it against <code>buf.length</code>
     * to get the correct buffer position to work on.
     */
    protected long readPos;
    
    /**
     * The virtual index of the next position where data will be written
     * Every operation requires executing a mathematical mod ({@code %}) operation on it against <code>buf.length</code>
     * to get the correct buffer position to work on.
     * 
     * At every time, {@code readPos <= writePos %} must be true
     * 
     */
    // Note to reviewers: What to do when long overflows? It will happen but does a documented warning suffice or... do I need to work with the overflows?
    protected long writePos;
	
	ChannelReader readFrom;
	ByteBuffer readBuffer;
	FileReadCallback<Object> readCallback;
	IOException nextException;
	
	transient boolean isReading;
	
	Thread parkingThread;
	
	boolean isClosed;
	boolean eof;

	interface ChannelReader extends Closeable{
		void read(ByteBuffer dst);
	}
	class AsyncFileChannelReader implements ChannelReader, CompletionHandler<Integer, ByteBuffer>{
		AsynchronousFileChannel inChannel;
		long currentPosition;
		AsyncFileChannelReader(AsynchronousFileChannel inChannel) {
			this.inChannel = inChannel;
			currentPosition = 0;
		}

		@Override
		public void read(ByteBuffer dst) {
			isReading = true;
			inChannel.read(dst, currentPosition, dst, this);
		}
		@Override
		public void close() throws IOException{
			inChannel.close();
		}
		
		@Override
		public void completed(Integer result, ByteBuffer attachment) {
			if(result > 0){
				currentPosition += result;
			}
			readCallback.completed(result, attachment);
		}

		@Override
		public void failed(Throwable exc, ByteBuffer attachment) {
			readCallback.failed(exc, attachment);
		}

	}
	
	class AsyncByteChannelReader implements ChannelReader{
		AsynchronousByteChannel inChannel;
		
		AsyncByteChannelReader(AsynchronousByteChannel inChannel) {
			this.inChannel = inChannel;
		}
		
		@Override
		public void read(ByteBuffer dst) {
			isReading = true;
			inChannel.read(dst, dst, readCallback);
		}
		@Override
		public void close() throws IOException {
			inChannel.close();
		}
		
	}

	class FileReadCallback<A> implements CompletionHandler<Integer, A>{

		@Override
		public void completed(Integer resultI, A attachment) {
			// unbox
			int result = resultI;
			if(result == -1){
				eof = true;
			}
			
			long freeBufSpace = writeSpaceAvailable();
			
			if(freeBufSpace < readBuffer.position()){
				// No enough space in buf to copy from readBuffer.
				// Just send it back to fill even more
				readFrom.read(readBuffer);
				return;
			}
			if(readBuffer.position() > readBuffer.capacity() / 2 || (eof && readBuffer.position() > 0)){
				readBuffer.flip();
				// Ints are faster and an array can only be defined using an int
				int writeBufferFrom = (int) (writePos % buf.length);
	            int canReadAmount =  Math.min((int) freeBufSpace, readBuffer.remaining());
	            int canWriteUntil = (writeBufferFrom + canReadAmount) % buf.length;
				
	            // This can only be done like this because ReadBufferSize is smaller than bufferSize
	            // and overflows are ignored
	        	if(canWriteUntil < writeBufferFrom){
	        		// Read in 2 parts
	        		readBuffer.get(buf, writeBufferFrom, buf.length - writeBufferFrom);
	        		readBuffer.get(buf, 0, canWriteUntil);
	        	} else {
	        		readBuffer.get(buf, writeBufferFrom, canWriteUntil - writeBufferFrom);
	        	}

	        	writePos += canReadAmount;
	        	// Reset the buffer for more reading
	        	readBuffer.clear();
	        	
	        	LockSupport.unpark(parkingThread);
			}
			if(eof){
				isReading = false;
				return;
			}
			readFrom.read(readBuffer);
		}

		@Override
		public void failed(Throwable exc, A attachment) {
			if(exc instanceof IOException){
				nextException = (IOException) exc;
			} else {
				nextException = new IOException("Failed to read more data to the buffer", exc);
			}
			isReading = false;
		}
		
	}
	
		
	public BufferedAsyncStream(InputStream inputStream) {
		super(inputStream);
	}
	
	public BufferedAsyncStream(AsynchronousFileChannel inputChannel) {
		this(inputChannel, DEFAULT_BUFFER_SIZE);
	}
	
	public BufferedAsyncStream(AsynchronousFileChannel inputChannel, int bufferSize) {
		this(inputChannel, bufferSize, Math.max(Math.min(bufferSize / 4, MAX_READ_BUFFER_SIZE), MIN_READ_BUFFER_SIZE));
	}
	
	public BufferedAsyncStream(AsynchronousFileChannel inputChannel,  int bufferSize, int readBufferSize) {
		super(null, Math.max(bufferSize, MIN_READ_BUFFER_SIZE + 1));
//		Objects.requireNonNull(inputChannel, "The file channel must not be null");
		init(new AsyncFileChannelReader(inputChannel), Math.max(bufferSize, MIN_READ_BUFFER_SIZE + 1), readBufferSize);
	}

	
	public BufferedAsyncStream(AsynchronousByteChannel inputChannel) {
		this(inputChannel, DEFAULT_BUFFER_SIZE);
	}
	
	public BufferedAsyncStream(AsynchronousByteChannel inputChannel, int bufferSize) {
		this(inputChannel, bufferSize, Math.max(Math.min(bufferSize / 4, MAX_READ_BUFFER_SIZE), MIN_READ_BUFFER_SIZE));
	}
	
	public BufferedAsyncStream(AsynchronousByteChannel inputChannel,  int bufferSize, int readBufferSize) {
		super(null, Math.max(bufferSize, MIN_READ_BUFFER_SIZE + 1));
//		Objects.requireNonNull(inputChannel, "The byte channel must not be null");
		init(new AsyncByteChannelReader(inputChannel), Math.max(bufferSize, MIN_READ_BUFFER_SIZE + 1), readBufferSize);
	}
	
	private void init(ChannelReader inputChannel, int bufferSize, int readBufferSize){

		this.readFrom = inputChannel;
		
		if(readBufferSize < 10){
			throw new IllegalArgumentException("Read buffer must be at least 10 bytes");
		}
		
		if(readBufferSize > bufferSize){
			throw new IllegalArgumentException("ReadBufferSize must be smaller than bufferSize");
		}
		
		// The read buffer must not be larger than the internal buffer.
		readBufferSize = Math.min(readBufferSize, MAX_BUFFER_SIZE - 1);
		this.readBuffer = ByteBuffer.allocateDirect(readBufferSize);
		
		this.readCallback = new FileReadCallback<>();
		
		isClosed = false;
		
		readFrom.read(readBuffer);
	}
	
	int readSpaceAvailable(){
		// This can be an int because buf size can never be larger than the int
		return (int) (writePos - readPos);
	}
	int writeSpaceAvailable(){
		return buf.length - readSpaceAvailable();
	}
	
	/**
	 * Blocks waiting for a write.
	 * After leaving the block, it is guaranteed that at least 1 byte is readable from the buffer 
	 * @throws IOException
	 */
	// This is /* synchronized */ to make sure only up to 1 thread is parked inside
	void waitForFill() throws IOException {
		while(readSpaceAvailable() < 1 && !eof){
			try{
				parkingThread = Thread.currentThread();
				// wait for 1 second tops. File/web reading really should be faster than that
				LockSupport.parkNanos(this, 1_000_000_000);
			}finally{
				parkingThread = null;			
			}
    	}
    }
 
	
	private int readingFormalities() throws IOException {
		if(!isReading){
			if(nextException != null){
				try{
					throw nextException;
				} finally {
					nextException = null;
				}
			}
			if(eof){
				return -1;
			}
			readFrom.read(readBuffer);
		}
		
		return readSpaceAvailable();
	}
	
	@Override
	public /* synchronized */ int read() throws IOException {
		if(readFrom == null){
			return super.read();			
		}
		
		switch(readingFormalities()){
			case -1:
				return -1;
			case 0:
				waitForFill();
				/* No Default */
		}
		
		return buf[(int)(readPos++ % buf.length)] & 0xff;
	}

	@Override
	public /* synchronized */ int read(byte[] b, int off, int len) throws IOException {
		if(readFrom == null){
			return super.read(b, off, len);
		}
		
		if ((off | len | (off + len) | (b.length - (off + len))) < 0) {
            throw new IndexOutOfBoundsException();
        } else if (len == 0) {
            return 0;
        }
		
		switch(readingFormalities()){
			case -1:
				return -1;
			case 0:
//				waitForFill();
				/* No Default */
		}
		
		// Note for reviewing: Some optimizations that take some time were not made here.
		// E.g.
		// -> If b consumes the whole buff, copy directly from ByteBuffer to this buff
		// -> As long as reading buffs are large enough (requires benchmarking), 
		// 	  not do use the buf array at all (late initialization too)

        int ByteBufferFrom = (int) (readPos % buf.length);
        int canReadAmount =  Math.min((int) readSpaceAvailable(), len - off);
        int canByteBufferUntil = (ByteBufferFrom + canReadAmount) % buf.length;
        
		if(canByteBufferUntil < ByteBufferFrom){
    		// For reviewer: Should it always read once? Or should it read twice?
			System.arraycopy(buf, ByteBufferFrom, b, off, buf.length - ByteBufferFrom);
			int ndStartAt = off + (buf.length - ByteBufferFrom);
			// This if should never evaluate "true". It's here just to make sure and make breakpoints
			if(off - ndStartAt >= len){
				readPos = canByteBufferUntil;
				return off - ndStartAt; 
			}
			System.arraycopy(buf, 0, b, ndStartAt, canByteBufferUntil);
			readPos += (buf.length - ByteBufferFrom) + canByteBufferUntil;
			return (buf.length - ByteBufferFrom) + canByteBufferUntil;
    	} else {
			System.arraycopy(buf, ByteBufferFrom, b, off, canByteBufferUntil - ByteBufferFrom);
			readPos += canByteBufferUntil - ByteBufferFrom;
			return canByteBufferUntil - ByteBufferFrom;
    	}
		
		
	}

	@Override
	public long skip(long arg0) throws IOException {
		if(readFrom == null){
			return super.skip(arg0);
		}
		
		// There's no skip in this poc
		// For the real implementation, I'd do (while I didn't skip enough):
		// I'd just "jump" the readPos accordingly (while keeping readPos <= writePos)
		// Then 
		// reposition (.position(int)) in buffer
		// OR
		// clear the buffer
		// Then
		// If SeekableChannel -> use (sord of) .position(.position()+ skipLeft)
		// else -> do not seek.
		
		
		throw new UnsupportedOperationException();
		
	}

	@Override
	public int available() throws IOException {
		if(readFrom == null){
			return super.available();
		}
		return readSpaceAvailable();
		
	}

	@Override
	public void mark(int arg0) {
		if(readFrom == null){
			super.mark(arg0);
		}
		
		// This is not hard to do but it requires quite some time!
		// I'll wait first if the poc passes without this method
	}

	@Override
	public void reset() throws IOException {
		if(readFrom == null){
			super.reset();
		}
		throw new IOException("The mark was lost");
	}

	@Override
	public boolean markSupported() {
		if(readFrom == null){
			return super.markSupported();
		}
		// false for now, at least.
		return false;
	}

	@Override
	public void close() throws IOException {
		if(readFrom != null){
			readFrom.close();
		}
		super.close();
	}

}
-------------- next part --------------
package org.sample.BufferedNon_BlockIO;

import java.io.BufferedInputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.channels.AsynchronousFileChannel;
import java.nio.channels.FileChannel;
import java.nio.file.StandardOpenOption;
import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.annotations.Threads;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;

/**
 * A sample file reading benchmark
 */
@BenchmarkMode(Mode.SingleShotTime)
@Fork(value = 1)
@Warmup(iterations = 0, time = 2, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2)
@Threads(1)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class BufferedNonBlockStreamBenchmark_try1 {

	public static boolean isWindows = System.getProperty("os.name").contains("Windows");
	
    @Param({"1gig_file"})
    String file;

//    @Param({"4100", "16400", "131100", "1048600"})
    @Param({"4100", "131100", "1048600"})
    int javaBufSize;
    
    @Param({"100", "1000"})
    int readSize;

//    @Param({"0", "1000", "100000"})
    @Param({"0", "1000"})
    long cpuWork;

    // 												This MUST be smaller than javaBufSize
    //  											@Param({"4096", "16384", "131072", "1048576"})
    @Param({"BufferedInputStream", "BufferedNonBlockStream|4096", "BufferedNonBlockStream|131072", "BufferedNonBlockStream|1048576",
    							   "BufferedAsyncStream|4096",    "BufferedAsyncStream|131072",    "BufferedAsyncStream|1048576"})
    String implType;

    interface ReadOp {
        int read(byte[] buf) throws IOException;
    }

    private ReadOp read;
    private Closeable close;
    private byte[] buf;

    @Setup(Level.Iteration)
    public void setup() throws IOException, InterruptedException {

        clearCache();
        
        String[] type_readSize = implType.split("\\|");

        switch (type_readSize[0]) {
            case "BufferedInputStream":{
                BufferedInputStream in = new BufferedInputStream(
                    new FileInputStream(file), javaBufSize);
                read = in::read;
                close = in::close;
            }
                break;
            case "BufferedNonBlockStream": {
            	BufferedNonBlockStream in = new BufferedNonBlockStream(
        			FileChannel.open(new File(file).toPath()), javaBufSize, Integer.parseInt(type_readSize[1]));
                read = in::read;
                close = in::close;
            }
            break;
            case "BufferedAsyncStream": {
            	BufferedAsyncStream in = new BufferedAsyncStream(
        			AsynchronousFileChannel.open(new File(file).toPath()), javaBufSize, Integer.parseInt(type_readSize[1]));
                read = in::read;
                close = in::close;
            }
            break;
            default:
                throw new IllegalArgumentException(
                    "Invalid parameter 'implType': " + implType);
        }

        buf = new byte[readSize];
    }

    @TearDown(Level.Iteration)
    public void tearDown() throws IOException {
        close.close();
    }

    /**
     * 
     * For linux:
     * 
     * Compile the following C program into clear_cache executable:
     * <pre>{@code
     *
     * #include <stdio.h>
     * #include <string.h>
     * #include <errno.h>
     *
     * #define PROC_FILE "/proc/sys/vm/drop_caches"
     *
     * int main(char *argv[], int argc) {
     *   FILE *f;
     *   f = fopen(PROC_FILE, "w");
     *   if (f) {
     *     fprintf(f, "3\n");
     *     fclose(f);
     *     return 0;
     *   } else {
     *     fprintf(stderr, "Can't write to: %s: %s\n", PROC_FILE, strerror(errno));
     *     return 1;
     *   }
     * }
     *
     * }</pre>
     * ... and make it SUID root!
     * 
     * For windows:
     * Use the provided clear_cache.exe
     * (source code and original author:)
     * https://gist.github.com/bitshifter/c87aa396446bbebeab29
     * Then run this program with administrator privileges
     */
    private static void clearCache() throws IOException, InterruptedException {
        // spawn an OS command to clear the FS cache...
    	if(isWindows){
    		new ProcessBuilder("clear_cache.exe").start().waitFor();
    	} else {
    		new ProcessBuilder("clear_cache").start().waitFor();    		
    	}
    }

    @Benchmark
    public int testRead() throws IOException {
        int nread = 0;
        int n;
        while ((n = read.read(buf)) >= 0) {
            nread += n;
            Blackhole.consumeCPU(cpuWork);
        }
        return nread;
    }
}

From ivan.gerasimov at oracle.com  Sun Oct 30 01:42:13 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Sun, 30 Oct 2016 04:42:13 +0300
Subject: [9] RFR: 8168921: Inconsistent Annotation.toString()
Message-ID: <82ad25c8-bf50-6842-3d1a-19a4e7e53bb6@oracle.com>

Hello!

Suppose an annotation is of form

     @interface MyAnnotation {
         long value();
     }

Create the annotation object with 
sun.reflect.annotation.AnnotationParser.annotationForMap(MyAnnotation.class, 
Map.of("value", some_value)).

Annotation.toString() method, called for the object, will produce a 
string containing that 'some_value' with the optional 'L' suffix appended.
Normally, the suffix is not appended for values that can be represented 
with int, and is appended otherwise.
However, the logic does not hold for values Long.MIN_VALUE and 
Integer.MIN_VALUE:  For the former the suffix is not appended and for 
the later the suffix L is appended.

Would you please help review the fix, which removes this tiny inconsistency?

BUGURL: https://bugs.openjdk.java.net/browse/JDK-8168921
WEBREV: http://cr.openjdk.java.net/~igerasim/8168921/00/webrev/

I also took a chance to perform some pico-optimizations around the 
modified code.

With kind regards,
Ivan


From ivan.gerasimov at oracle.com  Sun Oct 30 03:43:11 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Sun, 30 Oct 2016 06:43:11 +0300
Subject: [9] RFR: 8168923: Use unsigned random long in a temp directory name
Message-ID: <b1fb7b0b-0633-f502-9dfc-ed545b4431c0@oracle.com>

Hello everyone!

When File.createTempFile() is called, a random long value is generated, 
which is then made positive with Math.abs() and then used as a part of 
the directory name.

Instead of using Math.abs() and Long.toString(), it would be better to 
use Long.toUnsignedString():
1) no need to deal with the corner case of Long.MIN_VALUE,
2) increase the space of random values by the factor of 2.

Would you please help review the fix?

BUGURL: https://bugs.openjdk.java.net/browse/JDK-8168923
WEBREV: http://cr.openjdk.java.net/~igerasim/8168923/00/webrev/

With kind regards,
Ivan

From brian.burkhalter at oracle.com  Mon Oct 31 16:00:41 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Mon, 31 Oct 2016 09:00:41 -0700
Subject: [9] RFR: 8168923: Use unsigned random long in a temp directory
	name
In-Reply-To: <b1fb7b0b-0633-f502-9dfc-ed545b4431c0@oracle.com>
References: <b1fb7b0b-0633-f502-9dfc-ed545b4431c0@oracle.com>
Message-ID: <EAA48CF7-6140-4926-9F5D-F8F86E362B77@oracle.com>

Hello, Ivan,

Aside from the inevitable copyright year update this looks fine to me. What effect does it have on the regression test run?

Thanks,

Brian

On Oct 29, 2016, at 8:43 PM, Ivan Gerasimov <ivan.gerasimov at oracle.com> wrote:

> Hello everyone!
> 
> When File.createTempFile() is called, a random long value is generated, which is then made positive with Math.abs() and then used as a part of the directory name.
> 
> Instead of using Math.abs() and Long.toString(), it would be better to use Long.toUnsignedString():
> 1) no need to deal with the corner case of Long.MIN_VALUE,
> 2) increase the space of random values by the factor of 2.
> 
> Would you please help review the fix?
> 
> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8168923
> WEBREV: http://cr.openjdk.java.net/~igerasim/8168923/00/webrev/
> 
> With kind regards,
> Ivan


From joe.darcy at oracle.com  Mon Oct 31 17:11:44 2016
From: joe.darcy at oracle.com (joe darcy)
Date: Mon, 31 Oct 2016 10:11:44 -0700
Subject: [9] RFR: 8168921: Inconsistent Annotation.toString()
In-Reply-To: <82ad25c8-bf50-6842-3d1a-19a4e7e53bb6@oracle.com>
References: <82ad25c8-bf50-6842-3d1a-19a4e7e53bb6@oracle.com>
Message-ID: <1f7a2326-c39b-62e2-173d-9ab9d3df0ec8@oracle.com>

Hi Ivan,

The code change in src look fine, but please update the existing test

     test/java/lang/annotation/AnnotationToStringTest.java

rather than introducing a new test file.

Thanks,

-Joe


On 10/29/2016 6:42 PM, Ivan Gerasimov wrote:
> Hello!
>
> Suppose an annotation is of form
>
>     @interface MyAnnotation {
>         long value();
>     }
>
> Create the annotation object with 
> sun.reflect.annotation.AnnotationParser.annotationForMap(MyAnnotation.class, 
> Map.of("value", some_value)).
>
> Annotation.toString() method, called for the object, will produce a 
> string containing that 'some_value' with the optional 'L' suffix 
> appended.
> Normally, the suffix is not appended for values that can be 
> represented with int, and is appended otherwise.
> However, the logic does not hold for values Long.MIN_VALUE and 
> Integer.MIN_VALUE:  For the former the suffix is not appended and for 
> the later the suffix L is appended.
>
> Would you please help review the fix, which removes this tiny 
> inconsistency?
>
> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8168921
> WEBREV: http://cr.openjdk.java.net/~igerasim/8168921/00/webrev/
>
> I also took a chance to perform some pico-optimizations around the 
> modified code.
>
> With kind regards,
> Ivan
>


From Roger.Riggs at Oracle.com  Mon Oct 31 18:05:23 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 31 Oct 2016 14:05:23 -0400
Subject: RFR 9: 8158963 : RMI server-side multiplex protocol should be disabled
Message-ID: <e7305b40-9176-fcdb-da1d-7581acf3343d@Oracle.com>

The client side support for the RMI multiplex protocol was removed in 
1999, JDK 1.2.2  [1]
and is unused.

Please review a change to disable the server side RMI multiplex protocol 
by default
and provide a property to re-enable it if necessary.

Webrev:
http://cr.openjdk.java.net/~rriggs/webrev-disable-multiplex-8158963/
Issue:
   https://bugs.openjdk.java.net/browse/JDK-8158963

Thanks, Roger

[1]  https://bugs.openjdk.java.net/browse/JDK-4183204:  client-side 
multiplexing should be disabled


From brian.burkhalter at oracle.com  Mon Oct 31 18:23:46 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Mon, 31 Oct 2016 11:23:46 -0700
Subject: [9] RFR: 8168923: Use unsigned random long in a temp directory
	name
In-Reply-To: <EAA48CF7-6140-4926-9F5D-F8F86E362B77@oracle.com>
References: <b1fb7b0b-0633-f502-9dfc-ed545b4431c0@oracle.com>
	<EAA48CF7-6140-4926-9F5D-F8F86E362B77@oracle.com>
Message-ID: <8DFC271F-F855-47FC-AEBE-1DB1E8B3BD85@oracle.com>

One more thing ?

In [1] lines 58-59 could be corrected in the same way.

Thanks,

Brian

[1] http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/0086eb10182b/src/java.base/share/classes/java/nio/file/TempFileHelper.java

On Oct 31, 2016, at 9:00 AM, Brian Burkhalter <brian.burkhalter at oracle.com> wrote:

> Hello, Ivan,
> 
> Aside from the inevitable copyright year update this looks fine to me. What effect does it have on the regression test run?
> 
> Thanks,
> 
> Brian
> 
> On Oct 29, 2016, at 8:43 PM, Ivan Gerasimov <ivan.gerasimov at oracle.com> wrote:
> 
>> Hello everyone!
>> 
>> When File.createTempFile() is called, a random long value is generated, which is then made positive with Math.abs() and then used as a part of the directory name.
>> 
>> Instead of using Math.abs() and Long.toString(), it would be better to use Long.toUnsignedString():
>> 1) no need to deal with the corner case of Long.MIN_VALUE,
>> 2) increase the space of random values by the factor of 2.
>> 
>> Would you please help review the fix?
>> 
>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8168923
>> WEBREV: http://cr.openjdk.java.net/~igerasim/8168923/00/webrev/
>> 
>> With kind regards,
>> Ivan
> 


From stuart.marks at oracle.com  Mon Oct 31 19:28:46 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 31 Oct 2016 12:28:46 -0700
Subject: RFR 9: 8158963 : RMI server-side multiplex protocol should be
	disabled
In-Reply-To: <e7305b40-9176-fcdb-da1d-7581acf3343d@Oracle.com>
References: <e7305b40-9176-fcdb-da1d-7581acf3343d@Oracle.com>
Message-ID: <12ff3300-f54e-4da5-cc7a-db1c0ed4988e@oracle.com>

Hi Roger,

The change looks good.

Thanks,

s'marks

On 10/31/16 11:05 AM, Roger Riggs wrote:
> The client side support for the RMI multiplex protocol was removed in 1999, JDK
> 1.2.2  [1]
> and is unused.
>
> Please review a change to disable the server side RMI multiplex protocol by default
> and provide a property to re-enable it if necessary.
>
> Webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-disable-multiplex-8158963/
> Issue:
>   https://bugs.openjdk.java.net/browse/JDK-8158963
>
> Thanks, Roger
>
> [1]  https://bugs.openjdk.java.net/browse/JDK-4183204:  client-side multiplexing
> should be disabled
>

From xueming.shen at oracle.com  Mon Oct 31 20:26:15 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Mon, 31 Oct 2016 13:26:15 -0700
Subject: RFR 9: JDK-8168862:Tighten permissions granted to the jdk.zipfs module
Message-ID: <5817A8E7.2070203@oracle.com>

Please help review the change for

issue: https://bugs.openjdk.java.net/browse/JDK-8168862
webre: http://cr.openjdk.java.net/~sherman/8168862

The change is to tighten the permission to read "os.name" only.

Thanks,
Xueming

From lance.andersen at oracle.com  Mon Oct 31 20:27:54 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Mon, 31 Oct 2016 16:27:54 -0400
Subject: RFR 9: JDK-8168862:Tighten permissions granted to the jdk.zipfs
	module
In-Reply-To: <5817A8E7.2070203@oracle.com>
References: <5817A8E7.2070203@oracle.com>
Message-ID: <5DF984DA-EC25-4856-86AD-6EA4DF0FE160@oracle.com>

Looks fine
> On Oct 31, 2016, at 4:26 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
> 
> Please help review the change for
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8168862
> webre: http://cr.openjdk.java.net/~sherman/8168862
> 
> The change is to tighten the permission to read "os.name" only.
> 
> Thanks,
> Xueming

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From brian.burkhalter at oracle.com  Mon Oct 31 20:28:58 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Mon, 31 Oct 2016 13:28:58 -0700
Subject: RFR 9: JDK-8168862:Tighten permissions granted to the jdk.zipfs
	module
In-Reply-To: <5DF984DA-EC25-4856-86AD-6EA4DF0FE160@oracle.com>
References: <5817A8E7.2070203@oracle.com>
	<5DF984DA-EC25-4856-86AD-6EA4DF0FE160@oracle.com>
Message-ID: <B0E2C94E-4368-4C17-9104-0BED63672E93@oracle.com>

+1 

On Oct 31, 2016, at 1:27 PM, Lance Andersen <lance.andersen at oracle.com> wrote:

> Looks fine
>> On Oct 31, 2016, at 4:26 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
>> 
>> Please help review the change for
>> 
>> issue: https://bugs.openjdk.java.net/browse/JDK-8168862
>> webre: http://cr.openjdk.java.net/~sherman/8168862
>> 
>> The change is to tighten the permission to read "os.name" only.
>> 
>> Thanks,
>> Xueming
> 
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering 
> 1 Network Drive 
> Burlington, MA 01803
> Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>
> 
> 
> 


From mandy.chung at oracle.com  Mon Oct 31 20:32:43 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 31 Oct 2016 13:32:43 -0700
Subject: RFR 9: JDK-8168862:Tighten permissions granted to the jdk.zipfs
	module
In-Reply-To: <5817A8E7.2070203@oracle.com>
References: <5817A8E7.2070203@oracle.com>
Message-ID: <D3BC0C67-8997-4976-B567-5C15473814FF@oracle.com>

+1
Mandy

> On Oct 31, 2016, at 1:26 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
> 
> Please help review the change for
> 
> issue: https://bugs.openjdk.java.net/browse/JDK-8168862
> webre: http://cr.openjdk.java.net/~sherman/8168862
> 
> The change is to tighten the permission to read "os.name" only.
> 
> Thanks,
> Xueming


From sean.mullan at oracle.com  Mon Oct 31 20:56:19 2016
From: sean.mullan at oracle.com (Sean Mullan)
Date: Mon, 31 Oct 2016 16:56:19 -0400
Subject: RFR 9: JDK-8168862:Tighten permissions granted to the jdk.zipfs
	module
In-Reply-To: <D3BC0C67-8997-4976-B567-5C15473814FF@oracle.com>
References: <5817A8E7.2070203@oracle.com>
	<D3BC0C67-8997-4976-B567-5C15473814FF@oracle.com>
Message-ID: <5660258f-fb68-7458-b38e-b34e4cda447b@oracle.com>

It would be good to add or modify existing test(s) to run with a 
Security Manager, if there is not one already.

--Sean

On 10/31/16 4:32 PM, Mandy Chung wrote:
> +1
> Mandy
>
>> On Oct 31, 2016, at 1:26 PM, Xueming Shen <xueming.shen at oracle.com> wrote:
>>
>> Please help review the change for
>>
>> issue: https://bugs.openjdk.java.net/browse/JDK-8168862
>> webre: http://cr.openjdk.java.net/~sherman/8168862
>>
>> The change is to tighten the permission to read "os.name" only.
>>
>> Thanks,
>> Xueming
>

From paul.sandoz at oracle.com  Mon Oct 31 21:47:06 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 31 Oct 2016 14:47:06 -0700
Subject: RFR 8167974 MethodHandles.iteratedLoop(...) fails with CCE in the
	case of iterating over array
Message-ID: <6A383F19-410E-495E-96DB-BC747C9FDB8E@oracle.com>

Hi,

And? 8167966 MethodHandles.iteratedLoop fails with IAE in the case of correct arguments.

The two issues are closely intertwined in terms of the fix. Please review:

  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8167974-mhs-iterated-loop-no-arrays/webrev/

This is cleaning up previous specification and implementation assumptions when a null iterator method handle is passed to MethodHandles.iteratedLoop.

The specification states that an Iterator type *or* array type is supported. Originally I think the intention was to support the equivalent of Java for-each loops, but the implementation only supports Iterator. We can revisit this in the next release if necessary.

The implementation checking the constraints on the iterator loop method handles required some reorgnaization to clearly differentiate the checks performed if the iterator method handle is non-null or null, since it currently does not fully differentiate between the two cases leading to incorrect exceptions when the handle is non-null.

Thanks,
Paul.

From paul.sandoz at oracle.com  Mon Oct 31 21:58:29 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 31 Oct 2016 14:58:29 -0700
Subject: 8168841 The JavaDoc of java.util.stream.Collectors method
	collectingAndThen has incorrect code snippet
Message-ID: <3D33F959-13F9-4322-8AAA-B6A7012F28B8@oracle.com>

Hi,

Please review the following JavDoc fix for j.u.stream.Collectors:: collectingAndThen.

Thanks,
Paul.

diff -r 2e076c7e72d6 src/java.base/share/classes/java/util/stream/Collectors.java
--- a/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:51:23 2016 -0700
+++ b/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:55:16 2016 -0700
@@ -508,7 +508,7 @@
      * transformation.  For example, one could adapt the {@link #toList()}
      * collector to always produce an immutable list with:
      * <pre>{@code
-     *     List<String> people
+     *     List<String> list
      *         = people.stream().collect(collectingAndThen(toList(), Collections::unmodifiableList));
      * }</pre>
      *

From stuart.marks at oracle.com  Mon Oct 31 22:04:53 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Mon, 31 Oct 2016 15:04:53 -0700
Subject: 8168841 The JavaDoc of java.util.stream.Collectors method
	collectingAndThen has incorrect code snippet
In-Reply-To: <3D33F959-13F9-4322-8AAA-B6A7012F28B8@oracle.com>
References: <3D33F959-13F9-4322-8AAA-B6A7012F28B8@oracle.com>
Message-ID: <f520b9e3-6f37-1916-a28e-0aa6ba24fd71@oracle.com>

+1

On 10/31/16 2:58 PM, Paul Sandoz wrote:
> Hi,
>
> Please review the following JavDoc fix for j.u.stream.Collectors:: collectingAndThen.
>
> Thanks,
> Paul.
>
> diff -r 2e076c7e72d6 src/java.base/share/classes/java/util/stream/Collectors.java
> --- a/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:51:23 2016 -0700
> +++ b/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:55:16 2016 -0700
> @@ -508,7 +508,7 @@
>       * transformation.  For example, one could adapt the {@link #toList()}
>       * collector to always produce an immutable list with:
>       * <pre>{@code
> -     *     List<String> people
> +     *     List<String> list
>       *         = people.stream().collect(collectingAndThen(toList(), Collections::unmodifiableList));
>       * }</pre>
>       *
>

From lance.andersen at oracle.com  Mon Oct 31 22:07:01 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Mon, 31 Oct 2016 18:07:01 -0400
Subject: 8168841 The JavaDoc of java.util.stream.Collectors method
	collectingAndThen has incorrect code snippet
In-Reply-To: <3D33F959-13F9-4322-8AAA-B6A7012F28B8@oracle.com>
References: <3D33F959-13F9-4322-8AAA-B6A7012F28B8@oracle.com>
Message-ID: <980A814F-0C76-4AE1-87A8-FAAA84CECA92@oracle.com>

+1
> On Oct 31, 2016, at 5:58 PM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
> 
> Hi,
> 
> Please review the following JavDoc fix for j.u.stream.Collectors:: collectingAndThen.
> 
> Thanks,
> Paul.
> 
> diff -r 2e076c7e72d6 src/java.base/share/classes/java/util/stream/Collectors.java
> --- a/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:51:23 2016 -0700
> +++ b/src/java.base/share/classes/java/util/stream/Collectors.java	Mon Oct 31 14:55:16 2016 -0700
> @@ -508,7 +508,7 @@
>      * transformation.  For example, one could adapt the {@link #toList()}
>      * collector to always produce an immutable list with:
>      * <pre>{@code
> -     *     List<String> people
> +     *     List<String> list
>      *         = people.stream().collect(collectingAndThen(toList(), Collections::unmodifiableList));
>      * }</pre>
>      *

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>




From claes.redestad at oracle.com  Mon Oct 31 23:19:36 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 1 Nov 2016 00:19:36 +0100
Subject: RFR 8167974 MethodHandles.iteratedLoop(...) fails with CCE in the
	case of iterating over array
In-Reply-To: <6A383F19-410E-495E-96DB-BC747C9FDB8E@oracle.com>
References: <6A383F19-410E-495E-96DB-BC747C9FDB8E@oracle.com>
Message-ID: <5817D188.7080103@oracle.com>

Hi,

On 2016-10-31 22:47, Paul Sandoz wrote:
> Hi,
>
> And? 8167966 MethodHandles.iteratedLoop fails with IAE in the case of correct arguments.
>
> The two issues are closely intertwined in terms of the fix. Please review:
>
>    http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8167974-mhs-iterated-loop-no-arrays/webrev/

I think this imposes a reasonable restriction and nicely cleans up the
current specification. +1

I guess the if (!Iterable.class.isAssignableFrom(iterableType)) could
be moved inside the preceding else without losing readability.

Thanks!

/Claes

>
> This is cleaning up previous specification and implementation assumptions when a null iterator method handle is passed to MethodHandles.iteratedLoop.
>
> The specification states that an Iterator type *or* array type is supported. Originally I think the intention was to support the equivalent of Java for-each loops, but the implementation only supports Iterator. We can revisit this in the next release if necessary.
>
> The implementation checking the constraints on the iterator loop method handles required some reorgnaization to clearly differentiate the checks performed if the iterator method handle is non-null or null, since it currently does not fully differentiate between the two cases leading to incorrect exceptions when the handle is non-null.
>
> Thanks,
> Paul.
>