From brian.burkhalter at oracle.com Fri Jun 1 01:01:49 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 31 May 2018 18:01:49 -0700 Subject: 8201407: Files.move throws DirectoryNonEmptyException when moving directory across file system In-Reply-To: References: <11d848fb-4a9e-8edd-c60f-8a19acd77e59@oracle.com> <42F4C5D8-248A-47ED-A29A-0A9330540216@oracle.com> Message-ID: On May 25, 2018, at 7:05 AM, Brian Burkhalter wrote: > On May 25, 2018, at 6:03 AM, Alan Bateman wrote: > >> This version looks a lot better. I think the spec update is okay. >> >> The implementation changes would be cleaner if we add an isEmptyDir method. Also in the Windows implementation would be easier to read if you specify the filter as `e -> true`. > > I will look at the foregoing next week. An updated version is here: http://cr.openjdk.java.net/~bpb/8201407/webrev.02/ As ensureEmptyDir() method was added for each platform family. Note that the logic of this for UnixCopyFile has been updated especially at lines 381-393. This test passed on all platforms and further tests will be run. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Jun 3 16:18:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Jun 2018 17:18:39 +0100 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> Message-ID: <132b9ba3-1a81-de07-a659-92ed4d90a685@oracle.com> On 31/05/2018 15:52, David Lloyd wrote: > The testNioImplementation method covers IllegalArgumentException, but > I can add tests for CancelledKeyException easily... > Thanks. I've taken the patch from your last mail to check on a number of points and also to do a bit of tweaking. The javadoc is mostly okay. One thing that I may have missed in previous iterations is that you had interestOpsAnd declarig that it throws IllegalArgumentException. It doesn't of course as per the rational in the API note. Otherwise, I just moved the @apiNote and @implSpec to follow the method description and adjusted a few things to keep it as consistent with the existing spec and wording as possible. Tests for SelectionKey are in the SelectionKey rather than Selector directory so I've moved the test there. Also we don't need "SelectionKey" in the name once it is moved.? I've also refactored the test so that the same tests can be exercised for both the default and default provider implementations. For example, you added added tests to check for IAE and CKE but these aren't tested for the default implementation. The test used a ConnectionPair to get a connected SocketChannel but this isn't needed as an unconnected SocketChannel can be registered with a Selector. The patch changes the existing interestOps(int) to avoid queuing when the interest ops aren't changed. I was initially nervous about that for the case where a channel is initially registered with an interest set of 0. I think I've satisfied myself that this is safe. I've run the tests on all platforms and all the tests are passing so that helps the confidence on that part. I've put a webrev with the updated patch here: ?? http://cr.openjdk.java.net/~alanb/6350055/webrev/index.html If you are okay with these changes then the next step is the CSR. -Alan From philippe.marschall at gmail.com Sun Jun 3 16:48:04 2018 From: philippe.marschall at gmail.com (Philippe Marschall) Date: Sun, 3 Jun 2018 18:48:04 +0200 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> Message-ID: Hi Sorry if I step out of line here, I'm just a lurker. Personally I'd like to see @since 11 in the Javadoc. Cheers Philippe On Thu, May 31, 2018 at 4:55 PM, David Lloyd wrote: > And here [1] it is. > > [1] https://github.com/dmlloyd/openjdk/commit/atomic-nio-ops-v9 > > On Thu, May 31, 2018 at 9:52 AM, David Lloyd wrote: >> The testNioImplementation method covers IllegalArgumentException, but >> I can add tests for CancelledKeyException easily... >> >> On Thu, May 31, 2018 at 7:25 AM, Alan Bateman wrote: >>> >>> >>> On 30/05/2018 19:14, David Lloyd wrote: >>>> >>>> On Wed, May 23, 2018 at 9:00 AM, David Lloyd >>>> wrote: >>>>> >>>>> I can probably add tests to show that the ops work. I'm not so sure >>>>> that I can make a test that proves the atomicity of the operation >>>>> though. >>>> >>>> Updated with tests [1] & attached. >>>> >>> I quickly skimmed it and the test look reasonable. We should extend it to >>> cover the CancelledKeyException and IllegaArgumentException cases too, I >>> didn't spot anything testing these exceptions. >>> >>> -Alan >> >> >> >> -- >> - DML > > > > -- > - DML From Alan.Bateman at oracle.com Sun Jun 3 19:33:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 3 Jun 2018 20:33:23 +0100 Subject: 8201407: Files.move throws DirectoryNonEmptyException when moving directory across file system In-Reply-To: References: <11d848fb-4a9e-8edd-c60f-8a19acd77e59@oracle.com> <42F4C5D8-248A-47ED-A29A-0A9330540216@oracle.com> Message-ID: On 01/06/2018 02:01, Brian Burkhalter wrote: > > An updated version is here: > > http://cr.openjdk.java.net/~bpb/8201407/webrev.02/ > > > As ensureEmptyDir() method was added for each platform family. Note > that the logic of this for UnixCopyFile has been updated especially at > lines 381-393. This test passed on all platforms and further tests > will be run. > The javadoc and test update looks okay. The Windows implementation look okay too, just a minor nit in the comment where it says "is appropriate" instead of "not empty". The Unix implementation could be a lot simpler, can you try this instead: ?? static boolean isEmpty(UnixPath dir) throws IOException { ??????? try { ??????????? long ptr = opendir(dir); ??????????? try (UnixDirectoryStream stream = new UnixDirectoryStream(dir, ptr, e -> true)) { ??????????????? return !stream.iterator().hasNext(); ??????????? } ??????? } catch (UnixException e) { ??????????? e.rethrowAsIOException(dir); ??????????? return false; ??????? } ??? } -Alan From Alan.Bateman at oracle.com Mon Jun 4 08:38:25 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jun 2018 09:38:25 +0100 Subject: Callback Based Selectors In-Reply-To: <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> Message-ID: <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Any other comments on the API additions before I create the CSR? -Alan On 11/04/2018 09:27, Alan Bateman wrote: > I've re-based the patch that adds select(Consumer) and > friends. This follows the changes for JDK-8201315 so the action is > invoked while synchronized on the selector and the selected-key set, > not the key set for registrations. > > ?? http://cr.openjdk.java.net/~alanb/8199433/webrev/ > > There's a still a bit of work to do on the Solaris event port > implementation related to error recovery, also additional tests will > be needed, but otherwise I think all the interactions with the > existing spec have been worked out. > > -Alan From Alan.Bateman at oracle.com Mon Jun 4 08:45:10 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jun 2018 09:45:10 +0100 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> Message-ID: <00891c2e-c7bd-f6bf-48ba-672711960563@oracle.com> On 03/06/2018 17:48, Philippe Marschall wrote: > Hi > > Sorry if I step out of line here, I'm just a lurker. Personally I'd like to see > > @since 11 > > in the Javadoc. > Right, it will need @since tags. The first phase of rampdown for JDK 11 is in a few weeks so if it goes in before then they be 11, otherwise 12. -Alan From david.lloyd at redhat.com Mon Jun 4 12:49:43 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 4 Jun 2018 07:49:43 -0500 Subject: Callback Based Selectors In-Reply-To: <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Message-ID: Looks good to me! On Mon, Jun 4, 2018 at 3:38 AM, Alan Bateman wrote: > > Any other comments on the API additions before I create the CSR? > > -Alan > > > On 11/04/2018 09:27, Alan Bateman wrote: >> >> I've re-based the patch that adds select(Consumer) and >> friends. This follows the changes for JDK-8201315 so the action is invoked >> while synchronized on the selector and the selected-key set, not the key set >> for registrations. >> >> http://cr.openjdk.java.net/~alanb/8199433/webrev/ >> >> There's a still a bit of work to do on the Solaris event port >> implementation related to error recovery, also additional tests will be >> needed, but otherwise I think all the interactions with the existing spec >> have been worked out. >> >> -Alan > > -- - DML From david.lloyd at redhat.com Mon Jun 4 12:55:59 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 4 Jun 2018 07:55:59 -0500 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: <00891c2e-c7bd-f6bf-48ba-672711960563@oracle.com> References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> <00891c2e-c7bd-f6bf-48ba-672711960563@oracle.com> Message-ID: The updated patch looks OK. I had @since tags at one point (I think?) but I guess I lost them. At this point it's probably easier to just update the webrev than to start with me again just for two lines of JavaDoc? If I'm wrong about that let me know. On Mon, Jun 4, 2018 at 3:45 AM, Alan Bateman wrote: > > > On 03/06/2018 17:48, Philippe Marschall wrote: >> >> Hi >> >> Sorry if I step out of line here, I'm just a lurker. Personally I'd like >> to see >> >> @since 11 >> >> in the Javadoc. >> > Right, it will need @since tags. The first phase of rampdown for JDK 11 is > in a few weeks so if it goes in before then they be 11, otherwise 12. > > -Alan -- - DML From Alan.Bateman at oracle.com Mon Jun 4 15:21:47 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jun 2018 16:21:47 +0100 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> <00891c2e-c7bd-f6bf-48ba-672711960563@oracle.com> Message-ID: <218d044a-95c5-4be9-2e4f-334ec4b45b8c@oracle.com> On 04/06/2018 13:55, David Lloyd wrote: > The updated patch looks OK. I had @since tags at one point (I think?) > but I guess I lost them. At this point it's probably easier to just > update the webrev than to start with me again just for two lines of > JavaDoc? If I'm wrong about that let me know. That's fine, -Alan From brian.burkhalter at oracle.com Tue Jun 5 00:37:53 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 4 Jun 2018 17:37:53 -0700 Subject: 8201407: Files.move throws DirectoryNonEmptyException when moving directory across file system In-Reply-To: References: <11d848fb-4a9e-8edd-c60f-8a19acd77e59@oracle.com> <42F4C5D8-248A-47ED-A29A-0A9330540216@oracle.com> Message-ID: <75D50EC2-3779-4D52-A317-393E3F52D260@oracle.com> On Jun 3, 2018, at 12:33 PM, Alan Bateman wrote: > The Windows implementation look okay too, just a minor nit in the comment where it says "is appropriate" instead of "not empty?. So modified. > The Unix implementation could be a lot simpler, can you try this instead: That is in effect how I did it on Windows. I thought the more cumbersome Unix implementation might be a tad faster. I have modified the Unix implementation to be like the one I already had on Windows. http://cr.openjdk.java.net/~bpb/8201407/webrev.03/index.html Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Tue Jun 5 06:17:56 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 5 Jun 2018 06:17:56 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1341D16@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13961F3@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> Hi All, Here is the version #4 of the patch http://cr.openjdk.java.net/~ylu/8195160.04/ In this version, I have changed: 1. Remove jdk.net.RdmaSocketImplFactory. 2. Remove the inheritance from SocketChannelImpl/ServerSocketChannelImpl, replace it with SocketChannel/ServerSocketChannel. 3. Remove unnecessary setters from SocketImpl. 4. Override accept method to the ServerSocket returned by jdk.net.openServerSocket. 5. Add a small comment to PollSelectorImpl.poll method 6. Clean up a little in Lib-jdk.net.gmk I am still working on creating all the test cases. Do we need to replicate all the tests from both Socket/ServerSocket and SocketChannel/ServerSocketChannel, or we only need to pick up a subset of the cases? Please review the patch and let me know your feedback and comments. Thanks, Lucy >-----Original Message----- >From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, >Yingqi >Sent: Friday, May 11, 2018 9:38 AM >To: Alan Bateman ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: RE: adding rsockets support into JDK > >Hi Alan, > >Thank you very much for your feedback! > >I agree that jdk.net.RdmaSocketImplFactory is not necessary anymore. I will >remove it in next version. > >I will remove the inheritance from SocketChannelImpl/ServerSocketChannelImpl, >replace them with SocketChannel/ServerSocketChannel. My initial thought was to >avoid the code duplication, but I totally agree with you on your concern for >future maintenance. > >I will also make other changes following your suggestions and start to add test >cases. I will start from the examples of regular sockets and socket channels and >add ones specific to RDMA. > >Thank you!! >Lucy > >>-----Original Message----- >>From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >>Sent: Friday, May 11, 2018 5:18 AM >>To: Lu, Yingqi ; nio-dev at openjdk.java.net >>Cc: Viswanathan, Sandhya ; Aundhe, >>Shirish ; Kaczmarek, Eric >> >>Subject: Re: adding rsockets support into JDK >> >>On 04/05/2018 17:42, Lu, Yingqi wrote: >>> Hi All, >>> >>> The version 3 of the patch is available now at >>> http://cr.openjdk.java.net/~ylu/8195160.03/ >>> >>> In this version, I have changed following items: >>> >>> 1. Added Sockets.openRdmaSelector() function >>> >>> 2. Replaced "changing system-wide SocketImplFactory" with returning >>Socket(impl). I did the similar change to ServerSocket too. I also >>moved RdmaSocketImplFactory from jdk.net to rdma.ch. >>> >>> 3. Reduced number the factory methods to 2, openRdmaSocket() and >>openRdmaServerSocket(). >>> >>> 4. Removed the changes to InetAddress. >>> >>> 5. Removed the methods changed to public in Socket/ServerSocket >>> >>> Please let me know your feedback and comments. >>> >>I looked through the current version. >> >>The API additions to jdk.net.Sockets API look good. The javadoc will >>need to be fleshed out of course. >> >>Did you mean to include jdk.net.RdmaSocketImplFactory in the API? The >>factory methods can create the RdmaSocketImpl directly so it may not be >>needed. >> >>jdk.net.RdmaSocketOptions looks right. It will of course need javadoc >>and some refactoring so that hardcoded values aren't in shared code. >> >>There are a few changes to the java.net API that will need consideration. >>Having a protected constructor that takes a SocketImpl looks reasonable >>on first glance it is consistent with the protected constructor in Socket. >> >>The addition of protected setters to SocketImpl may be okay too >>although not strictly needed as the fd and address fields are protected >>so they can be set by implementations already. >> >>You've changed ServerSocket accept to support the RdmaSocketImpl but I >>don't think you need that. Instead, the ServerSocket returned by >>openServerSocket can override the accept method. I think this means you >>can drop the qualified export of jdk.net to java.base too. >> >>I'm a bit uncomfortable with extending SocketChannelImpl and >>ServerSocketChannelImpl as proposed because it tightly ties the RMDA >>implementation in jdk.net. It means we can't change anything in say >>SocketChannelImpl without breaking the RDMA implementation. Given that >>the RMDA implementations override almost everything then maybe they >>should just extend SocketChannel directly and we'll need to the >>duplication as needed. >> >>The extending of PollSelectorImpl is probably okay, just needs a >>comment to poll method to explain what it is protected. >> >>libextnet also builds on Solaris so the updates to Lib-jdk.net.gmk will >>need a few adjustments to ensure that it continues to build (you'll >>need to add - lrdmacm to LIBS_linux rather than LIBS for example). >> >>Are you planning to develop tests for this? We also have the issue how >>this is going to be tested and maintained over the long term. >> >>-Alan From Alan.Bateman at oracle.com Tue Jun 5 08:55:37 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 09:55:37 +0100 Subject: 8201407: Files.move throws DirectoryNonEmptyException when moving directory across file system In-Reply-To: <75D50EC2-3779-4D52-A317-393E3F52D260@oracle.com> References: <11d848fb-4a9e-8edd-c60f-8a19acd77e59@oracle.com> <42F4C5D8-248A-47ED-A29A-0A9330540216@oracle.com> <75D50EC2-3779-4D52-A317-393E3F52D260@oracle.com> Message-ID: <47d66fcc-7c5f-454c-299b-71240d48e1db@oracle.com> On 05/06/2018 01:37, Brian Burkhalter wrote: > > > That is in effect how I did it on Windows. I thought the more > cumbersome Unix implementation might be a tad faster. I have modified > the Unix implementation to be like the one I already had on Windows. > > http://cr.openjdk.java.net/~bpb/8201407/webrev.03/index.html > > This version looks okay. -Alan From Alan.Bateman at oracle.com Wed Jun 6 09:33:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 10:33:41 +0100 Subject: java.io.IOException: A non-blocking socket operation could not be completed immediately In-Reply-To: References: Message-ID: <9286beb4-7954-79ed-d384-642bb46ac482@oracle.com> Moving this mail to nio-dev. Simone - one thing to point out is that the SO_LINGER option is only specified for sockets that are configured blocking. Here's the relevant paragraph from the StandardSocketOption#SO_LINGER javadoc "This socket option is intended for use with sockets that are configured in blocking mode only. The behavior of the close method when this option is enabled on a non-blocking socket is not defined." The implementation has changed in JDK 11 so that SocketChannel close disables SO_LINGER when it's enabled with a value other than 0 and the channel is registered with a Selector. The eventual close will happen when the channel is flushed from the Selector (the stack trace in your bug report) but it won't hit this issue because SO_LINGER will be disabled. -Alan On 06/06/2018 09:35, Simone Bordet wrote: > Resend with proper subject, doh. > > ---------- Forwarded message ---------- > From: Simone Bordet > Date: Wed, Jun 6, 2018 at 10:29 AM > Subject: > To: "net-dev at openjdk.java.net >> OpenJDK Network Dev list" > > > > Hi, > > we have stumbled upon the error below on Windows. > > We suspect this is happening when closing sockets that have the linger > option set. > > Since SocketDispatcher.close0() eventually calls the Windows API > closesocket(), this seems to be confirmed by this article: > https://msdn.microsoft.com/en-us/library/windows/desktop/ms737582(v=vs.85).aspx > > I was wondering if this is a known issue, or anyone encountered this > error before. > > For more context, see also https://github.com/eclipse/jetty.project/issues/2468. > > Thanks ! > > > java.io.IOException: A non-blocking socket operation could not be > completed immediately > at sun.nio.ch.SocketDispatcher.close0(Native Method) > at sun.nio.ch.SocketDispatcher.close(SocketDispatcher.java:63) > at sun.nio.ch.SocketChannelImpl.kill(SocketChannelImpl.java:879) > at sun.nio.ch.WindowsSelectorImpl.implDereg(WindowsSelectorImpl.java:588) > at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:149) > at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:142) > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) > > -- > Simone Bordet > --- > Finally, no matter how good the architecture and design are, > to deliver bug-free software with optimal performance and reliability, > the implementation technique must be flawless. Victoria Livschitz From simone.bordet at gmail.com Wed Jun 6 12:04:29 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 6 Jun 2018 14:04:29 +0200 Subject: java.io.IOException: A non-blocking socket operation could not be completed immediately In-Reply-To: <9286beb4-7954-79ed-d384-642bb46ac482@oracle.com> References: <9286beb4-7954-79ed-d384-642bb46ac482@oracle.com> Message-ID: Hi, On Wed, Jun 6, 2018 at 11:33 AM, Alan Bateman wrote: > Moving this mail to nio-dev. > > Simone - one thing to point out is that the SO_LINGER option is only > specified for sockets that are configured blocking. Here's the relevant > paragraph from the StandardSocketOption#SO_LINGER javadoc > > "This socket option is intended for use with sockets that are configured in > blocking mode only. The behavior of the close method when this option is > enabled on a non-blocking socket is not defined." Ah, thanks for this pointer. > The implementation has changed in JDK 11 so that SocketChannel close > disables SO_LINGER when it's enabled with a value other than 0 and the > channel is registered with a Selector. The eventual close will happen when > the channel is flushed from the Selector (the stack trace in your bug > report) but it won't hit this issue because SO_LINGER will be disabled. Thanks for sharing the behavior change in JDK 11. -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From brian.burkhalter at oracle.com Wed Jun 6 19:26:53 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 6 Jun 2018 12:26:53 -0700 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: <132b9ba3-1a81-de07-a659-92ed4d90a685@oracle.com> References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> <132b9ba3-1a81-de07-a659-92ed4d90a685@oracle.com> Message-ID: <3DDB006C-71E1-4FF7-A014-9D239DC13150@oracle.com> On Jun 3, 2018, at 9:18 AM, Alan Bateman wrote: > I've put a webrev with the updated patch here: > http://cr.openjdk.java.net/~alanb/6350055/webrev/index.html Looks fine modulo three picayune items: 1) In SelectionKey I recommend putting a comma after ?e.g.? at line 245 and changing ?rom? to ?from? at line 246. 2) In AtomicUpdates at line 179 I think the indentation is one space too deep. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 7 06:38:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Jun 2018 07:38:56 +0100 Subject: [PATCH] 6350055: Atomic SelectionKey operations In-Reply-To: <3DDB006C-71E1-4FF7-A014-9D239DC13150@oracle.com> References: <3bd9bdad-5d83-2974-2644-c4400f36df7e@oracle.com> <132b9ba3-1a81-de07-a659-92ed4d90a685@oracle.com> <3DDB006C-71E1-4FF7-A014-9D239DC13150@oracle.com> Message-ID: On 06/06/2018 20:26, Brian Burkhalter wrote: > On Jun 3, 2018, at 9:18 AM, Alan Bateman > wrote: > >> I've put a webrev with the updated patch here: >> http://cr.openjdk.java.net/~alanb/6350055/webrev/index.html >> > > Looks fine modulo three picayune items: Thanks. The CSR [1] has been approved so I'll try to get this pushed on Thursday. > > 1) In SelectionKey I recommend putting a comma after ?e.g.? at line > 245 and changing ?rom? to ?from? at line 246. Okay. > > 2) In AtomicUpdates at line 179 I think the indentation is one space > too deep. > This is a try-with-resources with two resources. The second variable is just aligned with the first. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8204472 -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 7 08:14:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Jun 2018 09:14:39 +0100 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> Message-ID: <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> On 05/06/2018 07:17, Lu, Yingqi wrote: > Hi All, > > Here is the version #4 of the patch > http://cr.openjdk.java.net/~ylu/8195160.04/ > > In this version, I have changed: > > 1. Remove jdk.net.RdmaSocketImplFactory. > 2. Remove the inheritance from SocketChannelImpl/ServerSocketChannelImpl, replace it with SocketChannel/ServerSocketChannel. > 3. Remove unnecessary setters from SocketImpl. > 4. Override accept method to the ServerSocket returned by jdk.net.openServerSocket. > 5. Add a small comment to PollSelectorImpl.poll method > 6. Clean up a little in Lib-jdk.net.gmk > > I am still working on creating all the test cases. Do we need to replicate all the tests from both Socket/ServerSocket and SocketChannel/ServerSocketChannel, or we only need to pick up a subset of the cases? > > Please review the patch and let me know your feedback and comments. > For tests then I think we should have a basic set of tests that exercise each of the elements in the API. For each socket and channel type then each of the methods in the API should be exercised. There are several new socket options so they should be exercised too. So a good start is a maintainable set of unit tests. I don't think you should be concerned with duplicating all the existing tests as many of those tests are regression tests or stress specific areas where there were bugs in the past. In general, I think testing is going to be a challenge for this feature as it needs commitment to maintain the infrastructure needed to test this feature and then running the tests at each release. When we added the SDP support, it required maintaining a set of machines with Infiniband HCAs and running the tests on a regular basis. I assume something similar will be needed here. When developing the jtreg tests then it's important that don't fail on systems without RDMA hardware. For SDP, the tests checked for /proc/net/sdp and would pass silently when not present. I hope there is something easy to check for rsockets. One thing that isn't clear to me is whether communication between using sockets on the same machine, using an address plumbed to an RDMA interface, will use the network or not. I guess I'm asking if you are thinking about functional testing with two or more machines, I assume you will at least be doing some of that when getting performance data and comparing it to TCP or SDP connections. On the build changes, then one thing that would be useful to know is whether librdmacm is included in all distributions. If someone is building on say Ubuntu will they need to install an additional package installed. I bring this as we may need to do a bit more on the build to probe for the library. In other areas, at least in the past, we've had to resort to dynamic linking (dlopen/dlsym). I'm sure we should do that here, instead I'm just wondering whether the rsocket support will be compiled in when there it is not present on the build machine. I skimmed quickly through the webrev and I think it's looking quite good. RdmaNet defines only static methods so I don't think it needs to extends Net. There is more to do on the javadoc for the new methods, also RdmaSocketOptions. I can help out on that when the time comes. -Alan. From yingqi.lu at intel.com Thu Jun 7 18:10:46 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Thu, 7 Jun 2018 18:10:46 +0000 Subject: adding rsockets support into JDK In-Reply-To: <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> Hi Alan, Thank you very much for your guidance on the tests. This is very helpful. I will look into SDP test cases and use them as a reference. In addition, I tested rsocket client and server on the same machine for both sockets and socket channels. InetAddress.getLocalHost() is used as the address for both server and client. The tests pass. Using sar[1], I see CPU cores are busy, but network does not show traffic. I think it might due to how rsocket handles local traffic (rsocket does not use loopback device). I also tested the same programs on two different machines and I saw network traffic shown on the RDMA network card. Thanks, Lucy [1] https://linux.die.net/man/1/sar >-----Original Message----- >From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >Sent: Thursday, June 7, 2018 1:15 AM >To: Lu, Yingqi ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: Re: adding rsockets support into JDK > >On 05/06/2018 07:17, Lu, Yingqi wrote: >> Hi All, >> >> Here is the version #4 of the patch >> http://cr.openjdk.java.net/~ylu/8195160.04/ >> >> In this version, I have changed: >> >> 1. Remove jdk.net.RdmaSocketImplFactory. >> 2. Remove the inheritance from SocketChannelImpl/ServerSocketChannelImpl, >replace it with SocketChannel/ServerSocketChannel. >> 3. Remove unnecessary setters from SocketImpl. >> 4. Override accept method to the ServerSocket returned by >jdk.net.openServerSocket. >> 5. Add a small comment to PollSelectorImpl.poll method 6. Clean up a >> little in Lib-jdk.net.gmk >> >> I am still working on creating all the test cases. Do we need to replicate all the >tests from both Socket/ServerSocket and SocketChannel/ServerSocketChannel, >or we only need to pick up a subset of the cases? >> >> Please review the patch and let me know your feedback and comments. >> >For tests then I think we should have a basic set of tests that exercise each of the >elements in the API. For each socket and channel type then each of the methods >in the API should be exercised. There are several new socket options so they >should be exercised too. So a good start is a maintainable set of unit tests. I don't >think you should be concerned with duplicating all the existing tests as many of >those tests are regression tests or stress specific areas where there were bugs in >the past. > >In general, I think testing is going to be a challenge for this feature as it needs >commitment to maintain the infrastructure needed to test this feature and then >running the tests at each release. When we added the SDP support, it required >maintaining a set of machines with Infiniband HCAs and running the tests on a >regular basis. I assume something similar will be needed here. > >When developing the jtreg tests then it's important that don't fail on systems >without RDMA hardware. For SDP, the tests checked for /proc/net/sdp and would >pass silently when not present. I hope there is something easy to check for >rsockets. > >One thing that isn't clear to me is whether communication between using sockets >on the same machine, using an address plumbed to an RDMA interface, will use >the network or not. I guess I'm asking if you are thinking about functional testing >with two or more machines, I assume you will at least be doing some of that >when getting performance data and comparing it to TCP or SDP connections. > >On the build changes, then one thing that would be useful to know is whether >librdmacm is included in all distributions. If someone is building on say Ubuntu will >they need to install an additional package installed. I bring this as we may need to >do a bit more on the build to probe for the library. In other areas, at least in the >past, we've had to resort to dynamic linking (dlopen/dlsym). I'm sure we should >do that here, instead I'm just wondering whether the rsocket support will be >compiled in when there it is not present on the build machine. > >I skimmed quickly through the webrev and I think it's looking quite good. >RdmaNet defines only static methods so I don't think it needs to extends Net. >There is more to do on the javadoc for the new methods, also >RdmaSocketOptions. I can help out on that when the time comes. > >-Alan. > From brian.burkhalter at oracle.com Thu Jun 7 20:44:42 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Jun 2018 13:44:42 -0700 Subject: 8204576: (fs) java/nio/file/Files/CopyAndMove.java does not set isUnix = true on macOS Message-ID: <930421AE-626D-495D-ABFF-9B94C17ED9A4@oracle.com> Please review this test-only fix. https://bugs.openjdk.java.net/browse/JDK-8204576 http://cr.openjdk.java.net/~bpb/8204576/webrev.00/ Thanks, Brian From Alan.Bateman at oracle.com Thu Jun 7 20:49:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Jun 2018 21:49:42 +0100 Subject: 8204576: (fs) java/nio/file/Files/CopyAndMove.java does not set isUnix = true on macOS In-Reply-To: <930421AE-626D-495D-ABFF-9B94C17ED9A4@oracle.com> References: <930421AE-626D-495D-ABFF-9B94C17ED9A4@oracle.com> Message-ID: <66bc461a-d968-81f9-8f52-e2a73d7e4143@oracle.com> On 07/06/2018 21:44, Brian Burkhalter wrote: > Please review this test-only fix. > > https://bugs.openjdk.java.net/browse/JDK-8204576 > http://cr.openjdk.java.net/~bpb/8204576/webrev.00/ > The brace on L661 and L927 can be moved to the previous line, otherwise I guess it's okay (although it was deliberate in the original version to have isWindows and isUnix). -Alan From brian.burkhalter at oracle.com Thu Jun 7 20:57:33 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 7 Jun 2018 13:57:33 -0700 Subject: 8204576: (fs) java/nio/file/Files/CopyAndMove.java does not set isUnix = true on macOS In-Reply-To: <66bc461a-d968-81f9-8f52-e2a73d7e4143@oracle.com> References: <930421AE-626D-495D-ABFF-9B94C17ED9A4@oracle.com> <66bc461a-d968-81f9-8f52-e2a73d7e4143@oracle.com> Message-ID: <2760EFE5-2FCA-4990-8BB5-40CD10B6C1FD@oracle.com> On Jun 7, 2018, at 1:49 PM, Alan Bateman wrote: > On 07/06/2018 21:44, Brian Burkhalter wrote: >> Please review this test-only fix. >> >> https://bugs.openjdk.java.net/browse/JDK-8204576 >> http://cr.openjdk.java.net/~bpb/8204576/webrev.00/ >> > The brace on L661 and L927 can be moved to the previous line, Fixed. > otherwise I guess it's okay (although it was deliberate in the original version to have isWindows and isUnix). Unix == ! Windows seems to be used in many other tests. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Thu Jun 7 21:05:59 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Thu, 7 Jun 2018 21:05:59 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD146B34B@ORSMSX113.amr.corp.intel.com> Hi Alan, I checked on librdmacm availability on different Linux distros. It is at least available in Ubuntu, Arch Linux, Fedora/Centos/Red Hat, openSUSE and Gentoo. Thanks, Lucy >-----Original Message----- >From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, >Yingqi >Sent: Thursday, June 7, 2018 11:11 AM >To: Alan Bateman ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: RE: adding rsockets support into JDK > >Hi Alan, > >Thank you very much for your guidance on the tests. This is very helpful. > >I will look into SDP test cases and use them as a reference. > >In addition, I tested rsocket client and server on the same machine for both >sockets and socket channels. InetAddress.getLocalHost() is used as the address >for both server and client. The tests pass. Using sar[1], I see CPU cores are busy, >but network does not show traffic. I think it might due to how rsocket handles >local traffic (rsocket does not use loopback device). I also tested the same >programs on two different machines and I saw network traffic shown on the >RDMA network card. > >Thanks, >Lucy > >[1] https://linux.die.net/man/1/sar > > >>-----Original Message----- >>From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >>Sent: Thursday, June 7, 2018 1:15 AM >>To: Lu, Yingqi ; nio-dev at openjdk.java.net >>Cc: Viswanathan, Sandhya ; Aundhe, >>Shirish ; Kaczmarek, Eric >> >>Subject: Re: adding rsockets support into JDK >> >>On 05/06/2018 07:17, Lu, Yingqi wrote: >>> Hi All, >>> >>> Here is the version #4 of the patch >>> http://cr.openjdk.java.net/~ylu/8195160.04/ >>> >>> In this version, I have changed: >>> >>> 1. Remove jdk.net.RdmaSocketImplFactory. >>> 2. Remove the inheritance from >>> SocketChannelImpl/ServerSocketChannelImpl, >>replace it with SocketChannel/ServerSocketChannel. >>> 3. Remove unnecessary setters from SocketImpl. >>> 4. Override accept method to the ServerSocket returned by >>jdk.net.openServerSocket. >>> 5. Add a small comment to PollSelectorImpl.poll method 6. Clean up a >>> little in Lib-jdk.net.gmk >>> >>> I am still working on creating all the test cases. Do we need to >>> replicate all the >>tests from both Socket/ServerSocket and >>SocketChannel/ServerSocketChannel, >>or we only need to pick up a subset of the cases? >>> >>> Please review the patch and let me know your feedback and comments. >>> >>For tests then I think we should have a basic set of tests that >>exercise each of the elements in the API. For each socket and channel >>type then each of the methods in the API should be exercised. There are >>several new socket options so they should be exercised too. So a good >>start is a maintainable set of unit tests. I don't think you should be >>concerned with duplicating all the existing tests as many of those >>tests are regression tests or stress specific areas where there were bugs in the >past. >> >>In general, I think testing is going to be a challenge for this feature >>as it needs commitment to maintain the infrastructure needed to test >>this feature and then running the tests at each release. When we added >>the SDP support, it required maintaining a set of machines with >>Infiniband HCAs and running the tests on a regular basis. I assume something >similar will be needed here. >> >>When developing the jtreg tests then it's important that don't fail on >>systems without RDMA hardware. For SDP, the tests checked for >>/proc/net/sdp and would pass silently when not present. I hope there is >>something easy to check for rsockets. >> >>One thing that isn't clear to me is whether communication between using >>sockets on the same machine, using an address plumbed to an RDMA >>interface, will use the network or not. I guess I'm asking if you are >>thinking about functional testing with two or more machines, I assume >>you will at least be doing some of that when getting performance data and >comparing it to TCP or SDP connections. >> >>On the build changes, then one thing that would be useful to know is >>whether librdmacm is included in all distributions. If someone is >>building on say Ubuntu will they need to install an additional package >>installed. I bring this as we may need to do a bit more on the build to >>probe for the library. In other areas, at least in the past, we've had >>to resort to dynamic linking (dlopen/dlsym). I'm sure we should do that >>here, instead I'm just wondering whether the rsocket support will be compiled in >when there it is not present on the build machine. >> >>I skimmed quickly through the webrev and I think it's looking quite good. >>RdmaNet defines only static methods so I don't think it needs to extends Net. >>There is more to do on the javadoc for the new methods, also >>RdmaSocketOptions. I can help out on that when the time comes. >> >>-Alan. >> From mark.reinhold at oracle.com Fri Jun 8 16:09:31 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 8 Jun 2018 09:09:31 -0700 (PDT) Subject: New candidate JEP: 337: RDMA Sockets Message-ID: <20180608160931.BE2241DC7AC@eggemoggin.niobe.net> http://openjdk.java.net/jeps/337 - Mark From yingqi.lu at intel.com Fri Jun 8 16:38:57 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 8 Jun 2018 16:38:57 +0000 Subject: New candidate JEP: 337: RDMA Sockets In-Reply-To: <20180608160931.BE2241DC7AC@eggemoggin.niobe.net> References: <20180608160931.BE2241DC7AC@eggemoggin.niobe.net> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD146C6A8@ORSMSX113.amr.corp.intel.com> Big "thank you" to Alan, Chris, Paul and Mark for your help reviewing the JEP! Thanks, Lucy >-----Original Message----- >From: mark.reinhold at oracle.com [mailto:mark.reinhold at oracle.com] >Sent: Friday, June 8, 2018 9:10 AM >To: Lu, Yingqi >Cc: nio-dev at openjdk.java.net >Subject: New candidate JEP: 337: RDMA Sockets > >http://openjdk.java.net/jeps/337 > >- Mark From huizhe.wang at oracle.com Tue Jun 12 16:52:09 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 12 Jun 2018 09:52:09 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> Message-ID: <5B1FFA39.9000704@oracle.com> Hi all, It's been a while since the 1st round of reviews. Thank you all for the valuable comments and suggestions! Note that Roger's last response went to nio-dev, but not core-libs-dev, I've therefore attached it below. CSR [2]: the CSR is now approved. Note the write method has been changed to writeString. Impl [3]: for performance reason and the different requirement from byte-string conversion for malformed/unmappable character handing, the implementation uses a specific method separate from the existing ones. Both Sherman and Alan preferred specific method than adding parameters to the APIs that might be error prone. Thanks Sherman for the code! JMH benchmark tests showed the new read method performs better than an operation that reads the bytes and convert to string. The gains start to be significant even at quite reasonable sizes (< 10K). [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ Please review. Thanks, Joe On 5/15/18, 7:51 AM, Roger Riggs wrote: > Hi Joe, et. al. > > My $.02 on line separators: > > - We should avoid clever tricks trying to solve problems that are > infrequent such as cross OS file line operations. > Most files will be read/written on a single system so the line > separators will be as expected. > > - Have/use APIs that split lines consistently accepting both line > separators so developer code can be agnostic to line separators. > aka BufferedReader.readLine for developers that are processing the > contents *as lines*. > Those other methods already exist; if there are any gaps in line > oriented processing that's a separate task. > > - These new file methods are defined to handle Charset > encoding/decoding and buffering. > Since there are other methods to deal with files as lines these > methods should not look for or break to lines. > > - Performance: adding code to look for line characters will slow it > down and in most cases would have no impact > since the line endings will match the system. > > - The strings passed to writeString (CharSequence) should have been > constructed using the proper line separators. > There are any number of methods that insert the os specific line > terminator and its easy to write File.separator as needed. > > As for the method naming: > > I'd prefer "writeString" as a familiar term that isn't stretched too > far by making the argument be a CharSequence. > its fine to call a CharSequence a string (with a lower case s). > > $.02, Roger > > > On 4/27/18 6:33 PM, Joe Wang wrote: >> >> >> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>> ----- Mail original ----- >>>> De: "Joe Wang" >>>> ?: "Remi Forax" , "Alan Bateman" >>>> >>>> Cc: "nio-dev" , "core-libs-dev" >>>> >>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>> reading/writing a string from/to a file >>>> Hi R?mi, Alan, >>> Hi Joe, >>> >>>> I'm not sure we'd want to replace system dependent line separators >>>> with >>>> '\n'. There are cases as you described where the replacement makes >>>> sense. However, there are other cases too. For example, the >>>> purpose is >>>> to read, edit a text file and then write it back. If this is on >>>> Windows, >>>> and if line separators are replaced with '\n', that would cause the >>>> resulting file not formatted properly in for example Notepad. There >>>> are >>>> cases where users write text files on Windows using Java, and only >>>> found >>>> the lines were not separated in Notepad. >>> I agree that why the counterpart of readString() that write the >>> string should inserts the platform's line separator. >>> BTW, i just found that those methods are not named writeString, or >>> writeCharSequence, i think they should. >> >> While readString() does not modify the original content (e.g. by >> replacing the platform's line separator with '\n'), write(String) >> won't either, by adding extra characters such as the line separator. >> >> I would think interfaces shall read along with the parameters. >> readString(Path) == read as a String from the specified Path (one >> could argue for readToString, readAsString, but we generally omit the >> preps) >> write(Path, CharSequence) == write the CharSequence to the file, >> since CharSequence is already in the method signature as a parameter, >> we probably don't want to add that to the name, otherwise it would >> read like repeating the word CharSequence. >> >> It is in a similar situation as write(Path, Iterable) where it was >> defined as writeLines(Path, Iterable). >> >>> >>>> Files.write(Path, Iterable) is also specified to terminate each line >>>> with the platform's line separator. If readString does the >>>> replacement, >>>> it would be inconsistent. >>> >>> Anyway, if we look for consistency the methods writeCharSequence >>> should transform the \n in the CharSequence to the platform's line >>> separator. >>> >>> Files.write(Path, Iterable) is is not a counterpart of readString(), >>> it's consistent with Files.lines() or Files.readLines() (or >>> BufferedReader.readLine()) that all suppress the line separators. >>> Anyway, Files.write(path, readString(path)::line) will be consistent >>> if you replace the line separators or not because String.line() >>> suppresses the line separators. >> >> readString pairs with write(String), therefore it's more like >> Files.write(path, readString(path)) than readString(path)::line. The >> use case: >> >> String s = Files.read(path); >> Files.write(path, s.replace("/config/location", "/new/location")); >> >> would then work as expected. >> >> These two methods are one-off (open/read/write/close file) operation. >> write(String) therefore is not intended for adding/appending multiple >> Strings from other operations such as String.line(). If an app needs >> to put the result of String.line() or any other processes into a file >> using this method, it needs to take care of adding the line separator >> itself when necessary. "when necessary" would be a judgement call by >> the developer. >> >> That said, if there's a consensus on the idea of terminating the >> string with a line separator, then readString method would need to >> strip it off to go with the write method. >> >>> >>>> I would think readString behaves like readAllBytes, that would >>>> simply read all content faithfully. >>> readString does an interpolation due to the Charset so it's not the >>> real content, again, the idea is that developers should not have to >>> care about the platform's line separators (they have more important >>> things to think). >>> >>> The other solution is to just not introduce those new methods, after >>> all Files.lines().collect(Collectors.joining("\n")) already does the >>> job, no ? >> >> While there are many ways to do it, none is as straight-forward. >> "Read (entire) file to String"/"Write String to file" are popular >> requests from users. Read to string -> do some String manipulation -> >> write it back is such a simple use case, it really needs to be very >> easy to do as illustrated in the above code snippet. >> >> Cheers, >> Joe >> >>> >>>> Best, >>>> Joe >>> regards, >>> R?mi >>> >>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>> Hi Alan, >>>>> >>>>> People do not read the documentation. >>>>> So adding something in the documentation about when a method >>>>> should be used or >>>>> not is never a solution. >>>>> >>>>> Here the user want a String and provides a charset so you have no >>>>> way but to >>>>> decode the content to substitute the line separator. >>>>> >>>>> cheers, >>>>> R?mi >>>>> >>>>> ----- Mail original ----- >>>>>> De: "Alan Bateman" >>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>> I think that having a readString that includes OS dependent line >>>>>>> separators is a >>>>>>> mistake, >>>>>>> Java does a great job to try to shield the developer from the >>>>>>> kind of things >>>>>>> that makes programs behave differently on different platforms. >>>>>>> >>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>> people will do a call >>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>> hard way. >>>>>>> >>>>>>> raw string literal does the same substitution for the same reason. >>>>>>> >>>>>> Yes, there are several discussion points around this and somewhat >>>>>> timely >>>>>> with multi-string support. >>>>>> >>>>>> One thing that I think Joe will need in this API is some note to >>>>>> make it >>>>>> clearer what the intended usage is (as I think the intention is >>>>>> simple >>>>>> cases with mostly single lines of text). >>>>>> >>>>>> -Alan. >> > From Roger.Riggs at Oracle.com Tue Jun 12 17:37:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 13:37:56 -0400 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Hi Joe, Looks good to me. Typo: StringCoding.java:1026 "unmappble"? (no new webrev needed) Regards, Roger On 6/12/2018 12:52 PM, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. > > Thanks, > Joe > > On 5/15/18, 7:51 AM, Roger Riggs wrote: >> Hi Joe, et. al. >> >> My $.02 on line separators: >> >> ?- We should avoid clever tricks trying to solve problems that are >> infrequent such as cross OS file line operations. >> ??? Most files will be read/written on a single system so the line >> separators will be as expected. >> >> ?- Have/use APIs that split lines consistently accepting both line >> separators so developer code can be agnostic to line separators. >> ??? aka BufferedReader.readLine for developers that are processing >> the contents *as lines*. >> ?? Those other methods already exist; if there are any gaps in line >> oriented processing that's a separate task. >> >> ?- These new file methods are defined to handle Charset >> encoding/decoding and buffering. >> ??? Since there are other methods to deal with files as lines these >> methods should not look for or break to lines. >> >> ?- Performance: adding code to look for line characters will slow it >> down and in most cases would have no impact >> ??? since the line endings will match the system. >> >> ?- The strings passed to writeString (CharSequence) should have been >> constructed using the proper line separators. >> ??? There are any number of methods that insert the os specific line >> terminator and its easy to write File.separator as needed. >> >> As for the method naming: >> >> I'd prefer "writeString" as a familiar term that isn't stretched too >> far by making the argument be a CharSequence. >> its fine to call a CharSequence a string (with a lower case s). >> >> $.02, Roger >> >> >> On 4/27/18 6:33 PM, Joe Wang wrote: >>> >>> >>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>> ----- Mail original ----- >>>>> De: "Joe Wang" >>>>> ?: "Remi Forax" , "Alan Bateman" >>>>> >>>>> Cc: "nio-dev" , "core-libs-dev" >>>>> >>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>> reading/writing a string from/to a file >>>>> Hi R?mi, Alan, >>>> Hi Joe, >>>> >>>>> I'm not sure we'd want to replace system dependent line separators >>>>> with >>>>> '\n'. There are cases as you described where the replacement makes >>>>> sense. However,? there are other cases too. For example, the >>>>> purpose is >>>>> to read, edit a text file and then write it back. If this is on >>>>> Windows, >>>>> and if line separators are replaced with '\n', that would cause the >>>>> resulting file not formatted properly in for example Notepad. >>>>> There are >>>>> cases where users write text files on Windows using Java, and only >>>>> found >>>>> the lines were not separated in Notepad. >>>> I agree that why the counterpart of readString() that write the >>>> string should inserts the platform's line separator. >>>> BTW, i just found that those methods are not named writeString, or >>>> writeCharSequence, i think they should. >>> >>> While readString() does not modify the original content (e.g. by >>> replacing the platform's line separator with '\n'), write(String) >>> won't either, by adding extra characters such as the line separator. >>> >>> I would think interfaces shall read along with the parameters. >>> ??? readString(Path) == read as a String from the specified Path >>> (one could argue for readToString, readAsString, but we generally >>> omit the preps) >>> ??? write(Path, CharSequence) == write the CharSequence to the file, >>> since CharSequence is already in the method signature as a >>> parameter, we probably don't want to add that to the name, otherwise >>> it would read like repeating the word CharSequence. >>> >>> It is in a similar situation as write(Path, Iterable) where it was >>> defined as writeLines(Path, Iterable). >>> >>>> >>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>> with the platform's line separator. If readString does the >>>>> replacement, >>>>> it would be inconsistent. >>>> >>>> Anyway, if we look for consistency the methods writeCharSequence >>>> should transform the \n in the CharSequence to the platform's line >>>> separator. >>>> >>>> Files.write(Path, Iterable) is is not a counterpart of >>>> readString(), it's consistent with Files.lines() or >>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>> the line separators. Anyway, Files.write(path, >>>> readString(path)::line) will be consistent if you replace the line >>>> separators or not because String.line() suppresses the line >>>> separators. >>> >>> readString pairs with write(String), therefore it's more like >>> Files.write(path, readString(path)) than readString(path)::line. The >>> use case: >>> >>> ??? String s = Files.read(path); >>> ??? Files.write(path, s.replace("/config/location", "/new/location")); >>> >>> would then work as expected. >>> >>> These two methods are one-off (open/read/write/close file) >>> operation. write(String) therefore is not intended for >>> adding/appending multiple Strings from other operations such as >>> String.line(). If an app needs to put the result of String.line() or >>> any other processes into a file using this method, it needs to take >>> care of adding the line separator itself when necessary. "when >>> necessary" would be a judgement call by the developer. >>> >>> That said, if there's a consensus on the idea of terminating the >>> string with a line separator, then readString method would need to >>> strip it off to go with the write method. >>> >>>> >>>>> I would think readString behaves like readAllBytes, that would >>>>> simply read all content faithfully. >>>> readString does an interpolation due to the Charset so it's not the >>>> real content, again, the idea is that developers should not have to >>>> care about the platform's line separators (they have more important >>>> things to think). >>>> >>>> The other solution is to just not introduce those new methods, >>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>> does the job, no ? >>> >>> While there are many ways to do it, none is as straight-forward. >>> "Read (entire) file to String"/"Write String to file" are popular >>> requests from users. Read to string -> do some String manipulation >>> -> write it back is such a simple use case, it really needs to be >>> very easy to do as illustrated in the above code snippet. >>> >>> Cheers, >>> Joe >>> >>>> >>>>> Best, >>>>> Joe >>>> regards, >>>> R?mi >>>> >>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>> Hi Alan, >>>>>> >>>>>> People do not read the documentation. >>>>>> So adding something in the documentation about when a method >>>>>> should be used or >>>>>> not is never a solution. >>>>>> >>>>>> Here the user want a String and provides a charset so you have no >>>>>> way but to >>>>>> decode the content to substitute the line separator. >>>>>> >>>>>> cheers, >>>>>> R?mi >>>>>> >>>>>> ----- Mail original ----- >>>>>>> De: "Alan Bateman" >>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>> >>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>> >>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>> reading/writing a string from/to a file >>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>> I think that having a readString that includes OS dependent >>>>>>>> line separators is a >>>>>>>> mistake, >>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>> kind of things >>>>>>>> that makes programs behave differently on different platforms. >>>>>>>> >>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>> people will do a call >>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>> hard way. >>>>>>>> >>>>>>>> raw string literal does the same substitution for the same reason. >>>>>>>> >>>>>>> Yes, there are several discussion points around this and >>>>>>> somewhat timely >>>>>>> with multi-string support. >>>>>>> >>>>>>> One thing that I think Joe will need in this API is some note to >>>>>>> make it >>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>> simple >>>>>>> cases with mostly single lines of text). >>>>>>> >>>>>>> -Alan. >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jun 12 22:00:24 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 12 Jun 2018 15:00:24 -0700 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java Message-ID: https://bugs.openjdk.java.net/browse/JDK-8204915 http://cr.openjdk.java.net/~bpb/8204915/webrev.00/ Please review this simple, test-only change. Thanks, Brian From brent.christian at oracle.com Tue Jun 12 23:13:22 2018 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 12 Jun 2018 16:13:22 -0700 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java In-Reply-To: References: Message-ID: <645a0013-01bc-6be7-cdcc-58ffb1f54526@oracle.com> Looks fine to me, Brian. -Brent On 6/12/18 3:00 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8204915 > http://cr.openjdk.java.net/~bpb/8204915/webrev.00/ > > Please review this simple, test-only change. > > Thanks, > > Brian > From huizhe.wang at oracle.com Wed Jun 13 19:52:30 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 13 Jun 2018 12:52:30 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Message-ID: <5B2175FE.8040500@oracle.com> Thanks Roger! I pushed the changeset after s/unmappble/unmappable, it found three of them :-) Best, Joe On 6/12/18, 10:37 AM, Roger Riggs wrote: > Hi Joe, > > Looks good to me. > > Typo: StringCoding.java:1026 "unmappble" (no new webrev needed) > > Regards, Roger > > > On 6/12/2018 12:52 PM, Joe Wang wrote: >> Hi all, >> >> It's been a while since the 1st round of reviews. Thank you all for >> the valuable comments and suggestions! Note that Roger's last >> response went to nio-dev, but not core-libs-dev, I've therefore >> attached it below. >> >> CSR [2]: the CSR is now approved. Note the write method has been >> changed to writeString. >> >> Impl [3]: for performance reason and the different requirement from >> byte-string conversion for malformed/unmappable character handing, >> the implementation uses a specific method separate from the existing >> ones. Both Sherman and Alan preferred specific method than adding >> parameters to the APIs that might be error prone. Thanks Sherman for >> the code! >> >> JMH benchmark tests showed the new read method performs better than >> an operation that reads the bytes and convert to string. The gains >> start to be significant even at quite reasonable sizes (< 10K). >> >> >> [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 >> [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 >> [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ >> >> Please review. >> >> Thanks, >> Joe >> >> On 5/15/18, 7:51 AM, Roger Riggs wrote: >>> Hi Joe, et. al. >>> >>> My $.02 on line separators: >>> >>> - We should avoid clever tricks trying to solve problems that are >>> infrequent such as cross OS file line operations. >>> Most files will be read/written on a single system so the line >>> separators will be as expected. >>> >>> - Have/use APIs that split lines consistently accepting both line >>> separators so developer code can be agnostic to line separators. >>> aka BufferedReader.readLine for developers that are processing >>> the contents *as lines*. >>> Those other methods already exist; if there are any gaps in line >>> oriented processing that's a separate task. >>> >>> - These new file methods are defined to handle Charset >>> encoding/decoding and buffering. >>> Since there are other methods to deal with files as lines these >>> methods should not look for or break to lines. >>> >>> - Performance: adding code to look for line characters will slow it >>> down and in most cases would have no impact >>> since the line endings will match the system. >>> >>> - The strings passed to writeString (CharSequence) should have been >>> constructed using the proper line separators. >>> There are any number of methods that insert the os specific line >>> terminator and its easy to write File.separator as needed. >>> >>> As for the method naming: >>> >>> I'd prefer "writeString" as a familiar term that isn't stretched too >>> far by making the argument be a CharSequence. >>> its fine to call a CharSequence a string (with a lower case s). >>> >>> $.02, Roger >>> >>> >>> On 4/27/18 6:33 PM, Joe Wang wrote: >>>> >>>> >>>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>>> ----- Mail original ----- >>>>>> De: "Joe Wang" >>>>>> ?: "Remi Forax" , "Alan Bateman" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> Hi R?mi, Alan, >>>>> Hi Joe, >>>>> >>>>>> I'm not sure we'd want to replace system dependent line >>>>>> separators with >>>>>> '\n'. There are cases as you described where the replacement makes >>>>>> sense. However, there are other cases too. For example, the >>>>>> purpose is >>>>>> to read, edit a text file and then write it back. If this is on >>>>>> Windows, >>>>>> and if line separators are replaced with '\n', that would cause the >>>>>> resulting file not formatted properly in for example Notepad. >>>>>> There are >>>>>> cases where users write text files on Windows using Java, and >>>>>> only found >>>>>> the lines were not separated in Notepad. >>>>> I agree that why the counterpart of readString() that write the >>>>> string should inserts the platform's line separator. >>>>> BTW, i just found that those methods are not named writeString, or >>>>> writeCharSequence, i think they should. >>>> >>>> While readString() does not modify the original content (e.g. by >>>> replacing the platform's line separator with '\n'), write(String) >>>> won't either, by adding extra characters such as the line separator. >>>> >>>> I would think interfaces shall read along with the parameters. >>>> readString(Path) == read as a String from the specified Path >>>> (one could argue for readToString, readAsString, but we generally >>>> omit the preps) >>>> write(Path, CharSequence) == write the CharSequence to the >>>> file, since CharSequence is already in the method signature as a >>>> parameter, we probably don't want to add that to the name, >>>> otherwise it would read like repeating the word CharSequence. >>>> >>>> It is in a similar situation as write(Path, Iterable) where it was >>>> defined as writeLines(Path, Iterable). >>>> >>>>> >>>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>>> with the platform's line separator. If readString does the >>>>>> replacement, >>>>>> it would be inconsistent. >>>>> >>>>> Anyway, if we look for consistency the methods writeCharSequence >>>>> should transform the \n in the CharSequence to the platform's line >>>>> separator. >>>>> >>>>> Files.write(Path, Iterable) is is not a counterpart of >>>>> readString(), it's consistent with Files.lines() or >>>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>>> the line separators. Anyway, Files.write(path, >>>>> readString(path)::line) will be consistent if you replace the line >>>>> separators or not because String.line() suppresses the line >>>>> separators. >>>> >>>> readString pairs with write(String), therefore it's more like >>>> Files.write(path, readString(path)) than readString(path)::line. >>>> The use case: >>>> >>>> String s = Files.read(path); >>>> Files.write(path, s.replace("/config/location", "/new/location")); >>>> >>>> would then work as expected. >>>> >>>> These two methods are one-off (open/read/write/close file) >>>> operation. write(String) therefore is not intended for >>>> adding/appending multiple Strings from other operations such as >>>> String.line(). If an app needs to put the result of String.line() >>>> or any other processes into a file using this method, it needs to >>>> take care of adding the line separator itself when necessary. "when >>>> necessary" would be a judgement call by the developer. >>>> >>>> That said, if there's a consensus on the idea of terminating the >>>> string with a line separator, then readString method would need to >>>> strip it off to go with the write method. >>>> >>>>> >>>>>> I would think readString behaves like readAllBytes, that would >>>>>> simply read all content faithfully. >>>>> readString does an interpolation due to the Charset so it's not >>>>> the real content, again, the idea is that developers should not >>>>> have to care about the platform's line separators (they have more >>>>> important things to think). >>>>> >>>>> The other solution is to just not introduce those new methods, >>>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>>> does the job, no ? >>>> >>>> While there are many ways to do it, none is as straight-forward. >>>> "Read (entire) file to String"/"Write String to file" are popular >>>> requests from users. Read to string -> do some String manipulation >>>> -> write it back is such a simple use case, it really needs to be >>>> very easy to do as illustrated in the above code snippet. >>>> >>>> Cheers, >>>> Joe >>>> >>>>> >>>>>> Best, >>>>>> Joe >>>>> regards, >>>>> R?mi >>>>> >>>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>>> Hi Alan, >>>>>>> >>>>>>> People do not read the documentation. >>>>>>> So adding something in the documentation about when a method >>>>>>> should be used or >>>>>>> not is never a solution. >>>>>>> >>>>>>> Here the user want a String and provides a charset so you have >>>>>>> no way but to >>>>>>> decode the content to substitute the line separator. >>>>>>> >>>>>>> cheers, >>>>>>> R?mi >>>>>>> >>>>>>> ----- Mail original ----- >>>>>>>> De: "Alan Bateman" >>>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>>> >>>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>>> >>>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>>> reading/writing a string from/to a file >>>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>>> I think that having a readString that includes OS dependent >>>>>>>>> line separators is a >>>>>>>>> mistake, >>>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>>> kind of things >>>>>>>>> that makes programs behave differently on different platforms. >>>>>>>>> >>>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>>> people will do a call >>>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>>> hard way. >>>>>>>>> >>>>>>>>> raw string literal does the same substitution for the same >>>>>>>>> reason. >>>>>>>>> >>>>>>>> Yes, there are several discussion points around this and >>>>>>>> somewhat timely >>>>>>>> with multi-string support. >>>>>>>> >>>>>>>> One thing that I think Joe will need in this API is some note >>>>>>>> to make it >>>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>>> simple >>>>>>>> cases with mostly single lines of text). >>>>>>>> >>>>>>>> -Alan. >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 14 00:53:26 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 17:53:26 -0700 Subject: Callback Based Selectors In-Reply-To: References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Message-ID: Looks good to me also aside from a few minor things. Selector: (Note: There is some if not complete duplication here with my minor editing of / commenting on the CSR.) 111: performs -> perform 197: contain -> contains 433: ?ready operation set to a subset? is awkward. Maybe something like ?ready operation set being a subset?? 448: Remove un-italicized ?action" 459 and 536: re-entrant -> reentrant 495: operation -> operating SelectorImpl: 64: Trailing white space. EventPortSelectorImpl: What is the significance, if any, of moving the old lines 203-204 to be ahead of the new lines 198-201 versus being after the old lines 190-200? SelectWithConsumer test: 142: I assume the n == 1 assertion is to allow line 143 or 144 to identify which action was *not* performed. 214: Blank line 305: Comment -> ?Test invoking select operation with interrupt status set." 387: Comment is inconsistent with the key set *not* being locked. Thanks, Brian On Jun 4, 2018, at 5:49 AM, David Lloyd wrote: > Looks good to me! > > On Mon, Jun 4, 2018 at 3:38 AM, Alan Bateman wrote: >> >> Any other comments on the API additions before I create the CSR? >> >> -Alan >> >> >> On 11/04/2018 09:27, Alan Bateman wrote: >>> >>> I've re-based the patch that adds select(Consumer) and >>> friends. This follows the changes for JDK-8201315 so the action is invoked >>> while synchronized on the selector and the selected-key set, not the key set >>> for registrations. >>> >>> http://cr.openjdk.java.net/~alanb/8199433/webrev/ >>> >>> There's a still a bit of work to do on the Solaris event port >>> implementation related to error recovery, also additional tests will be >>> needed, but otherwise I think all the interactions with the existing spec >>> have been worked out. From Alan.Bateman at oracle.com Thu Jun 14 06:32:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 07:32:24 +0100 Subject: Callback Based Selectors In-Reply-To: References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Message-ID: On 14/06/2018 01:53, Brian Burkhalter wrote: > Looks good to me also aside from a few minor things. > > Selector: > > (Note: There is some if not complete duplication here with my minor editing of / commenting on the CSR.) > > 111: performs -> perform > 197: contain -> contains > 433: ?ready operation set to a subset? is awkward. Maybe something like ?ready operation set being a subset?? It should be ready operation set containing a subset. > 448: Remove un-italicized ?action" > 459 and 536: re-entrant -> reentrant > 495: operation -> operating > Thanks for the proof read, I've fixed up all these typos. -Alan From Alan.Bateman at oracle.com Thu Jun 14 08:17:07 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 09:17:07 +0100 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java In-Reply-To: References: Message-ID: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> On 12/06/2018 23:00, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8204915 > http://cr.openjdk.java.net/~bpb/8204915/webrev.00/ > > Please review this simple, test-only change. > This is a bit ugly and I think needs a bit of discussion to fully understand what you are trying to do. There are a number of issues with the patch, the main being that it means getFileStore will be called for each copy/move scenario and we know this can be problematic on some of the test systems due to statle NFS mounts and other issues. I think it would be a lot cleaner to just have the main method print the test directories as it does the setup for all the tests. The other issue is "catch Exception" (x2) to print the various attributes. These look really ugly. Are these asserts really failing? I assume you know which assert is failing by the line number but if additional info is needed then it would be better to just add a message to the assert. -Alan From Alan.Bateman at oracle.com Thu Jun 14 11:30:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 12:30:29 +0100 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> On 12/06/2018 17:52, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. The javadoc looks good. One part of the implementation that could be cleaner is the exception thrown for the malformed or unmappable cases. There are sub-classes of CharacterCodingException defined for this that would be clearer than an IOException with an IllegalArgumentException as cause. A minor nit in Files is that you can import jdk.internal.misc.JavaLangAccess rather than repeating the fully qualified class name. You can also move the definition of JLA up to the top. There's also a few inconsistencies with the existing code that would be goof to fix too (indenting and line length issues mostly). The test looks reasonable. In getData() then then "shouldn't happen" case should throw an exception as a NPE here might be tricky to diagnose there. Another nit is the sb field - can that be removed. -Alan From jness at proofpoint.com Thu Jun 14 14:56:38 2018 From: jness at proofpoint.com (Jeremiah Ness) Date: Thu, 14 Jun 2018 14:56:38 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: <8AA88B82C5DF994DB7950AF7EE88AD87D7B196@M0075109.ushostedarchiving.net> References: <8AA88B82C5DF994DB7950AF7EE88AD87D7B196@M0075109.ushostedarchiving.net> Message-ID: Thank you for filing the ticket Brian. Recently we have seen an increase in the occurrences of this bug. Roughly 2-3 times per week for some of our applications/roles/servers. I have attempted to make the test program a little simpler and removed the reflection hacks. I have attached the new version here. I am able to reproduce this issue on the following platforms: * Linux - java version "1.8.0_172" - build 1.8.0_172-b11 * Linux - java version "9.0.4" - build 9.0.4+11 * Linux - java version "10" 2018-03-20 - build 10+46 * Mac - java version "1.8.0_131" - build 1.8.0_131-b11 Additionally I have taken a closer look at EPollPort.java and feel like the following change may address is issue: --- a/src/java.base/linux/classes/sun/nio/ch/EPollPort.java +++ b/src/java.base/linux/classes/sun/nio/ch/EPollPort.java @@ -85,6 +85,7 @@ final class EPollPort private final ArrayBlockingQueue queue; private final Event NEED_TO_POLL = new Event(null, 0); private final Event EXECUTE_TASK_OR_SHUTDOWN = new Event(null, 0); + private final Event NOOP = new Event(null, 0); EPollPort(AsynchronousChannelProvider provider, ThreadPool pool) throws IOException @@ -194,7 +195,6 @@ final class EPollPort private class EventHandlerTask implements Runnable { private Event poll() throws IOException { try { - for (;;) { int n; do { n = EPoll.wait(epfd, address, MAX_EPOLL_EVENTS, -1); @@ -250,7 +250,8 @@ final class EPollPort } finally { fdToChannelLock.readLock().unlock(); } - } + // There is no real event to return. So return NOOP. + return NOOP; } finally { // to ensure that some thread will poll when all events have // been consumed @@ -288,6 +289,11 @@ final class EPollPort continue; } + // there is nothing to do + if (ev == NOOP) { + continue; + } + // handle wakeup to execute task or shutdown if (ev == EXECUTE_TASK_OR_SHUTDOWN) { Runnable task = pollTask(); Removal of the for(;;) loop I believe is the key important change because with that loop it is possible to overflow the queue, and to lose IO events. What do you think? Thanks, Miah From: Brian Burkhalter Date: Thursday, January 12, 2017 at 8:06 PM To: Jeremiah Ness Cc: Alan Bateman , "nio-dev at openjdk.java.net" Subject: Re: all EventHandlerTasks in EPollPort waiting on queue On Jan 12, 2017, at 8:03 AM, Jeremiah Ness > wrote: There does appear to be an issue here. Can you create a standalone test case to tickle test so that we can include it in a bug report? That would really help get to a regression test to include with the fix. I do have a test program that can trigger the condition for me. I have attached the source code for PortTest.java. PortTest has successfully demonstrated the issue in the following configurations: - OSX 10.11.6 with java version "1.8.0_112" - CentOs7 with kernel 3.10.0-514.2.2.el7.x86_64 and with openjdk version "1.8.0_111" I have filed https://bugs.openjdk.java.net/browse/JDK-8172750 to track this problem. Thank you for providing a test case. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PortTest.java Type: application/octet-stream Size: 6945 bytes Desc: PortTest.java URL: From brian.burkhalter at oracle.com Thu Jun 14 15:14:35 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 14 Jun 2018 08:14:35 -0700 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java In-Reply-To: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> Message-ID: <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> On Jun 14, 2018, at 1:17 AM, Alan Bateman wrote: > On 12/06/2018 23:00, Brian Burkhalter wrote: >> https://bugs.openjdk.java.net/browse/JDK-8204915 >> http://cr.openjdk.java.net/~bpb/8204915/webrev.00/ >> >> Please review this simple, test-only change. >> > This is a bit ugly and I think needs a bit of discussion to fully understand what you are trying to do. It?s already pushed but could be revisited. When investigating why there were failures when test.dir was set to a different device (https://bugs.openjdk.java.net/browse/JDK-8203803) I found that this information was useful in identifying the root cause of the failures on Linux, macOS, and Windows. Having it in the test log simplifies this analysis. > There are a number of issues with the patch, the main being that it means getFileStore will be called for each copy/move scenario and we know this can be problematic on some of the test systems due to statle NFS mounts and other issues. I suppose you mean that the FileStore would be null so we?d end up with an NPE on fs.type(). > I think it would be a lot cleaner to just have the main method print the test directories as it does the setup for all the tests. > > The other issue is "catch Exception" (x2) to print the various attributes. These look really ugly. Are these asserts really failing? I assume you know which assert is failing by the line number but if additional info is needed then it would be better to just add a message to the assert. Yes the line number can be used. The exact nature of the failure is better shown by seeing what the values of the compared items are. The catch does present a problem as it requires examining the print to see exactly which assert failed. On the other hand, if the first assert fails for example one can see whether the subsequent ones would as well. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 14 15:30:41 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 14 Jun 2018 08:30:41 -0700 Subject: Callback Based Selectors In-Reply-To: References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Message-ID: <6F8807CE-5103-4FEC-9B2F-E7AD25FD970B@oracle.com> On Jun 13, 2018, at 11:32 PM, Alan Bateman wrote: > Thanks for the proof read, I've fixed up all these typos. You?re welcome. I guess it?s ready to go then. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 14 16:06:07 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 17:06:07 +0100 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java In-Reply-To: <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> Message-ID: On 14/06/2018 16:14, Brian Burkhalter wrote: > : > > It?s already pushed but could be revisited. > > When investigating why there were failures when test.dir was set to a > different device (https://bugs.openjdk.java.net/browse/JDK-8203803) I > found that this information was useful in identifying the root cause > of the failures on Linux, macOS, and Windows. Having it in the test > log simplifies this analysis. The main method chooses the directory (or directories if possible) and I think that is the place to add the trace messages. I'd like to avoid the getFileStore as we've had so many issues with this hanging on specific machines (due to stale NFS mounts for example). > : >> I think it would be a lot cleaner to just have the main method print >> the test directories as it does the setup for all the tests. >> >> The other issue is "catch Exception" (x2) to print the various >> attributes. These look really ugly. Are these asserts really failing? >> I assume you know which assert is failing by the line number but if >> additional info is needed then it would be better to just add a >> message to the assert. > > Yes the line number can be used. The exact nature of the failure is > better shown by seeing what the values of the compared items are. The > catch does present a problem as it requires examining the print to see > exactly which assert failed. ?On the other hand, if the first assert > fails for example one can see whether the subsequent ones would as well. > I hope you see my point that the catching of Exception in these methods is a bit ugly. If you really need to print out all the attributes then it can be done before the asserts. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 14 16:13:12 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 14 Jun 2018 09:13:12 -0700 Subject: 8204915: (fs) Add some print diagnostics to java/nio/file/Files/CopyAndMove.java In-Reply-To: References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> Message-ID: <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> On Jun 14, 2018, at 9:06 AM, Alan Bateman wrote: > The main method chooses the directory (or directories if possible) and I think that is the place to add the trace messages. I'd like to avoid the getFileStore as we've had so many issues with this hanging on specific machines (due to stale NFS mounts for example). > >> [?] >> > I hope you see my point that the catching of Exception in these methods is a bit ugly. If you really need to print out all the attributes then it can be done before the asserts. I?ll file another issue to address these problems. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jun 15 00:53:59 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 14 Jun 2018 17:53:59 -0700 Subject: 8205062: (fs) Improve some print diagnostics in java/nio/file/Files/CopyAndMove.java In-Reply-To: <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> Message-ID: <9737F9B7-720B-4330-BF52-F9639DAAB990@oracle.com> Continued from http://mail.openjdk.java.net/pipermail/nio-dev/2018-June/005220.html. New issue: https://bugs.openjdk.java.net/browse/JDK-8205062 On Jun 14, 2018, at 9:13 AM, Brian Burkhalter wrote: > On Jun 14, 2018, at 9:06 AM, Alan Bateman wrote: > >> The main method chooses the directory (or directories if possible) and I think that is the place to add the trace messages. I'd like to avoid the getFileStore as we've had so many issues with this hanging on specific machines (due to stale NFS mounts for example). As it happens there were already getFileStore calls in there before these changes and I think they are inevitable. I have modified the main method to print the information for dir{1,2,3}. >>> [?] >>> >> I hope you see my point that the catching of Exception in these methods is a bit ugly. If you really need to print out all the attributes then it can be done before the asserts. I?ve added a second version of assertTrue which allows for printing the information while retaining the line number correlation between the exception and the assert method. http://cr.openjdk.java.net/~bpb/8205062/webrev.00/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jun 15 14:28:57 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jun 2018 15:28:57 +0100 Subject: Callback Based Selectors In-Reply-To: References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> Message-ID: <94a0ae8b-6d0f-4f63-ddc1-d30839a7a1cf@oracle.com> On 14/06/2018 01:53, Brian Burkhalter wrote: > : > > EventPortSelectorImpl: > > What is the significance, if any, of moving the old lines 203-204 to be ahead of the new lines 198-201 versus being after the old lines 190-200? It's not significant. The only important thing is that the keys are re-queued before any actions are invoked as the action is arbitrary code that could cause the selection operation to terminate with an exception. > > SelectWithConsumer test: > > 142: I assume the n == 1 assertion is to allow line 143 or 144 to identify which action was *not* performed. > 214: Blank line > 305: Comment -> ?Test invoking select operation with interrupt status set." > 387: Comment is inconsistent with the key set *not* being locked. > The test dates back to the prototype in April and needed update to cover the 3 methods. I've done that and refreshed the webrev. There are no changes to implementation (that you've already reviewed), just minor fixes to the javadoc that you pointed out and replacing the SelectWithConsumer test with a more complete test. http://cr.openjdk.java.net/~alanb/8199433/webrev/ The CSR has been approved so I think we are close to the finish line on this one. -Alan From Alan.Bateman at oracle.com Fri Jun 15 14:56:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jun 2018 15:56:58 +0100 Subject: 8205062: (fs) Improve some print diagnostics in java/nio/file/Files/CopyAndMove.java In-Reply-To: <9737F9B7-720B-4330-BF52-F9639DAAB990@oracle.com> References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> <9737F9B7-720B-4330-BF52-F9639DAAB990@oracle.com> Message-ID: <39319f57-9376-6b1a-ee8e-f48793879516@oracle.com> On 15/06/2018 01:53, Brian Burkhalter wrote: > I?ve added a second version of assertTrue which allows for printing > the information while retaining the line number correlation between > the exception and the assert method. > > http://cr.openjdk.java.net/~bpb/8205062/webrev.00/ > > This looks much better. In printDirInfo then store can't be null (right?) so I assume that check can be removed. -Alan From brian.burkhalter at oracle.com Fri Jun 15 15:29:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jun 2018 08:29:21 -0700 Subject: 8205062: (fs) Improve some print diagnostics in java/nio/file/Files/CopyAndMove.java In-Reply-To: <39319f57-9376-6b1a-ee8e-f48793879516@oracle.com> References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> <9737F9B7-720B-4330-BF52-F9639DAAB990@oracle.com> <39319f57-9376-6b1a-ee8e-f48793879516@oracle.com> Message-ID: On Jun 15, 2018, at 7:56 AM, Alan Bateman wrote: >> http://cr.openjdk.java.net/~bpb/8205062/webrev.00/ >> > This looks much better. In printDirInfo then store can't be null (right?) so I assume that check can be removed. So may I assume that this is good to go provided I make the aforementioned change (and re-test) before pushing? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Jun 15 15:36:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Jun 2018 16:36:20 +0100 Subject: 8205062: (fs) Improve some print diagnostics in java/nio/file/Files/CopyAndMove.java In-Reply-To: References: <95350c02-ee53-b88f-d44e-9b26fceedb2c@oracle.com> <40C0C2F6-ED58-477F-8ECF-651A612C5058@oracle.com> <79E5055A-E8C6-4428-8B20-B55AFAE5EF53@oracle.com> <9737F9B7-720B-4330-BF52-F9639DAAB990@oracle.com> <39319f57-9376-6b1a-ee8e-f48793879516@oracle.com> Message-ID: <9ed92c63-37d1-f24e-b9b6-715ffcde741f@oracle.com> On 15/06/2018 16:29, Brian Burkhalter wrote: > On Jun 15, 2018, at 7:56 AM, Alan Bateman > wrote: > >>> http://cr.openjdk.java.net/~bpb/8205062/webrev.00/ >>> >>> >> This looks much better. In printDirInfo then store can't be null >> (right?) so I assume that check can be removed. > > So may I assume that this is good to go provided I make the > aforementioned change (and re-test) before pushing? > Yes, I think so. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Jun 15 20:31:59 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 Jun 2018 13:31:59 -0700 Subject: Callback Based Selectors In-Reply-To: <94a0ae8b-6d0f-4f63-ddc1-d30839a7a1cf@oracle.com> References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> <94a0ae8b-6d0f-4f63-ddc1-d30839a7a1cf@oracle.com> Message-ID: <4451E039-7239-4CAC-9229-84D346B079F4@oracle.com> On Jun 15, 2018, at 7:28 AM, Alan Bateman wrote: > There are no changes to implementation (that you've already reviewed), just minor fixes to the javadoc that you pointed out and replacing the SelectWithConsumer test with a more complete test. > > http://cr.openjdk.java.net/~alanb/8199433/webrev/ > > The CSR has been approved so I think we are close to the finish line on this one. Minor comments on the SelectWithConsumer test: 56: checks -> check 538: removes -> remove In the assertTrue() calls which involve a duration check, might it be useful to print the value of ?duration? if the assertion fails? I think that this can go forward without further review if the foregoing items are changed. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sat Jun 16 09:28:51 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 16 Jun 2018 10:28:51 +0100 Subject: Callback Based Selectors In-Reply-To: <4451E039-7239-4CAC-9229-84D346B079F4@oracle.com> References: <0fe28e06-ef26-9c07-034f-f79feef2a66a@oracle.com> <64ea3b2b-efae-d11f-1724-d5d96c6790d8@oracle.com> <2b00b979-61a7-d610-9ca2-7196973a6354@oracle.com> <94a0ae8b-6d0f-4f63-ddc1-d30839a7a1cf@oracle.com> <4451E039-7239-4CAC-9229-84D346B079F4@oracle.com> Message-ID: <329134ec-53a7-4359-ff21-91365c218c96@oracle.com> On 15/06/2018 21:31, Brian Burkhalter wrote: > > > Minor comments on the SelectWithConsumer test: > > 56:checks -> check > 538:removes -> remove > > In the assertTrue() calls which involve a duration check, might it be > useful to print the value of ?duration? if the assertion fails? > Good idea. I'll change these asserts from `assertTrue(e)` to `assertTrue(e, "select took " + duration + " ms")` so the jtr file has the duration in the event of failure. I've run this test on all platforms several hundred times (with high concurrency) so I'm not expecting any issues but it is always tricky to test timed selection operations. > I think that this can go forward without further review if the > foregoing items are changed. > Thanks, I'll get this pushed so that it's in jdk-11-ea+19. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Sat Jun 16 18:21:57 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Sat, 16 Jun 2018 11:21:57 -0700 (PDT) Subject: Callback Based Selectors Message-ID: All good! Nice to get this one in. Brian ----- Original Message ----- From: Alan.Bateman at oracle.com To: brian.burkhalter at oracle.com Cc: nio-dev at openjdk.java.net Sent: Saturday, 16 June, 2018 2:28:52 AM GMT -08:00 US/Canada Pacific Subject: Re: Callback Based Selectors On 15/06/2018 21:31, Brian Burkhalter wrote: Minor comments on the SelectWithConsumer test: 56: checks -> check 538: removes -> remove In the assertTrue() calls which involve a duration check, might it be useful to print the value of ?duration? if the assertion fails? Good idea. I'll change these asserts from `assertTrue(e)` to `assertTrue(e, "select took " + duration + " ms")` so the jtr file has the duration in the event of failure. I've run this test on all platforms several hundred times (with high concurrency) so I'm not expecting any issues but it is always tricky to test timed selection operations. I think that this can go forward without further review if the foregoing items are changed. Thanks, I'll get this pushed so that it's in jdk-11-ea+19. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Sun Jun 17 06:58:54 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Sun, 17 Jun 2018 06:58:54 +0000 Subject: adding rsockets support into JDK References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> Hi All, Here is the webrev.05 version of the patch: http://cr.openjdk.java.net/~ylu/8195160.05/ In this version, I did following changes: 1. Fixed a small bug inside RdmaSocketImpl.GetOption. 2. Took a subset of test cases from java.net.Socket, java.net.ServerSocket, java.nio.channels.SocketChannel, java.nio.channels.ServerSocketChannel and java.nio.channels.Selector tests and modified them for RDMA based sockets, channels and selectors. I am still working on test cases for the 3 new RDMA socket options and will submit webrev.06 on Monday. At the meantime, please let me know your feedback and comments on the current test cases. Thanks, Lucy >-----Original Message----- >From: Lu, Yingqi >Sent: Thursday, June 7, 2018 2:06 PM >To: 'Lu, Yingqi' ; Alan Bateman >; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: RE: adding rsockets support into JDK > >Hi Alan, > >I checked on librdmacm availability on different Linux distros. It is at least >available in Ubuntu, Arch Linux, Fedora/Centos/Red Hat, openSUSE and Gentoo. > >Thanks, >Lucy > >>-----Original Message----- >>From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of >>Lu, Yingqi >>Sent: Thursday, June 7, 2018 11:11 AM >>To: Alan Bateman ; nio-dev at openjdk.java.net >>Cc: Viswanathan, Sandhya ; Aundhe, >>Shirish ; Kaczmarek, Eric >> >>Subject: RE: adding rsockets support into JDK >> >>Hi Alan, >> >>Thank you very much for your guidance on the tests. This is very helpful. >> >>I will look into SDP test cases and use them as a reference. >> >>In addition, I tested rsocket client and server on the same machine for >>both sockets and socket channels. InetAddress.getLocalHost() is used as >>the address for both server and client. The tests pass. Using sar[1], I >>see CPU cores are busy, but network does not show traffic. I think it >>might due to how rsocket handles local traffic (rsocket does not use >>loopback device). I also tested the same programs on two different >>machines and I saw network traffic shown on the RDMA network card. >> >>Thanks, >>Lucy >> >>[1] https://linux.die.net/man/1/sar >> >> >>>-----Original Message----- >>>From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >>>Sent: Thursday, June 7, 2018 1:15 AM >>>To: Lu, Yingqi ; nio-dev at openjdk.java.net >>>Cc: Viswanathan, Sandhya ; Aundhe, >>>Shirish ; Kaczmarek, Eric >>> >>>Subject: Re: adding rsockets support into JDK >>> >>>On 05/06/2018 07:17, Lu, Yingqi wrote: >>>> Hi All, >>>> >>>> Here is the version #4 of the patch >>>> http://cr.openjdk.java.net/~ylu/8195160.04/ >>>> >>>> In this version, I have changed: >>>> >>>> 1. Remove jdk.net.RdmaSocketImplFactory. >>>> 2. Remove the inheritance from >>>> SocketChannelImpl/ServerSocketChannelImpl, >>>replace it with SocketChannel/ServerSocketChannel. >>>> 3. Remove unnecessary setters from SocketImpl. >>>> 4. Override accept method to the ServerSocket returned by >>>jdk.net.openServerSocket. >>>> 5. Add a small comment to PollSelectorImpl.poll method 6. Clean up a >>>> little in Lib-jdk.net.gmk >>>> >>>> I am still working on creating all the test cases. Do we need to >>>> replicate all the >>>tests from both Socket/ServerSocket and >>>SocketChannel/ServerSocketChannel, >>>or we only need to pick up a subset of the cases? >>>> >>>> Please review the patch and let me know your feedback and comments. >>>> >>>For tests then I think we should have a basic set of tests that >>>exercise each of the elements in the API. For each socket and channel >>>type then each of the methods in the API should be exercised. There >>>are several new socket options so they should be exercised too. So a >>>good start is a maintainable set of unit tests. I don't think you >>>should be concerned with duplicating all the existing tests as many of >>>those tests are regression tests or stress specific areas where there >>>were bugs in the >>past. >>> >>>In general, I think testing is going to be a challenge for this >>>feature as it needs commitment to maintain the infrastructure needed >>>to test this feature and then running the tests at each release. When >>>we added the SDP support, it required maintaining a set of machines >>>with Infiniband HCAs and running the tests on a regular basis. I >>>assume something >>similar will be needed here. >>> >>>When developing the jtreg tests then it's important that don't fail on >>>systems without RDMA hardware. For SDP, the tests checked for >>>/proc/net/sdp and would pass silently when not present. I hope there >>>is something easy to check for rsockets. >>> >>>One thing that isn't clear to me is whether communication between >>>using sockets on the same machine, using an address plumbed to an RDMA >>>interface, will use the network or not. I guess I'm asking if you are >>>thinking about functional testing with two or more machines, I assume >>>you will at least be doing some of that when getting performance data >>>and >>comparing it to TCP or SDP connections. >>> >>>On the build changes, then one thing that would be useful to know is >>>whether librdmacm is included in all distributions. If someone is >>>building on say Ubuntu will they need to install an additional package >>>installed. I bring this as we may need to do a bit more on the build >>>to probe for the library. In other areas, at least in the past, we've >>>had to resort to dynamic linking (dlopen/dlsym). I'm sure we should do >>>that here, instead I'm just wondering whether the rsocket support will >>>be compiled in >>when there it is not present on the build machine. >>> >>>I skimmed quickly through the webrev and I think it's looking quite good. >>>RdmaNet defines only static methods so I don't think it needs to extends Net. >>>There is more to do on the javadoc for the new methods, also >>>RdmaSocketOptions. I can help out on that when the time comes. >>> >>>-Alan. >>> From yingqi.lu at intel.com Mon Jun 18 21:19:31 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Mon, 18 Jun 2018 21:19:31 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139A56A@ORSMSX113.amr.corp.intel.com> <4b993019-3038-5584-67fd-31594afe4420@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD139D506@ORSMSX113.amr.corp.intel.com> <75e2d375-0760-20f8-f97e-41a11ecfd364@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13A6B5F@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> Hi All, I just created a CSR at the meantime. The link is available at https://bugs.openjdk.java.net/browse/JDK-8205186. Please review this and let me know your feedback. Thank you! Lucy >-----Original Message----- >From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, >Yingqi >Sent: Saturday, June 16, 2018 11:59 PM >To: Alan Bateman ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: RE: adding rsockets support into JDK > >Hi All, > >Here is the webrev.05 version of the patch: >http://cr.openjdk.java.net/~ylu/8195160.05/ > >In this version, I did following changes: > >1. Fixed a small bug inside RdmaSocketImpl.GetOption. >2. Took a subset of test cases from java.net.Socket, java.net.ServerSocket, >java.nio.channels.SocketChannel, java.nio.channels.ServerSocketChannel and >java.nio.channels.Selector tests and modified them for RDMA based sockets, >channels and selectors. > >I am still working on test cases for the 3 new RDMA socket options and will >submit webrev.06 on Monday. At the meantime, please let me know your >feedback and comments on the current test cases. > >Thanks, >Lucy > >>-----Original Message----- >>From: Lu, Yingqi >>Sent: Thursday, June 7, 2018 2:06 PM >>To: 'Lu, Yingqi' ; Alan Bateman >>; nio-dev at openjdk.java.net >>Cc: Viswanathan, Sandhya ; Aundhe, >>Shirish ; Kaczmarek, Eric >> >>Subject: RE: adding rsockets support into JDK >> >>Hi Alan, >> >>I checked on librdmacm availability on different Linux distros. It is >>at least available in Ubuntu, Arch Linux, Fedora/Centos/Red Hat, openSUSE and >Gentoo. >> >>Thanks, >>Lucy >> >>>-----Original Message----- >>>From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of >>>Lu, Yingqi >>>Sent: Thursday, June 7, 2018 11:11 AM >>>To: Alan Bateman ; nio-dev at openjdk.java.net >>>Cc: Viswanathan, Sandhya ; Aundhe, >>>Shirish ; Kaczmarek, Eric >>> >>>Subject: RE: adding rsockets support into JDK >>> >>>Hi Alan, >>> >>>Thank you very much for your guidance on the tests. This is very helpful. >>> >>>I will look into SDP test cases and use them as a reference. >>> >>>In addition, I tested rsocket client and server on the same machine >>>for both sockets and socket channels. InetAddress.getLocalHost() is >>>used as the address for both server and client. The tests pass. Using >>>sar[1], I see CPU cores are busy, but network does not show traffic. I >>>think it might due to how rsocket handles local traffic (rsocket does >>>not use loopback device). I also tested the same programs on two >>>different machines and I saw network traffic shown on the RDMA network card. >>> >>>Thanks, >>>Lucy >>> >>>[1] https://linux.die.net/man/1/sar >>> >>> >>>>-----Original Message----- >>>>From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >>>>Sent: Thursday, June 7, 2018 1:15 AM >>>>To: Lu, Yingqi ; nio-dev at openjdk.java.net >>>>Cc: Viswanathan, Sandhya ; Aundhe, >>>>Shirish ; Kaczmarek, Eric >>>> >>>>Subject: Re: adding rsockets support into JDK >>>> >>>>On 05/06/2018 07:17, Lu, Yingqi wrote: >>>>> Hi All, >>>>> >>>>> Here is the version #4 of the patch >>>>> http://cr.openjdk.java.net/~ylu/8195160.04/ >>>>> >>>>> In this version, I have changed: >>>>> >>>>> 1. Remove jdk.net.RdmaSocketImplFactory. >>>>> 2. Remove the inheritance from >>>>> SocketChannelImpl/ServerSocketChannelImpl, >>>>replace it with SocketChannel/ServerSocketChannel. >>>>> 3. Remove unnecessary setters from SocketImpl. >>>>> 4. Override accept method to the ServerSocket returned by >>>>jdk.net.openServerSocket. >>>>> 5. Add a small comment to PollSelectorImpl.poll method 6. Clean up >>>>> a little in Lib-jdk.net.gmk >>>>> >>>>> I am still working on creating all the test cases. Do we need to >>>>> replicate all the >>>>tests from both Socket/ServerSocket and >>>>SocketChannel/ServerSocketChannel, >>>>or we only need to pick up a subset of the cases? >>>>> >>>>> Please review the patch and let me know your feedback and comments. >>>>> >>>>For tests then I think we should have a basic set of tests that >>>>exercise each of the elements in the API. For each socket and channel >>>>type then each of the methods in the API should be exercised. There >>>>are several new socket options so they should be exercised too. So a >>>>good start is a maintainable set of unit tests. I don't think you >>>>should be concerned with duplicating all the existing tests as many >>>>of those tests are regression tests or stress specific areas where >>>>there were bugs in the >>>past. >>>> >>>>In general, I think testing is going to be a challenge for this >>>>feature as it needs commitment to maintain the infrastructure needed >>>>to test this feature and then running the tests at each release. When >>>>we added the SDP support, it required maintaining a set of machines >>>>with Infiniband HCAs and running the tests on a regular basis. I >>>>assume something >>>similar will be needed here. >>>> >>>>When developing the jtreg tests then it's important that don't fail >>>>on systems without RDMA hardware. For SDP, the tests checked for >>>>/proc/net/sdp and would pass silently when not present. I hope there >>>>is something easy to check for rsockets. >>>> >>>>One thing that isn't clear to me is whether communication between >>>>using sockets on the same machine, using an address plumbed to an >>>>RDMA interface, will use the network or not. I guess I'm asking if >>>>you are thinking about functional testing with two or more machines, >>>>I assume you will at least be doing some of that when getting >>>>performance data and >>>comparing it to TCP or SDP connections. >>>> >>>>On the build changes, then one thing that would be useful to know is >>>>whether librdmacm is included in all distributions. If someone is >>>>building on say Ubuntu will they need to install an additional >>>>package installed. I bring this as we may need to do a bit more on >>>>the build to probe for the library. In other areas, at least in the >>>>past, we've had to resort to dynamic linking (dlopen/dlsym). I'm sure >>>>we should do that here, instead I'm just wondering whether the >>>>rsocket support will be compiled in >>>when there it is not present on the build machine. >>>> >>>>I skimmed quickly through the webrev and I think it's looking quite good. >>>>RdmaNet defines only static methods so I don't think it needs to extends Net. >>>>There is more to do on the javadoc for the new methods, also >>>>RdmaSocketOptions. I can help out on that when the time comes. >>>> >>>>-Alan. >>>> From Alan.Bateman at oracle.com Tue Jun 19 06:53:48 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Jun 2018 07:53:48 +0100 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> Message-ID: <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> On 18/06/2018 22:19, Lu, Yingqi wrote: > Hi All, > > I just created a CSR at the meantime. The link is available at https://bugs.openjdk.java.net/browse/JDK-8205186. > > Please review this and let me know your feedback. > The CSR will need to include the javadoc for each of the methods. That isn't fully fleshed out in the webrev yet. Do you need help on this? The other thing is that adds a protected constructor to java.net.ServerSocket (to match the equivalent in java.net.Socket) so that will need to be included too. This addition means the scope changes to Java SE too. -Alan From yingqi.lu at intel.com Tue Jun 19 07:00:09 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 19 Jun 2018 07:00:09 +0000 Subject: adding rsockets support into JDK In-Reply-To: <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1478898@ORSMSX113.amr.corp.intel.com> Hi Alan, Yes, help on Javadoc is definitely very much appreciated :-) >>The CSR will need to include the javadoc for each of the methods. That isn't fully fleshed out in the webrev yet. Do you need help on this? By the way, I found couple of other small issues in the code during testing and already fixed it. I am still working on the last couple test cases for socket options. Once I finish that, I will submit webrev version 6. Thanks, Lucy >-----Original Message----- >From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >Sent: Monday, June 18, 2018 11:54 PM >To: Lu, Yingqi ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: Re: adding rsockets support into JDK > >On 18/06/2018 22:19, Lu, Yingqi wrote: >> Hi All, >> >> I just created a CSR at the meantime. The link is available at >https://bugs.openjdk.java.net/browse/JDK-8205186. >> >> Please review this and let me know your feedback. >> >The CSR will need to include the javadoc for each of the methods. That isn't fully >fleshed out in the webrev yet. Do you need help on this? > >The other thing is that adds a protected constructor to java.net.ServerSocket (to >match the equivalent in java.net.Socket) so that will need to be included too. This >addition means the scope changes to Java SE too. > >-Alan From jness at proofpoint.com Tue Jun 19 12:35:35 2018 From: jness at proofpoint.com (Jeremiah Ness) Date: Tue, 19 Jun 2018 12:35:35 +0000 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: References: <8AA88B82C5DF994DB7950AF7EE88AD87D7B196@M0075109.ushostedarchiving.net> Message-ID: <6ECA0DF3-B465-4566-AD84-4433F5831807@proofpoint.com> Hi Brian and Alan, Thanks for considering my report. Is there anything I could do to help triage or prioritize an investigation? Couple thoughts on my mind: * Is the issue understood? * Would it be useful to summarize? * A few of my colleagues are able to reproduce the problem with the provided test program. Would seeing the log output or stack traces be helpful? Thanks, Miah From: Jeremiah Ness Date: Thursday, June 14, 2018 at 10:56 AM To: Brian Burkhalter Cc: Alan Bateman , "nio-dev at openjdk.java.net" Subject: Re: all EventHandlerTasks in EPollPort waiting on queue Thank you for filing the ticket Brian. Recently we have seen an increase in the occurrences of this bug. Roughly 2-3 times per week for some of our applications/roles/servers. I have attempted to make the test program a little simpler and removed the reflection hacks. I have attached the new version here. I am able to reproduce this issue on the following platforms: * Linux - java version "1.8.0_172" - build 1.8.0_172-b11 * Linux - java version "9.0.4" - build 9.0.4+11 * Linux - java version "10" 2018-03-20 - build 10+46 * Mac - java version "1.8.0_131" - build 1.8.0_131-b11 Additionally I have taken a closer look at EPollPort.java and feel like the following change may address is issue: --- a/src/java.base/linux/classes/sun/nio/ch/EPollPort.java +++ b/src/java.base/linux/classes/sun/nio/ch/EPollPort.java @@ -85,6 +85,7 @@ final class EPollPort private final ArrayBlockingQueue queue; private final Event NEED_TO_POLL = new Event(null, 0); private final Event EXECUTE_TASK_OR_SHUTDOWN = new Event(null, 0); + private final Event NOOP = new Event(null, 0); EPollPort(AsynchronousChannelProvider provider, ThreadPool pool) throws IOException @@ -194,7 +195,6 @@ final class EPollPort private class EventHandlerTask implements Runnable { private Event poll() throws IOException { try { - for (;;) { int n; do { n = EPoll.wait(epfd, address, MAX_EPOLL_EVENTS, -1); @@ -250,7 +250,8 @@ final class EPollPort } finally { fdToChannelLock.readLock().unlock(); } - } + // There is no real event to return. So return NOOP. + return NOOP; } finally { // to ensure that some thread will poll when all events have // been consumed @@ -288,6 +289,11 @@ final class EPollPort continue; } + // there is nothing to do + if (ev == NOOP) { + continue; + } + // handle wakeup to execute task or shutdown if (ev == EXECUTE_TASK_OR_SHUTDOWN) { Runnable task = pollTask(); Removal of the for(;;) loop I believe is the key important change because with that loop it is possible to overflow the queue, and to lose IO events. What do you think? Thanks, Miah From: Brian Burkhalter Date: Thursday, January 12, 2017 at 8:06 PM To: Jeremiah Ness Cc: Alan Bateman , "nio-dev at openjdk.java.net" Subject: Re: all EventHandlerTasks in EPollPort waiting on queue On Jan 12, 2017, at 8:03 AM, Jeremiah Ness > wrote: There does appear to be an issue here. Can you create a standalone test case to tickle test so that we can include it in a bug report? That would really help get to a regression test to include with the fix. I do have a test program that can trigger the condition for me. I have attached the source code for PortTest.java. PortTest has successfully demonstrated the issue in the following configurations: - OSX 10.11.6 with java version "1.8.0_112" - CentOs7 with kernel 3.10.0-514.2.2.el7.x86_64 and with openjdk version "1.8.0_111" I have filed https://bugs.openjdk.java.net/browse/JDK-8172750 to track this problem. Thank you for providing a test case. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jun 19 12:46:19 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Jun 2018 13:46:19 +0100 Subject: all EventHandlerTasks in EPollPort waiting on queue In-Reply-To: <6ECA0DF3-B465-4566-AD84-4433F5831807@proofpoint.com> References: <8AA88B82C5DF994DB7950AF7EE88AD87D7B196@M0075109.ushostedarchiving.net> <6ECA0DF3-B465-4566-AD84-4433F5831807@proofpoint.com> Message-ID: <7b6af97f-3685-d316-1832-7cfca27e4682@oracle.com> On 19/06/2018 13:35, Jeremiah Ness wrote: > > Hi Brian and Alan, > > Thanks for considering my report. > > Is there anything I could do to help triage or prioritize an > investigation? > > I saw your mail with the test case (thanks) but I haven't had time to look at it. It's likely the bug exists on other platforms too btw. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 20 00:23:53 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 Jun 2018 17:23:53 -0700 Subject: trivial patch to reduce allocations in nio open* methods In-Reply-To: <4b6b5715-f643-b945-053d-b77889a7d554@oracle.com> References: <158BFF8B-586B-4C1C-8DBE-5C5DD3E8D66B@oracle.com> <2ba60a55-baf3-8542-dac6-70c33e3f5f25@oracle.com> <0803a885-5938-6549-262e-40064d73c513@oracle.com> <5aa714cf.c30c1c0a.fbbdc.b3fb@mx.google.com> <4b6b5715-f643-b945-053d-b77889a7d554@oracle.com> Message-ID: <17A3D1A3-9157-444B-8B93-77D3174F9CA8@oracle.com> Picking up this thread from [1] after a few months. A new patch is available at [2]. The changes versus version .00 are at [3]. On Mar 30, 2018, at 1:27 AM, Alan Bateman wrote: > On 30/03/2018 02:11, Brian Burkhalter wrote: >> I created a webrev of this patch: >> >> http://cr.openjdk.java.net/~bpb/8199124/webrev.00/ >> >> I don?t see any problem in moving forward with it, possibly with some minor cleanup. >> > Just a few comments. > > The sizing of the hash map doesn't take the load factor into account. You might find `new HashMap<>(1 + (4 * options.length/ 3)` a bit better with a larger array of options. I don?t see what you are referring to. There is no use of HashMap in these classes. The HashSets are created to be of a fixed size as they are not resized later. Perhaps Set.of(E? elements) should be used to create these Sets? > You can use an immutable set to avoid wrapping the EnumSet, e.g. > private static final Set DEFAULT_CREATE_OPTIONS = Set.of(CREATE_NEW, WRITE); The two occurrences like this are changed according to the suggestion. Thanks, Brian [1] http://mail.openjdk.java.net/pipermail/nio-dev/2018-March/004950.html [2] http://cr.openjdk.java.net/~bpb/8199124/webrev.01/ [3] http://cr.openjdk.java.net/~bpb/8199124/webrev.00-01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From huizhe.wang at oracle.com Wed Jun 20 03:32:45 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 19 Jun 2018 20:32:45 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> Message-ID: <5B29CADD.7090200@oracle.com> Thanks Alan. I created 8205058 to capture your suggestions. Please see below for more details. On 6/14/18, 4:30 AM, Alan Bateman wrote: > On 12/06/2018 17:52, Joe Wang wrote: >> Hi all, >> >> It's been a while since the 1st round of reviews. Thank you all for >> the valuable comments and suggestions! Note that Roger's last >> response went to nio-dev, but not core-libs-dev, I've therefore >> attached it below. >> >> CSR [2]: the CSR is now approved. Note the write method has been >> changed to writeString. >> >> Impl [3]: for performance reason and the different requirement from >> byte-string conversion for malformed/unmappable character handing, >> the implementation uses a specific method separate from the existing >> ones. Both Sherman and Alan preferred specific method than adding >> parameters to the APIs that might be error prone. Thanks Sherman for >> the code! >> >> JMH benchmark tests showed the new read method performs better than >> an operation that reads the bytes and convert to string. The gains >> start to be significant even at quite reasonable sizes (< 10K). >> >> >> [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 >> [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 >> [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ >> >> Please review. > The javadoc looks good. > > One part of the implementation that could be cleaner is the exception > thrown for the malformed or unmappable cases. There are sub-classes of > CharacterCodingException defined for this that would be clearer than > an IOException with an IllegalArgumentException as cause. Changed the internal APIs to throw CCE instead. In the same way as the previous changeset for 8201276, these methods are made specific for the use cases (though they are now for Files.read/writeString only) so that they are not mixed up with existing ones that may inadvertently affect other usages. One thing to note is that MalformedInputException or UnmappableCharacterException will lose one piece of information in comparison to the existing IAE, that is where it happens (offset). Should there be an improvement in the future, we could consider add it back to this part of code. > > A minor nit in Files is that you can import > jdk.internal.misc.JavaLangAccess rather than repeating the fully > qualified class name. You can also move the definition of JLA up to > the top. There's also a few inconsistencies with the existing code > that would be goof to fix too (indenting and line length issues mostly). Moved JLA and others to the top. Fixed also the indentations (mostly method declarations) and a few long lines. > > The test looks reasonable. In getData() then then "shouldn't happen" > case should throw an exception as a NPE here might be tricky to > diagnose there. Another nit is the sb field - can that be removed. Fixed too. JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ Regards, Joe > > -Alan > From Alan.Bateman at oracle.com Wed Jun 20 08:52:01 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 09:52:01 +0100 Subject: trivial patch to reduce allocations in nio open* methods In-Reply-To: <17A3D1A3-9157-444B-8B93-77D3174F9CA8@oracle.com> References: <158BFF8B-586B-4C1C-8DBE-5C5DD3E8D66B@oracle.com> <2ba60a55-baf3-8542-dac6-70c33e3f5f25@oracle.com> <0803a885-5938-6549-262e-40064d73c513@oracle.com> <5aa714cf.c30c1c0a.fbbdc.b3fb@mx.google.com> <4b6b5715-f643-b945-053d-b77889a7d554@oracle.com> <17A3D1A3-9157-444B-8B93-77D3174F9CA8@oracle.com> Message-ID: On 20/06/2018 01:23, Brian Burkhalter wrote: > : >> >> The sizing of the hash map doesn't take the load factor into account. >> You might find `new HashMap<>(1 + (4 * options.length/ 3)` a bit >> better with a larger array of options. > > I don?t see what you are referring to. There is no use of HashMap in > these classes. The HashSets are created to be of a fixed size as they > are not resized later. Perhaps Set.of(E? elements) should be used to > create these Sets? > > FileSystemProvider L430. This has been changed to create the HashSet with an initial capacity of len+1 so it's ignoring the load factor. I'm just saying that the 4 places where this has changed can be improved to avoid resizing every time. Given that the number of options is small is almost all cases then it may be simpler to just use the no-arg constructor for now. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jun 20 12:24:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 13:24:31 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor Message-ID: A requirement that comes up periodically is some way to create a SelectableChannel implementation that can be used with the built-in Selector implementations. It's a project in itself because the FileDescriptor representation is not exposed and because it requires bidirectional translations of events. An alternative approach for the common case of just read and write operations is to provide a simple API to create a SelectableChannel to a file descriptor. The proposal here is to extend the JDK-specific jdk.net module with support for creating a SelectableChannel to an existing file descriptor. The resulting channel is associated with the default SelectorProvider and supports an operation-set of OP_READ and OP_WRITE. The API is as minimal ass possible, the only complexity is closing when the channel is registered with a Selector. The webrev with the proposed API is here: ?? http://cr.openjdk.java.net/~alanb/8198372/webrev/index.html -Alan From david.lloyd at redhat.com Wed Jun 20 13:16:35 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 20 Jun 2018 08:16:35 -0500 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: A few comments... + public interface SelectableChannelCloser { I wonder if some basic implementation should be provided for this. I can't think of a case where a user would be able to implement this interface without native code of some sort (is this the intention?). + public static SelectableChannel readWriteSelectableChannel(FileDescriptor fd, + SelectableChannelCloser closer) { + Objects.requireNonNull(fd); + Objects.requireNonNull(closer); + if (!fd.valid()) + throw new IllegalArgumentException("file descriptor is not valid"); + + SecurityManager sm = System.getSecurityManager(); + if (sm != null) { + sm.checkRead(fd); + sm.checkWrite(fd); + } I disagree with these security checks. Nothing in this API allows the caller to actually read or write the FD; that would have to be provided separately. Given that each of the checks are relatively expensive, IMO they should be deferred to the point where an object is constructed that actually has read and/or write capability. On Wed, Jun 20, 2018 at 7:24 AM Alan Bateman wrote: > > > A requirement that comes up periodically is some way to create a > SelectableChannel implementation that can be used with the built-in > Selector implementations. It's a project in itself because the > FileDescriptor representation is not exposed and because it requires > bidirectional translations of events. An alternative approach for the > common case of just read and write operations is to provide a simple API > to create a SelectableChannel to a file descriptor. > > The proposal here is to extend the JDK-specific jdk.net module with > support for creating a SelectableChannel to an existing file descriptor. > The resulting channel is associated with the default SelectorProvider > and supports an operation-set of OP_READ and OP_WRITE. The API is as > minimal ass possible, the only complexity is closing when the channel is > registered with a Selector. > > The webrev with the proposed API is here: > http://cr.openjdk.java.net/~alanb/8198372/webrev/index.html > > -Alan > > -- - DML From Alan.Bateman at oracle.com Wed Jun 20 13:41:26 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 14:41:26 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On 20/06/2018 14:16, David Lloyd wrote: > A few comments... > > + public interface SelectableChannelCloser { > > I wonder if some basic implementation should be provided for this. I > can't think of a case where a user would be able to implement this > interface without native code of some sort (is this the intention?). If you managing the file descriptor yourself then it means native code, so yes, that is the intention. > : > > I disagree with these security checks. Nothing in this API allows the > caller to actually read or write the FD; that would have to be > provided separately. Given that each of the checks are relatively > expensive, IMO they should be deferred to the point where an object is > constructed that actually has read and/or write capability. A permission check is needed here. The two permission targets that RuntimePermission specifies for file descriptors are readFileDescriptor and writeFileDescriptor, it hardly seems worth introducing new targets to poll a file descriptor for reading and writing. Or maybe you are suggesting an alternative permission check? -Alan From david.lloyd at redhat.com Wed Jun 20 13:48:14 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 20 Jun 2018 08:48:14 -0500 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On Wed, Jun 20, 2018 at 8:41 AM Alan Bateman wrote: > > I disagree with these security checks. Nothing in this API allows the > > caller to actually read or write the FD; that would have to be > > provided separately. Given that each of the checks are relatively > > expensive, IMO they should be deferred to the point where an object is > > constructed that actually has read and/or write capability. > A permission check is needed here. The two permission targets that > RuntimePermission specifies for file descriptors are readFileDescriptor > and writeFileDescriptor, it hardly seems worth introducing new targets > to poll a file descriptor for reading and writing. Or maybe you are > suggesting an alternative permission check? Why is the permission check needed? I believe that there is no material sensitive information that can be gleaned from the readability or writability of a file descriptor. -- - DML From chris.hegarty at oracle.com Wed Jun 20 14:30:44 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Jun 2018 15:30:44 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On 20/06/18 14:16, David Lloyd wrote: > A few comments... > > + public interface SelectableChannelCloser { > > I wonder if some basic implementation should be provided for this. I > can't think of a case where a user would be able to implement this > interface without native code of some sort (is this the intention?). I don't think that a default implementation is not needed here, or even how one could be written without making a number of assumptions. Close, just like read and write, needs to be handled by whatever external entity is interested in the readiness information. > The webrev with the proposed API is here: > http://cr.openjdk.java.net/~alanb/8198372/webrev/index.html A few minor typos: "selected-kkey" "wth" Otherwise, I think this quite good. It provides a low-level mechanism to inter-operate with other socket types on the system. -Chris. From chris.hegarty at oracle.com Wed Jun 20 14:43:19 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Jun 2018 15:43:19 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: <5c3132c2-f46c-e4c4-3f90-ee0c838bad78@oracle.com> What I meant to say: I don't think that a default implementation is needed here, or even how one could be written without making a number of assumptions. Close, just like read and write, needs to be handled by whatever external entity is interested in the readiness information. The callback to coordinate closure with the Selector is required to safely de-register registered channels. I think this is fine as is. -Chris. From Alan.Bateman at oracle.com Wed Jun 20 14:52:16 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jun 2018 15:52:16 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On 20/06/2018 14:48, David Lloyd wrote: > Why is the permission check needed? I believe that there is no > material sensitive information that can be gleaned from the > readability or writability of a file descriptor. > Readiness is an interesting side channel so a permission check is needed. The proposal here is attempting to be consistent with existing APIs that do the same check when creating streams to an existing FileDescriptor. -Alan From brian.burkhalter at oracle.com Wed Jun 20 19:28:35 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Jun 2018 12:28:35 -0700 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On Jun 20, 2018, at 7:30 AM, Chris Hegarty wrote: > > The webrev with the proposed API is here: > > http://cr.openjdk.java.net/~alanb/8198372/webrev/index.html > > A few minor typos: "selected-kkey" "wth" > > Otherwise, I think this quite good. It provides a low-level > mechanism to inter-operate with other socket types on the > system. I concur. Just one minor comment: Channels.java: 156: Could this line be removed as a null fd would be detected implicitly at 158? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 20 19:40:36 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Jun 2018 12:40:36 -0700 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On Jun 20, 2018, at 12:28 PM, Brian Burkhalter wrote: > On Jun 20, 2018, at 7:30 AM, Chris Hegarty wrote: > >> > The webrev with the proposed API is here: >> > http://cr.openjdk.java.net/~alanb/8198372/webrev/index.html >> >> A few minor typos: "selected-kkey" "wth" >> >> Otherwise, I think this quite good. It provides a low-level >> mechanism to inter-operate with other socket types on the >> system. > > I concur. Just one minor comment: > > Channels.java: > > 156: Could this line be removed as a null fd would be detected implicitly at 158? Actually I have one more comment. Might readWriteSelectableChannel() be better named newReadWriteSelectableChannel for consistency with java.nio.channels.Channels? Or does this too much imply the existence of a non-existent entity named ReadWriteSelectableChannel? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Jun 20 19:57:10 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Jun 2018 12:57:10 -0700 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: <7D4468EF-590A-4EB2-BCD6-8D63246D9106@oracle.com> On Jun 20, 2018, at 12:40 PM, Brian Burkhalter wrote: > Actually I have one more comment. Some more comments (which I initially had as comments on the CSR): 1: The phrasing "create a SelectableChannel TO a FileDescriptor" seems awkward. The verbiage in the java.nio.channels package description uses "from," e.g., "An appropriate channel can be constructed from an InputStream or an OutputStream, and conversely an InputStream or an OutputStream can be constructed from a channel." 2: There are two occurrences of {@link java.nio.channels.Selector Selectors} and one of {@code Selector}s. Should not the "s" be either in or out consistently? 3: In the API Note of implReleaseChannel it states "may be invoked while synchronized on the selector and its selected-key set." Should "may" instead be "should?? 4: For readWriteSelectableChannel might the phrasing "read and write directly with the file descriptor" be better as "read from and write to the resource referenced by the file descriptor" ? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu Jun 21 00:53:18 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 20 Jun 2018 17:53:18 -0700 Subject: trivial patch to reduce allocations in nio open* methods In-Reply-To: References: <158BFF8B-586B-4C1C-8DBE-5C5DD3E8D66B@oracle.com> <2ba60a55-baf3-8542-dac6-70c33e3f5f25@oracle.com> <0803a885-5938-6549-262e-40064d73c513@oracle.com> <5aa714cf.c30c1c0a.fbbdc.b3fb@mx.google.com> <4b6b5715-f643-b945-053d-b77889a7d554@oracle.com> <17A3D1A3-9157-444B-8B93-77D3174F9CA8@oracle.com> Message-ID: On Jun 20, 2018, at 1:52 AM, Alan Bateman wrote: >> I don?t see what you are referring to. There is no use of HashMap in these classes. The HashSets are created to be of a fixed size as they are not resized later. Perhaps Set.of(E? elements) should be used to create these Sets? >> >> > FileSystemProvider L430. This has been changed to create the HashSet with an initial capacity of len+1 so it's ignoring the load factor. I'm just saying that the 4 places where this has changed can be improved to avoid resizing every time. My doubt is that I don?t see where these would ever be resized. > Given that the number of options is small is almost all cases then it may be simpler to just use the no-arg constructor for now. So changed (see below). Thanks, Brian --- a/src/java.base/share/classes/java/nio/channels/AsynchronousFileChannel.java +++ b/src/java.base/share/classes/java/nio/channels/AsynchronousFileChannel.java @@ -308,1 +308,1 @@ - set = new HashSet<>(options.length); + set = new HashSet<>(); --- a/src/java.base/share/classes/java/nio/channels/FileChannel.java +++ b/src/java.base/share/classes/java/nio/channels/FileChannel.java @@ -342,1 +342,1 @@ - set = new HashSet<>(options.length); + set = new HashSet<>(); --- a/src/java.base/share/classes/java/nio/file/Files.java +++ b/src/java.base/share/classes/java/nio/file/Files.java @@ -417,1 +417,1 @@ - set = new HashSet<>(options.length); + set = new HashSet<>(); --- a/src/java.base/share/classes/java/nio/file/spi/FileSystemProvider.java +++ b/src/java.base/share/classes/java/nio/file/spi/FileSystemProvider.java @@ -430,1 +430,1 @@ - opts = new HashSet<>(len + 1); + opts = new HashSet<>(); -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 21 06:42:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jun 2018 07:42:34 +0100 Subject: trivial patch to reduce allocations in nio open* methods In-Reply-To: References: <158BFF8B-586B-4C1C-8DBE-5C5DD3E8D66B@oracle.com> <2ba60a55-baf3-8542-dac6-70c33e3f5f25@oracle.com> <0803a885-5938-6549-262e-40064d73c513@oracle.com> <5aa714cf.c30c1c0a.fbbdc.b3fb@mx.google.com> <4b6b5715-f643-b945-053d-b77889a7d554@oracle.com> <17A3D1A3-9157-444B-8B93-77D3174F9CA8@oracle.com> Message-ID: On 21/06/2018 01:53, Brian Burkhalter wrote: > > So changed (see below). This looks okay when added to the patch in the previous mail. -Alan From Alan.Bateman at oracle.com Thu Jun 21 17:12:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jun 2018 18:12:38 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: References: Message-ID: On 20/06/2018 20:28, Brian Burkhalter wrote: > : > > Channels.java: > > 156:Could this line be removed as a null fd would be detected > implicitly at 158? > Yes, an implicit null check should be okay here. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Jun 21 17:32:52 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jun 2018 18:32:52 +0100 Subject: 8198372: API to create a SelectableChannel to a FileDescriptor In-Reply-To: <7D4468EF-590A-4EB2-BCD6-8D63246D9106@oracle.com> References: <7D4468EF-590A-4EB2-BCD6-8D63246D9106@oracle.com> Message-ID: <958e5f1e-e6a7-e09e-c962-f1f79a961def@oracle.com> On 20/06/2018 20:57, Brian Burkhalter wrote: > : > > Some more comments (which I initially had as comments on the CSR): > > 1: The phrasing "create a SelectableChannel /TO/ a FileDescriptor" > seems awkward. The verbiage in the java.nio.channels package > description uses "from," e.g., > > "An appropriate channel can be constructed from an InputStream or an > OutputStream, and conversely an InputStream or an OutputStream can be > constructed from a channel." > Channel is defined to be ".. an open connection to an entity such as ..." so I tried to be consistent with that. You are right that the wording in the Channels utility class is a bit different as it is defines factory methods for creating adaptors.? So I think I'll leave it as is. > 2: There are two occurrences of {@link java.nio.channels.Selector > Selectors} and one of {@code Selector}s. Should not the "s" be either > in or out consistently? You are right, we should try to keep these consistent. > > 3: In the API Note of implReleaseChannel it states "may be invoked > while synchronized on the selector and its selected-key set." Should > "may" instead be "should?? Good point, it should "is invoked" rather than "may be invoked". > > 4: For readWriteSelectableChannel might the phrasing > > "read and write directly with the file descriptor" > > be better as > > "read from and write to the resource referenced by the file descriptor" ? > Yes, that would be better. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Thu Jun 21 19:44:42 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Thu, 21 Jun 2018 19:44:42 +0000 Subject: adding rsockets support into JDK In-Reply-To: <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> Hi All, Here is the webrev.06 version of the patch. http://cr.openjdk.java.net/~ylu/8195160.06/ In this version, I did following changes: 1. Adding test cases for supported RDMA socket options. 2. Fixed an issue inside jdk.net.Sockets.openRdmaSocket. Previously, a new Socket is created to check the support for SO_REUSEPORT and TCP_QUICKACT. This sets the static variable optionSet in java.net.Socket. Later, when RdmaSocket is created, the TCP supportedOptions will be returned instead of RDMA socket supportedOptions. To solve this, I reset optionSet variable to FALSE when creating the RdmaSocket 3. Fixed an issue inside java.net.Socket. When a Rdma Socket is created with RdmaSocketImpl, java.net.Socket cannot directly call jdk.net.RdmaSocketImpl.setSocket. Instead, java.net.Socketimpl.setSocket is called. Same issue with Rdma ServerSocket. Current solution is to make java.net.SocketImpl.setSocket from package private to protected and override it inside rdma.ch.RdmaSocketImpl. This causes additional API changes. I also tried to keep setSocket from java.base as package private but make rdma.ch.setSocket public and have java.net.Socket directly call it. However, this does not work for some reason. Test shows java.net.SocketImpl.setSocket being called instead. If you have any suggestions on this item, please let me know! 4. Set the value limit for SO_SNDBUF and SO_RCVBUF due to RDMA sockets are not using kernel limits. 5. Throw UOE when setting SO_SNDBUF, SO_RCVBUF and SO_REUSEADDR after bind and connect. Please review the patch and let me know your feedback. Thanks, Lucy >-----Original Message----- >From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >Sent: Monday, June 18, 2018 11:54 PM >To: Lu, Yingqi ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, Shirish >; Kaczmarek, Eric >Subject: Re: adding rsockets support into JDK > >On 18/06/2018 22:19, Lu, Yingqi wrote: >> Hi All, >> >> I just created a CSR at the meantime. The link is available at >https://bugs.openjdk.java.net/browse/JDK-8205186. >> >> Please review this and let me know your feedback. >> >The CSR will need to include the javadoc for each of the methods. That isn't fully >fleshed out in the webrev yet. Do you need help on this? > >The other thing is that adds a protected constructor to java.net.ServerSocket (to >match the equivalent in java.net.Socket) so that will need to be included too. This >addition means the scope changes to Java SE too. > >-Alan From yingqi.lu at intel.com Sat Jun 23 00:56:05 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Sat, 23 Jun 2018 00:56:05 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD0D0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AD4CC@ORSMSX113.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> Hi All, Here is the version 7 of the patch. http://cr.openjdk.java.net/~ylu/8195160.07/ In this version, I did: 1. Added SO_LINGER supports (librdmacm supports it) to RDMA based sockets and socket channels. Tests are also updated with this newly added Socket Option. 2. Added a private static variable rdmaOptionsSet to both java.net.Socket and java.net.ServerSocket. This way, I think the supportedOptions will return correct values between RDMA based sockets and regular sockets regardless of who is created first. Please let me know if I missed any cases here. Please review the patch and let me know your feedback. Thanks, Lucy >-----Original Message----- >From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, >Yingqi >Sent: Thursday, June 21, 2018 12:45 PM >To: Alan Bateman ; nio-dev at openjdk.java.net >Cc: Viswanathan, Sandhya ; Aundhe, >Shirish ; Kaczmarek, Eric > >Subject: RE: adding rsockets support into JDK > >Hi All, > >Here is the webrev.06 version of the patch. >http://cr.openjdk.java.net/~ylu/8195160.06/ > >In this version, I did following changes: > >1. Adding test cases for supported RDMA socket options. > >2. Fixed an issue inside jdk.net.Sockets.openRdmaSocket. Previously, a new >Socket is created to check the support for SO_REUSEPORT and >TCP_QUICKACT. This sets the static variable optionSet in java.net.Socket. >Later, when RdmaSocket is created, the TCP supportedOptions will be >returned instead of RDMA socket supportedOptions. To solve this, I reset >optionSet variable to FALSE when creating the RdmaSocket > >3. Fixed an issue inside java.net.Socket. When a Rdma Socket is created with >RdmaSocketImpl, java.net.Socket cannot directly call >jdk.net.RdmaSocketImpl.setSocket. Instead, java.net.Socketimpl.setSocket is >called. Same issue with Rdma ServerSocket. Current solution is to make >java.net.SocketImpl.setSocket from package private to protected and >override it inside rdma.ch.RdmaSocketImpl. This causes additional API >changes. I also tried to keep setSocket from java.base as package private but >make rdma.ch.setSocket public and have java.net.Socket directly call it. >However, this does not work for some reason. Test shows >java.net.SocketImpl.setSocket being called instead. If you have any >suggestions on this item, please let me know! > >4. Set the value limit for SO_SNDBUF and SO_RCVBUF due to RDMA sockets >are not using kernel limits. > >5. Throw UOE when setting SO_SNDBUF, SO_RCVBUF and SO_REUSEADDR >after bind and connect. > >Please review the patch and let me know your feedback. > >Thanks, >Lucy > >>-----Original Message----- >>From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >>Sent: Monday, June 18, 2018 11:54 PM >>To: Lu, Yingqi ; nio-dev at openjdk.java.net >>Cc: Viswanathan, Sandhya ; Aundhe, >>Shirish ; Kaczmarek, Eric >> >>Subject: Re: adding rsockets support into JDK >> >>On 18/06/2018 22:19, Lu, Yingqi wrote: >>> Hi All, >>> >>> I just created a CSR at the meantime. The link is available at >>https://bugs.openjdk.java.net/browse/JDK-8205186. >>> >>> Please review this and let me know your feedback. >>> >>The CSR will need to include the javadoc for each of the methods. That >>isn't fully fleshed out in the webrev yet. Do you need help on this? >> >>The other thing is that adds a protected constructor to >>java.net.ServerSocket (to match the equivalent in java.net.Socket) so >>that will need to be included too. This addition means the scope changes to >Java SE too. >> >>-Alan From Alan.Bateman at oracle.com Sun Jun 24 15:58:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 24 Jun 2018 16:58:11 +0100 Subject: Please test jdk-11+19 Message-ID: <23d66d97-6cfa-9ac4-71bf-180236f28ddd@oracle.com> Mark's mail to jdk-dev [1] reminds us that JDK 11 enters Rampdown Phase One next week and that jdk/jdk will be forked to create the JDK 11 stabilization repository. Those following the patches and discussions here over the last few months will know that there have been a lot of changes to the SelectableChannel and Selector implementations in JDK 11. In particular, 5 of the 6 Selector implementations have been mostly re-implemented (the Windows implementation will need to wait for another day), SocketChannel and the other selectable channel implementations have had significant updates to replace the locking for asynchronous close and interrupt, and there have been several changes to align the specification and implementation. In addition, Selector and SelectionKey have several new APIs. There are also improvements to the buffer implementations and a number of API addition to the Files API. This mail is just a reminder that if you are invested in this area then now is the right time to test with the latest JDK 11 early access build [2]. The releases notes [3] document all the behavioral changes that we identified here. So far the only bug reports with the changes have been with unspecified behavior but it's important to discuss all behavior changes as they may not be intentional. -Alan [1] http://mail.openjdk.java.net/pipermail/jdk-dev/2018-June/001462.html [2] http://jdk.java.net/11/ [3] http://jdk.java.net/11/release-notes From Alan.Bateman at oracle.com Sun Jun 24 19:11:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 24 Jun 2018 20:11:04 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B29CADD.7090200@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> Message-ID: On 20/06/2018 04:32, Joe Wang wrote: > Thanks Alan.? I created 8205058 to capture your suggestions. Please > see below for more details. > > Changed the internal APIs to throw CCE instead. In the same way as the > previous changeset for 8201276, these methods are made specific for > the use cases (though they are now for Files.read/writeString only) so > that they are not mixed up with existing ones that may inadvertently > affect other usages. > > One thing to note is that MalformedInputException or > UnmappableCharacterException will lose one piece of information in > comparison to the existing IAE, that is where it happens (offset). > Should there be an improvement in the future, we could consider add it > back to this part of code. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 > webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ I see your point on MalformedInputException and UnmappableCharacterException so maybe we can submit a new issue to track the follow-on work. The update means a lot of duplication in StringCoding. Did you consider (or measure) having the private encode/decode methods take a parameter to indicate the exception handling? Sherman might have suggestions on this. In the tests, shouldn't testMalformedRead and testMalformedWrite be updated so that expectedExceptions lists the specify exception that is expected? If I read the update correctly then isn't checking for MalformedInputException and UnmappableCharacterException anywhere (it passes if IOException is thrown). -Alan From Alan.Bateman at oracle.com Mon Jun 25 11:52:00 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jun 2018 12:52:00 +0100 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> Message-ID: <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> On 23/06/2018 01:56, Lu, Yingqi wrote: > Hi All, > > Here is the version 7 of the patch. http://cr.openjdk.java.net/~ylu/8195160.07/ > > In this version, I did: > > 1. Added SO_LINGER supports (librdmacm supports it) to RDMA based sockets and socket channels. Tests are also updated with this newly added Socket Option. > > 2. Added a private static variable rdmaOptionsSet to both java.net.Socket and java.net.ServerSocket. This way, I think the supportedOptions will return correct values between RDMA based sockets and regular sockets regardless of who is created first. Please let me know if I missed any cases here. > I went over the API again. The factory methods don't have complete javadoc yet and I'm happy to help with that, see below. The other part of the API is jdk.net.RdmaSocketOptions. The SQSIZE, RQSIZE and INLINE constants are currently public so they are leaking into the API. The API elements are RMDA_SQSIZE, RMDA_RQSIZE and RQSIZE_INLINE and these will need javadoc. Can you java.net.StandardSocketOption as an example of how the existing socket options are specified? -Alan. A suggestion to add to the jdk.net.Sockets class description: ?*

The {@link #openRdmaSelector() openRdmaSelector}, {@link ?* #openRdmaSocketChannel() openRdmaSocketChannel}, and {@link ?* #openRdmaServerSocketChannel() openRdmaServerSocketChannel} methods ?* create selectors and selectable channels for use with RDMA sockets. ?* These objects are created a {@link java.nio.channels.spi.SelectorProvider ?* SelectorProvider} that is not the default {@code SelectorProvider}. ?* Consequently, selectable channels to RDMA sockets may not be multiplexed ?* with selectable channel created by the default selector provider. The ?* selector provider does not support datagram channels, its {@code ?* openDatagramChannel} methods throw {@link UnsupportedOperationException}. A suggestion for openRdmaSocketChannel: ??? /** ???? * Opens a socket channel to a RDMA socket. A newly-created socket channel ???? * is {@link SocketChannel#isOpen() open}, not yet bound to a {@link ???? * SocketChannel#getLocalAddress() local address}, and not yet ???? * {@link SocketChannel#isConnected() connected}. ???? * ???? *

A socket channel to a RDMA socket supports all of the socket options ???? * specified by {@code SocketChannel}. In addition, it also supports the ???? * socket options specified by {@link RdmaSocketOptions}. ???? * ???? * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. ???? * Consequently, attempting to bind or connect to an IPv6 address will fail ???? * with {@code UnsupportedAddressTypeException}. ???? * ???? * @throws IOException ???? *???????? If an I/O error occurs ???? * @throws UnsupportedOperationException ???? *???????? If RDMA sockets are not supported on this platform ???? * ???? * @since N ???? */ A suggestion for openRdmaServerSocketChannel: ??? /** ???? * Opens a server-socket channel to a RDMA socket. A newly-created socket ???? * channel is {@link SocketChannel#isOpen() open} but not yet bound to a ???? * {@link SocketChannel#getLocalAddress() local address}. ???? * ???? * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. ???? * Consequently, attempting to bind to an IPv6 address will fail with ???? * {@code UnsupportedAddressTypeException}. ???? * ???? * @throws IOException ???? *???????? If an I/O error occurs ???? * @throws UnsupportedOperationException ???? *???????? If RDMA sockets are not supported on this platform ???? * ???? * @since N ???? */ A suggestion for openRdmaSelector: ??? /** ???? * Opens a selector to multiplex selectable channels to RDMA sockets. ???? * ???? * @throws IOException ???? *???????? If an I/O error occurs ???? * @throws UnsupportedOperationException ???? *???????? If RDMA sockets are not supported on this platform ???? * ???? * @since N ???? */ A suggestion for openRdmaSocket: ??? /** ???? * Creates an unconnected RDMA socket. ???? * ???? *

A RDMA socket supports the same socket options that that {@code ???? * java.net.Socket} defines. In addition, it also supports the socket options ???? * specified by {@link RdmaSocketOptions}. ???? * ???? * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. ???? * Consequently, attempting to bind or connect to an IPv6 address will fail ???? * with {@code IllegalArgumentException}. ???? * ???? * @throws IOException ???? *???????? If an I/O error occurs ???? * @throws UnsupportedOperationException ???? *???????? If RDMA sockets are not supported on this platform ???? * ???? * @since N ???? */ A suggestion for openRdmaServerSocket: ??? /** ???? * Creates an unbound RDMA server socket. ???? * ???? *

A RDMA socket supports the same socket options that that {@code ???? * java.net.ServerSocket} defines. ???? * ???? * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. ???? * Consequently, attempting to bind to an IPv6 address will fail with ???? * {@code IllegalArgumentException}. ???? * ???? * @throws IOException ???? *???????? If an I/O error occurs ???? * @throws UnsupportedOperationException ???? *???????? If RDMA sockets are not supported on this platform ???? * ???? * @since N ???? */ From yingqi.lu at intel.com Mon Jun 25 14:19:21 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Mon, 25 Jun 2018 14:19:21 +0000 Subject: adding rsockets support into JDK In-Reply-To: <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <1c7d0657-3327-1349-d9e6-d12113c10aa4@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13AF403@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13B4D12@ORSMSX113.amr.corp.intel.com> <440c90b4-ad38-568c-541c-9914d2139603@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com>, <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> Message-ID: Hi Alan, Thank you very much for the help on the javadoc changes. I will put them into new webrev. I will modify the RDMA socket options following the example of standard socket options as well and submit version 8 for review today. Thank, Lucy Sent from my iPhone > On Jun 25, 2018, at 4:51 AM, Alan Bateman wrote: > >> On 23/06/2018 01:56, Lu, Yingqi wrote: >> Hi All, >> >> Here is the version 7 of the patch. http://cr.openjdk.java.net/~ylu/8195160.07/ >> >> In this version, I did: >> >> 1. Added SO_LINGER supports (librdmacm supports it) to RDMA based sockets and socket channels. Tests are also updated with this newly added Socket Option. >> >> 2. Added a private static variable rdmaOptionsSet to both java.net.Socket and java.net.ServerSocket. This way, I think the supportedOptions will return correct values between RDMA based sockets and regular sockets regardless of who is created first. Please let me know if I missed any cases here. >> > I went over the API again. The factory methods don't have complete javadoc yet and I'm happy to help with that, see below. > > The other part of the API is jdk.net.RdmaSocketOptions. The SQSIZE, RQSIZE and INLINE constants are currently public so they are leaking into the API. The API elements are RMDA_SQSIZE, RMDA_RQSIZE and RQSIZE_INLINE and these will need javadoc. Can you java.net.StandardSocketOption as an example of how the existing socket options are specified? > > -Alan. > > > A suggestion to add to the jdk.net.Sockets class description: > > *

The {@link #openRdmaSelector() openRdmaSelector}, {@link > * #openRdmaSocketChannel() openRdmaSocketChannel}, and {@link > * #openRdmaServerSocketChannel() openRdmaServerSocketChannel} methods > * create selectors and selectable channels for use with RDMA sockets. > * These objects are created a {@link java.nio.channels.spi.SelectorProvider > * SelectorProvider} that is not the default {@code SelectorProvider}. > * Consequently, selectable channels to RDMA sockets may not be multiplexed > * with selectable channel created by the default selector provider. The > * selector provider does not support datagram channels, its {@code > * openDatagramChannel} methods throw {@link UnsupportedOperationException}. > > A suggestion for openRdmaSocketChannel: > > /** > * Opens a socket channel to a RDMA socket. A newly-created socket channel > * is {@link SocketChannel#isOpen() open}, not yet bound to a {@link > * SocketChannel#getLocalAddress() local address}, and not yet > * {@link SocketChannel#isConnected() connected}. > * > *

A socket channel to a RDMA socket supports all of the socket options > * specified by {@code SocketChannel}. In addition, it also supports the > * socket options specified by {@link RdmaSocketOptions}. > * > * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. > * Consequently, attempting to bind or connect to an IPv6 address will fail > * with {@code UnsupportedAddressTypeException}. > * > * @throws IOException > * If an I/O error occurs > * @throws UnsupportedOperationException > * If RDMA sockets are not supported on this platform > * > * @since N > */ > > A suggestion for openRdmaServerSocketChannel: > > /** > * Opens a server-socket channel to a RDMA socket. A newly-created socket > * channel is {@link SocketChannel#isOpen() open} but not yet bound to a > * {@link SocketChannel#getLocalAddress() local address}. > * > * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. > * Consequently, attempting to bind to an IPv6 address will fail with > * {@code UnsupportedAddressTypeException}. > * > * @throws IOException > * If an I/O error occurs > * @throws UnsupportedOperationException > * If RDMA sockets are not supported on this platform > * > * @since N > */ > > A suggestion for openRdmaSelector: > > /** > * Opens a selector to multiplex selectable channels to RDMA sockets. > * > * @throws IOException > * If an I/O error occurs > * @throws UnsupportedOperationException > * If RDMA sockets are not supported on this platform > * > * @since N > */ > > A suggestion for openRdmaSocket: > > /** > * Creates an unconnected RDMA socket. > * > *

A RDMA socket supports the same socket options that that {@code > * java.net.Socket} defines. In addition, it also supports the socket options > * specified by {@link RdmaSocketOptions}. > * > * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. > * Consequently, attempting to bind or connect to an IPv6 address will fail > * with {@code IllegalArgumentException}. > * > * @throws IOException > * If an I/O error occurs > * @throws UnsupportedOperationException > * If RDMA sockets are not supported on this platform > * > * @since N > */ > > A suggestion for openRdmaServerSocket: > > /** > * Creates an unbound RDMA server socket. > * > *

A RDMA socket supports the same socket options that that {@code > * java.net.ServerSocket} defines. > * > * @apiNote The rsocket implementation on Linux only supports IPv4 addresses. > * Consequently, attempting to bind to an IPv6 address will fail with > * {@code IllegalArgumentException}. > * > * @throws IOException > * If an I/O error occurs > * @throws UnsupportedOperationException > * If RDMA sockets are not supported on this platform > * > * @since N > */ From Alan.Bateman at oracle.com Mon Jun 25 15:40:27 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jun 2018 16:40:27 +0100 Subject: 8205612: (fc) Files.readAllBytes fails with ClosedByInterruptException when interrupt status set Message-ID: I brought up the problems with FileChannel implementing InterruptibleChannel a few months ago but we didn't make progress on agreeing an API change at the time. In the mean-time, "simple APIs" such as Files.readAllBytes(...), Files.newInputStream(...).read(...) and other fail with ClosedByInterruptException when called with the interrupt status set. Another example is class loading from exploding modules and that also makes use of FileChannel under the covers too. I'd like to put in an interim solution so that the simple APIs can use FileChannel without being concerned about the interrupt status. The changes mean InputStream, OutputStream, BufferedReader, and BufferedWriter objects obtained from the default provider are non-interruptible. Simple APIs such as readString, readAllBytes, lines, ... will also be non-interruptible. APIs that expose channels are not impacted, this will be continue to be interruptible. The changes are very simple. We also had some of this in place already for the jimage code (as it uses FileChannel when configured not to mmap the jimage file). http://cr.openjdk.java.net/~alanb/8205612/webrev/index.html -Alan From brian.burkhalter at oracle.com Mon Jun 25 16:38:14 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 25 Jun 2018 09:38:14 -0700 Subject: 8205612: (fc) Files.readAllBytes fails with ClosedByInterruptException when interrupt status set In-Reply-To: References: Message-ID: <83B74ACB-3CB2-4122-A0A4-21348EE4D1D6@oracle.com> Looks fine. Brian On Jun 25, 2018, at 8:40 AM, Alan Bateman wrote: > I'd like to put in an interim solution so that the simple APIs can use FileChannel without being concerned about the interrupt status. [?] > > The changes are very simple. We also had some of this in place already for the jimage code (as it uses FileChannel when configured not to mmap the jimage file). > > http://cr.openjdk.java.net/~alanb/8205612/webrev/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From huizhe.wang at oracle.com Mon Jun 25 16:48:02 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 09:48:02 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> Message-ID: <5B311CC2.2060005@oracle.com> On 6/24/18, 12:11 PM, Alan Bateman wrote: > On 20/06/2018 04:32, Joe Wang wrote: >> Thanks Alan. I created 8205058 to capture your suggestions. Please >> see below for more details. >> >> Changed the internal APIs to throw CCE instead. In the same way as >> the previous changeset for 8201276, these methods are made specific >> for the use cases (though they are now for Files.read/writeString >> only) so that they are not mixed up with existing ones that may >> inadvertently affect other usages. >> >> One thing to note is that MalformedInputException or >> UnmappableCharacterException will lose one piece of information in >> comparison to the existing IAE, that is where it happens (offset). >> Should there be an improvement in the future, we could consider add >> it back to this part of code. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ > > I see your point on MalformedInputException and > UnmappableCharacterException so maybe we can submit a new issue to > track the follow-on work. Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 > > The update means a lot of duplication in StringCoding. Did you > consider (or measure) having the private encode/decode methods take a > parameter to indicate the exception handling? Sherman might have > suggestions on this. I did. I didn't like the code duplication at all. But it would be even messier if we reuse the code since the previous usage didn't declare checked exception, but did capture the position (offset) and length information, which means the new method while unnecessary to capture these information for Files read/writeString would still need to save them in case Exception occurs. Because of the complication, I thought you and Sherman would again prefer a specific methods than adding parameters (need fields as well). Furthermore, in the first round of the review, Sherman mentioned that he's been working on an improvement in this area. But I'll wait till Sherman could comment on this. > > In the tests, shouldn't testMalformedRead and testMalformedWrite be > updated so that expectedExceptions lists the specify exception that is > expected? If I read the update correctly then isn't checking for > MalformedInputException and UnmappableCharacterException anywhere (it > passes if IOException is thrown). MalformedInputException and UnmappableCharacterException are implementation details, the tests only verified what the spec required (IOE). -Joe > > -Alan > From huizhe.wang at oracle.com Mon Jun 25 22:41:58 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 15:41:58 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B311CC2.2060005@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> Message-ID: <5B316FB6.7020308@oracle.com> Hi Alan, The test testMalformedRead and testMalformedWrite now expect an UnmappableCharacterException instead of IOE: webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ Thanks, Joe On 6/25/18, 9:48 AM, Joe Wang wrote: > > > On 6/24/18, 12:11 PM, Alan Bateman wrote: >> On 20/06/2018 04:32, Joe Wang wrote: >>> Thanks Alan. I created 8205058 to capture your suggestions. Please >>> see below for more details. >>> >>> Changed the internal APIs to throw CCE instead. In the same way as >>> the previous changeset for 8201276, these methods are made specific >>> for the use cases (though they are now for Files.read/writeString >>> only) so that they are not mixed up with existing ones that may >>> inadvertently affect other usages. >>> >>> One thing to note is that MalformedInputException or >>> UnmappableCharacterException will lose one piece of information in >>> comparison to the existing IAE, that is where it happens (offset). >>> Should there be an improvement in the future, we could consider add >>> it back to this part of code. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >> >> I see your point on MalformedInputException and >> UnmappableCharacterException so maybe we can submit a new issue to >> track the follow-on work. > > Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 > >> >> The update means a lot of duplication in StringCoding. Did you >> consider (or measure) having the private encode/decode methods take a >> parameter to indicate the exception handling? Sherman might have >> suggestions on this. > > I did. I didn't like the code duplication at all. But it would be even > messier if we reuse the code since the previous usage didn't declare > checked exception, but did capture the position (offset) and length > information, which means the new method while unnecessary to capture > these information for Files read/writeString would still need to save > them in case Exception occurs. Because of the complication, I thought > you and Sherman would again prefer a specific methods than adding > parameters (need fields as well). > > Furthermore, in the first round of the review, Sherman mentioned that > he's been working on an improvement in this area. But I'll wait till > Sherman could comment on this. > >> >> In the tests, shouldn't testMalformedRead and testMalformedWrite be >> updated so that expectedExceptions lists the specify exception that >> is expected? If I read the update correctly then isn't checking for >> MalformedInputException and UnmappableCharacterException anywhere (it >> passes if IOException is thrown). > > MalformedInputException and UnmappableCharacterException are > implementation details, the tests only verified what the spec required > (IOE). > > -Joe > >> >> -Alan >> From brian.burkhalter at oracle.com Mon Jun 25 23:26:19 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 25 Jun 2018 16:26:19 -0700 Subject: 8205641: Make java/nio/channels/Selector/RacyDeregister.java noisier and flush after prints Message-ID: <5A8231BB-61DF-410E-9940-91C2F2ECA9D7@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8205641 http://cr.openjdk.java.net/~bpb/8205641/webrev.00/ Please review this small sub-task patch to make some changes to the test which will hopefully facilitate diagnosing the timeouts observed. Thanks, Brian From huizhe.wang at oracle.com Tue Jun 26 04:50:13 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 25 Jun 2018 21:50:13 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B316FB6.7020308@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> Message-ID: <5B31C605.5070001@oracle.com> Hi Alan, Sherman, Here's a version where we, as Sherman suggested, throw an IAE with CCE as the cause. This approach reduces code duplication in SC, although it complicates the impl a little bit with the added parameter and the different behavior between the existing usages of the methods and the new ones. The existing code paths are kept intact so there's no compatibility issue for the existing code. This version also did not remove the try-catch in Files as Alan suggested earlier. http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ Thanks, Joe On 6/25/18, 3:41 PM, Joe Wang wrote: > Hi Alan, > > The test testMalformedRead and testMalformedWrite now expect an > UnmappableCharacterException instead of IOE: > > webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ > > Thanks, > Joe > > On 6/25/18, 9:48 AM, Joe Wang wrote: >> >> >> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>> On 20/06/2018 04:32, Joe Wang wrote: >>>> Thanks Alan. I created 8205058 to capture your suggestions. Please >>>> see below for more details. >>>> >>>> Changed the internal APIs to throw CCE instead. In the same way as >>>> the previous changeset for 8201276, these methods are made specific >>>> for the use cases (though they are now for Files.read/writeString >>>> only) so that they are not mixed up with existing ones that may >>>> inadvertently affect other usages. >>>> >>>> One thing to note is that MalformedInputException or >>>> UnmappableCharacterException will lose one piece of information in >>>> comparison to the existing IAE, that is where it happens (offset). >>>> Should there be an improvement in the future, we could consider add >>>> it back to this part of code. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>> >>> I see your point on MalformedInputException and >>> UnmappableCharacterException so maybe we can submit a new issue to >>> track the follow-on work. >> >> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >> >>> >>> The update means a lot of duplication in StringCoding. Did you >>> consider (or measure) having the private encode/decode methods take >>> a parameter to indicate the exception handling? Sherman might have >>> suggestions on this. >> >> I did. I didn't like the code duplication at all. But it would be >> even messier if we reuse the code since the previous usage didn't >> declare checked exception, but did capture the position (offset) and >> length information, which means the new method while unnecessary to >> capture these information for Files read/writeString would still need >> to save them in case Exception occurs. Because of the complication, I >> thought you and Sherman would again prefer a specific methods than >> adding parameters (need fields as well). >> >> Furthermore, in the first round of the review, Sherman mentioned that >> he's been working on an improvement in this area. But I'll wait till >> Sherman could comment on this. >> >>> >>> In the tests, shouldn't testMalformedRead and testMalformedWrite be >>> updated so that expectedExceptions lists the specify exception that >>> is expected? If I read the update correctly then isn't checking for >>> MalformedInputException and UnmappableCharacterException anywhere >>> (it passes if IOException is thrown). >> >> MalformedInputException and UnmappableCharacterException are >> implementation details, the tests only verified what the spec >> required (IOE). >> >> -Joe >> >>> >>> -Alan >>> From xueming.shen at oracle.com Tue Jun 26 05:28:16 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 25 Jun 2018 22:28:16 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31C605.5070001@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: <5B31CEF0.2070504@oracle.com> Hi Joe, I would suggest always embed a malformed exception into the iae as showed below, then the extra flag is no longer needed. private static void throwMalformed(int off, int nb) { String msg = "malformed input off : " + off + ", length : " + nb; throw new IllegalArgumentException(msg, new MalformedInputException(nb)); } It's an implementation details, so we might be able to get rid of the iae later if we can figure out the better way to pass the malformed exception for both ZipCoder/Files later, without touch the api. An alternative is to move the iae-catch into StringCoding, with pair of similar methods to throw IOE (to add into the JLA), not sure if it's better though. It makes the Files impl cleaner at least. -Sherman On 6/25/18, 9:50 PM, Joe Wang wrote: > Hi Alan, Sherman, > > Here's a version where we, as Sherman suggested, throw an IAE with CCE > as the cause. This approach reduces code duplication in SC, although > it complicates the impl a little bit with the added parameter and the > different behavior between the existing usages of the methods and the > new ones. The existing code paths are kept intact so there's no > compatibility issue for the existing code. > > This version also did not remove the try-catch in Files as Alan > suggested earlier. > > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ > > Thanks, > Joe > > On 6/25/18, 3:41 PM, Joe Wang wrote: >> Hi Alan, >> >> The test testMalformedRead and testMalformedWrite now expect an >> UnmappableCharacterException instead of IOE: >> >> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ >> >> Thanks, >> Joe >> >> On 6/25/18, 9:48 AM, Joe Wang wrote: >>> >>> >>> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>>> On 20/06/2018 04:32, Joe Wang wrote: >>>>> Thanks Alan. I created 8205058 to capture your suggestions. >>>>> Please see below for more details. >>>>> >>>>> Changed the internal APIs to throw CCE instead. In the same way as >>>>> the previous changeset for 8201276, these methods are made >>>>> specific for the use cases (though they are now for >>>>> Files.read/writeString only) so that they are not mixed up with >>>>> existing ones that may inadvertently affect other usages. >>>>> >>>>> One thing to note is that MalformedInputException or >>>>> UnmappableCharacterException will lose one piece of information in >>>>> comparison to the existing IAE, that is where it happens (offset). >>>>> Should there be an improvement in the future, we could consider >>>>> add it back to this part of code. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>>> >>>> I see your point on MalformedInputException and >>>> UnmappableCharacterException so maybe we can submit a new issue to >>>> track the follow-on work. >>> >>> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >>> >>>> >>>> The update means a lot of duplication in StringCoding. Did you >>>> consider (or measure) having the private encode/decode methods take >>>> a parameter to indicate the exception handling? Sherman might have >>>> suggestions on this. >>> >>> I did. I didn't like the code duplication at all. But it would be >>> even messier if we reuse the code since the previous usage didn't >>> declare checked exception, but did capture the position (offset) and >>> length information, which means the new method while unnecessary to >>> capture these information for Files read/writeString would still >>> need to save them in case Exception occurs. Because of the >>> complication, I thought you and Sherman would again prefer a >>> specific methods than adding parameters (need fields as well). >>> >>> Furthermore, in the first round of the review, Sherman mentioned >>> that he's been working on an improvement in this area. But I'll wait >>> till Sherman could comment on this. >>> >>>> >>>> In the tests, shouldn't testMalformedRead and testMalformedWrite be >>>> updated so that expectedExceptions lists the specify exception that >>>> is expected? If I read the update correctly then isn't checking for >>>> MalformedInputException and UnmappableCharacterException anywhere >>>> (it passes if IOException is thrown). >>> >>> MalformedInputException and UnmappableCharacterException are >>> implementation details, the tests only verified what the spec >>> required (IOE). >>> >>> -Joe >>> >>>> >>>> -Alan >>>> From Alan.Bateman at oracle.com Tue Jun 26 08:23:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jun 2018 09:23:33 +0100 Subject: 8205641: Make java/nio/channels/Selector/RacyDeregister.java noisier and flush after prints In-Reply-To: <5A8231BB-61DF-410E-9940-91C2F2ECA9D7@oracle.com> References: <5A8231BB-61DF-410E-9940-91C2F2ECA9D7@oracle.com> Message-ID: <65dd8d5c-8958-ddc5-3b42-7c0f2b739642@oracle.com> On 26/06/2018 00:26, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8205641 > http://cr.openjdk.java.net/~bpb/8205641/webrev.00/ > > Please review this small sub-task patch to make some changes to the test which will hopefully facilitate diagnosing the timeouts observed. > This looks okay to me. BTW: Looking through the trail of issues on this test then it may be worth just re-writing to avoid the spinning threads that seems to cause to take a long time on some Windows machines. -Alan From Alan.Bateman at oracle.com Tue Jun 26 13:54:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jun 2018 14:54:50 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31C605.5070001@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: On 26/06/2018 05:50, Joe Wang wrote: > Hi Alan, Sherman, > > Here's a version where we, as Sherman suggested, throw an IAE with CCE > as the cause. This approach reduces code duplication in SC, although > it complicates the impl a little bit with the added parameter and the > different behavior between the existing usages of the methods and the > new ones. The existing code paths are kept intact so there's no > compatibility issue for the existing code. > > This version also did not remove the try-catch in Files as Alan > suggested earlier. > > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ This version looks much better. In StringCoding, do you really need throwCCE? The encode/decode methods do a replace or throw so I assume one flag will do. If combined with Sherman suggestion then it would be minimal changes to StringCoding. It would be nice to get rid of the IAE completely but that is for another day. In Files then you don't need to check if cause is null before testing its type. The update tests to check for UnmappedCharacterException and MalformedInputException look good. -Alan From huizhe.wang at oracle.com Tue Jun 26 17:39:58 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 10:39:58 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B31CEF0.2070504@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B31CEF0.2070504@oracle.com> Message-ID: <5B327A6E.4000107@oracle.com> Hi Sherman, Combining the msg and cause is a great idea! That allowed us to satisfy the existing code without changes/additional parameter and also the new method for a CCE. I also moved the iae-catch into StringCoding, that makes Files clean of such things. Here's an updated webrev with both suggestions: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ Thanks, Joe On 6/25/18, 10:28 PM, Xueming Shen wrote: > Hi Joe, > > I would suggest always embed a malformed exception into the iae as showed > below, then the extra flag is no longer needed. > > private static void throwMalformed(int off, int nb) { > String msg = "malformed input off : " + off + ", length : " > + nb; > throw new IllegalArgumentException(msg, new > MalformedInputException(nb)); > } > > It's an implementation details, so we might be able to get rid of the > iae later if we > can figure out the better way to pass the malformed exception for both > ZipCoder/Files > later, without touch the api. > > An alternative is to move the iae-catch into StringCoding, with pair > of similar methods > to throw IOE (to add into the JLA), not sure if it's better though. It > makes the Files > impl cleaner at least. > > -Sherman > > On 6/25/18, 9:50 PM, Joe Wang wrote: >> Hi Alan, Sherman, >> >> Here's a version where we, as Sherman suggested, throw an IAE with >> CCE as the cause. This approach reduces code duplication in SC, >> although it complicates the impl a little bit with the added >> parameter and the different behavior between the existing usages of >> the methods and the new ones. The existing code paths are kept intact >> so there's no compatibility issue for the existing code. >> >> This version also did not remove the try-catch in Files as Alan >> suggested earlier. >> >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >> >> Thanks, >> Joe >> >> On 6/25/18, 3:41 PM, Joe Wang wrote: >>> Hi Alan, >>> >>> The test testMalformedRead and testMalformedWrite now expect an >>> UnmappableCharacterException instead of IOE: >>> >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev01/ >>> >>> Thanks, >>> Joe >>> >>> On 6/25/18, 9:48 AM, Joe Wang wrote: >>>> >>>> >>>> On 6/24/18, 12:11 PM, Alan Bateman wrote: >>>>> On 20/06/2018 04:32, Joe Wang wrote: >>>>>> Thanks Alan. I created 8205058 to capture your suggestions. >>>>>> Please see below for more details. >>>>>> >>>>>> Changed the internal APIs to throw CCE instead. In the same way >>>>>> as the previous changeset for 8201276, these methods are made >>>>>> specific for the use cases (though they are now for >>>>>> Files.read/writeString only) so that they are not mixed up with >>>>>> existing ones that may inadvertently affect other usages. >>>>>> >>>>>> One thing to note is that MalformedInputException or >>>>>> UnmappableCharacterException will lose one piece of information >>>>>> in comparison to the existing IAE, that is where it happens >>>>>> (offset). Should there be an improvement in the future, we could >>>>>> consider add it back to this part of code. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8205058 >>>>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev/ >>>>> >>>>> I see your point on MalformedInputException and >>>>> UnmappableCharacterException so maybe we can submit a new issue to >>>>> track the follow-on work. >>>> >>>> Submitted: https://bugs.openjdk.java.net/browse/JDK-8205613 >>>> >>>>> >>>>> The update means a lot of duplication in StringCoding. Did you >>>>> consider (or measure) having the private encode/decode methods >>>>> take a parameter to indicate the exception handling? Sherman might >>>>> have suggestions on this. >>>> >>>> I did. I didn't like the code duplication at all. But it would be >>>> even messier if we reuse the code since the previous usage didn't >>>> declare checked exception, but did capture the position (offset) >>>> and length information, which means the new method while >>>> unnecessary to capture these information for Files read/writeString >>>> would still need to save them in case Exception occurs. Because of >>>> the complication, I thought you and Sherman would again prefer a >>>> specific methods than adding parameters (need fields as well). >>>> >>>> Furthermore, in the first round of the review, Sherman mentioned >>>> that he's been working on an improvement in this area. But I'll >>>> wait till Sherman could comment on this. >>>> >>>>> >>>>> In the tests, shouldn't testMalformedRead and testMalformedWrite >>>>> be updated so that expectedExceptions lists the specify exception >>>>> that is expected? If I read the update correctly then isn't >>>>> checking for MalformedInputException and >>>>> UnmappableCharacterException anywhere (it passes if IOException is >>>>> thrown). >>>> >>>> MalformedInputException and UnmappableCharacterException are >>>> implementation details, the tests only verified what the spec >>>> required (IOE). >>>> >>>> -Joe >>>> >>>>> >>>>> -Alan >>>>> > From huizhe.wang at oracle.com Tue Jun 26 17:41:35 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 10:41:35 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> Message-ID: <5B327ACF.3080804@oracle.com> On 6/26/18, 6:54 AM, Alan Bateman wrote: > On 26/06/2018 05:50, Joe Wang wrote: >> Hi Alan, Sherman, >> >> Here's a version where we, as Sherman suggested, throw an IAE with >> CCE as the cause. This approach reduces code duplication in SC, >> although it complicates the impl a little bit with the added >> parameter and the different behavior between the existing usages of >> the methods and the new ones. The existing code paths are kept intact >> so there's no compatibility issue for the existing code. >> >> This version also did not remove the try-catch in Files as Alan >> suggested earlier. >> >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ > This version looks much better. In StringCoding, do you really need > throwCCE? The encode/decode methods do a replace or throw so I assume > one flag will do. If combined with Sherman suggestion then it would be > minimal changes to StringCoding. It would be nice to get rid of the > IAE completely but that is for another day. In Files then you don't > need to check if cause is null before testing its type. Yes, combined with Sherman's suggestion eliminated the need for the new parameter. Here's the updated webrev: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ > > The update tests to check for UnmappedCharacterException and > MalformedInputException look good. Thanks, Joe > > -Alan > > From lance.andersen at oracle.com Tue Jun 26 18:24:27 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 26 Jun 2018 14:24:27 -0400 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B327ACF.3080804@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: Hi Joe, Should src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java ????? /** * Constructs a new {@code String} by decoding the specified subarray of * bytes using the specified {@linkplain java.nio.charset.Charset charset}. * * The caller of this method shall relinquish and transfer the ownership of * the byte array to the callee since the later will not make a copy. * * @param bytes the byte array source * @param cs the Charset * @return the newly created string * @throws IllegalArgumentException for malformed or unmappable bytes */ String newStringNoRepl(byte[] bytes, Charset cs) throws CharacterCodingException; ???????? include an @throws for CharacterCodingException? > On Jun 26, 2018, at 1:41 PM, Joe Wang wrote: > > > > On 6/26/18, 6:54 AM, Alan Bateman wrote: >> On 26/06/2018 05:50, Joe Wang wrote: >>> Hi Alan, Sherman, >>> >>> Here's a version where we, as Sherman suggested, throw an IAE with CCE as the cause. This approach reduces code duplication in SC, although it complicates the impl a little bit with the added parameter and the different behavior between the existing usages of the methods and the new ones. The existing code paths are kept intact so there's no compatibility issue for the existing code. >>> >>> This version also did not remove the try-catch in Files as Alan suggested earlier. >>> >>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >> This version looks much better. In StringCoding, do you really need throwCCE? The encode/decode methods do a replace or throw so I assume one flag will do. If combined with Sherman suggestion then it would be minimal changes to StringCoding. It would be nice to get rid of the IAE completely but that is for another day. In Files then you don't need to check if cause is null before testing its type. > > Yes, combined with Sherman's suggestion eliminated the need for the new parameter. Here's the updated webrev: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> >> The update tests to check for UnmappedCharacterException and MalformedInputException look good. > > Thanks, > Joe > >> >> -Alan Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oracle_sig_logo.gif Type: image/gif Size: 658 bytes Desc: not available URL: From Alan.Bateman at oracle.com Tue Jun 26 18:57:17 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jun 2018 19:57:17 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B327ACF.3080804@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> On 26/06/2018 18:41, Joe Wang wrote: > > : > > Yes, combined with Sherman's suggestion eliminated the need for the > new parameter. Here's the updated webrev: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ This looks good. Just a minor nit but newStringNoRepl and getBytesNoReply don't need to check if the cause is null. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Jun 26 18:59:59 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 26 Jun 2018 11:59:59 -0700 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive Message-ID: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8202252 http://cr.openjdk.java.net/~bpb/8202252/webrev.00/ Set instance variables {connect,read,write}Handler to null to allow the respective objects to be GCed. Without applying the source portion of the patch, the test fails on Linux, macOS, and Solaris. Thanks, Brian From huizhe.wang at oracle.com Tue Jun 26 19:14:57 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 12:14:57 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> Message-ID: <5B3290B1.4010500@oracle.com> Thanks Lance! Indeed it has been changed to CCE. I'm still familiarizing myself with IntelliJ. Opened it in NetBeans, it clearly indicates "missing @throws tag for " CCE, but I don't see anything like that in IntelliJ. Updated webrev: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ -Joe On 6/26/18, 11:24 AM, Lance Andersen wrote: > Hi Joe, > > Should src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java > > ????? > /** > * Constructs a new {@code String} by decoding the specified subarray of > * bytes using the specified {@linkplain java.nio.charset.Charset charset}. > * > * The caller of this method shall relinquish and transfer the ownership of > * the byte array to the callee since the later will not make a copy. > * > * @param bytes the byte array source > * @param cs the Charset > * @return the newly created string > * @throws IllegalArgumentException for malformed or unmappable bytes > */ > String newStringNoRepl(byte[] bytes, Charset cs) throws CharacterCodingException; > > ???????? > include an @throws for CharacterCodingException? > > >> On Jun 26, 2018, at 1:41 PM, Joe Wang > > wrote: >> >> >> >> On 6/26/18, 6:54 AM, Alan Bateman wrote: >>> On 26/06/2018 05:50, Joe Wang wrote: >>>> Hi Alan, Sherman, >>>> >>>> Here's a version where we, as Sherman suggested, throw an IAE with >>>> CCE as the cause. This approach reduces code duplication in SC, >>>> although it complicates the impl a little bit with the added >>>> parameter and the different behavior between the existing usages of >>>> the methods and the new ones. The existing code paths are kept >>>> intact so there's no compatibility issue for the existing code. >>>> >>>> This version also did not remove the try-catch in Files as Alan >>>> suggested earlier. >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev02/ >>>> >>> This version looks much better. In StringCoding, do you really need >>> throwCCE? The encode/decode methods do a replace or throw so I >>> assume one flag will do. If combined with Sherman suggestion then it >>> would be minimal changes to StringCoding. It would be nice to get >>> rid of the IAE completely but that is for another day. In Files then >>> you don't need to check if cause is null before testing its type. >> >> Yes, combined with Sherman's suggestion eliminated the need for the >> new parameter. Here's the updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> >>> >>> The update tests to check for UnmappedCharacterException and >>> MalformedInputException look good. >> >> Thanks, >> Joe >> >>> >>> -Alan > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 658 bytes Desc: not available URL: From huizhe.wang at oracle.com Tue Jun 26 19:49:13 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 26 Jun 2018 12:49:13 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> Message-ID: <5B3298B9.2020603@oracle.com> On 6/26/18, 11:57 AM, Alan Bateman wrote: > > > On 26/06/2018 18:41, Joe Wang wrote: >> >> : >> >> Yes, combined with Sherman's suggestion eliminated the need for the >> new parameter. Here's the updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ > This looks good. > > Just a minor nit but newStringNoRepl and getBytesNoReply don't need to > check if the cause is null. Removed the null check. The internal impl and test guarantees it's not null indeed: http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ Thanks, Joe > > -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Tue Jun 26 19:55:22 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 26 Jun 2018 12:55:22 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B3298B9.2020603@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> Message-ID: <5B329A2A.1070305@oracle.com> +1 On 6/26/18, 12:49 PM, Joe Wang wrote: > > > On 6/26/18, 11:57 AM, Alan Bateman wrote: >> >> >> On 26/06/2018 18:41, Joe Wang wrote: >>> >>> : >>> >>> Yes, combined with Sherman's suggestion eliminated the need for the >>> new parameter. Here's the updated webrev: >>> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev03/ >> This looks good. >> >> Just a minor nit but newStringNoRepl and getBytesNoReply don't need >> to check if the cause is null. > > Removed the null check. The internal impl and test guarantees it's not > null indeed: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ > > Thanks, > Joe > >> >> -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Jun 27 07:44:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jun 2018 08:44:58 +0100 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B3298B9.2020603@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> Message-ID: <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> On 26/06/2018 20:49, Joe Wang wrote: > : > > Removed the null check. The internal impl and test guarantees it's not > null indeed: > http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ Looks good. -Alan From Roger.Riggs at Oracle.com Wed Jun 27 14:18:55 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jun 2018 10:18:55 -0400 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive In-Reply-To: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> References: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> Message-ID: <1415e8fc-511a-1424-1d9d-442e01416a58@Oracle.com> Hi Brian, Looks fine, Roger On 6/26/2018 2:59 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8202252 > http://cr.openjdk.java.net/~bpb/8202252/webrev.00/ > > Set instance variables {connect,read,write}Handler to null to allow the respective objects to be GCed. > > Without applying the source portion of the patch, the test fails on Linux, macOS, and Solaris. > > Thanks, > > Brian From brian.burkhalter at oracle.com Wed Jun 27 14:24:04 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 27 Jun 2018 07:24:04 -0700 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive In-Reply-To: <1415e8fc-511a-1424-1d9d-442e01416a58@Oracle.com> References: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> <1415e8fc-511a-1424-1d9d-442e01416a58@Oracle.com> Message-ID: <8D41DB10-EBE1-4BB9-B94A-90B944903A33@oracle.com> Thanks, Roger Brian On Jun 27, 2018, at 7:18 AM, Roger Riggs wrote: > Hi Brian, > > Looks fine, > > Roger > > > On 6/26/2018 2:59 PM, Brian Burkhalter wrote: >> https://bugs.openjdk.java.net/browse/JDK-8202252 >> http://cr.openjdk.java.net/~bpb/8202252/webrev.00/ >> >> Set instance variables {connect,read,write}Handler to null to allow the respective objects to be GCed. >> >> Without applying the source portion of the patch, the test fails on Linux, macOS, and Solaris. >> >> Thanks, >> >> Brian > From Alan.Bateman at oracle.com Wed Jun 27 15:25:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jun 2018 16:25:21 +0100 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive In-Reply-To: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> References: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> Message-ID: <95001e70-61dc-d005-b92b-f5f97d3d9e2f@oracle.com> On 26/06/2018 19:59, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8202252 > http://cr.openjdk.java.net/~bpb/8202252/webrev.00/ > > Set instance variables {connect,read,write}Handler to null to allow the respective objects to be GCed. > > Without applying the source portion of the patch, the test fails on Linux, macOS, and Solaris. > The updates to finishRead and finishWrite look okay. The change to set connectHandler to null needs more checking, I can't immediately say if this if right. Is this part of a series of patches? I assumed the Windows code would be changed, also the AsynchronousServerSocketChannel implementations. The test does this: System.gc(); assertNull(ref.get()); The more usual check is something like this: while (ref.get() != null) { Thread.sleep(20); } so that the test waits until the reference is cleared. Is there a reason why the thread uses a custom channel group? I assume it can use the default channel group, would reduce the code a bit if you want. -Alan From brian.burkhalter at oracle.com Wed Jun 27 16:27:31 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 27 Jun 2018 09:27:31 -0700 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive In-Reply-To: <95001e70-61dc-d005-b92b-f5f97d3d9e2f@oracle.com> References: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> <95001e70-61dc-d005-b92b-f5f97d3d9e2f@oracle.com> Message-ID: <27C5B3CC-D74B-4312-B59C-009AC45F4395@oracle.com> On Jun 27, 2018, at 8:25 AM, Alan Bateman wrote: > The updates to finishRead and finishWrite look okay. The change to set connectHandler to null needs more checking, I can't immediately say if this if right. Is this part of a series of patches? No. > I assumed the Windows code would be changed, also the AsynchronousServerSocketChannel implementations. The Windows code does not have this problem: it does not cache handler that I can see. The test passes on Windows without the source update. > The test does this: > > System.gc(); > assertNull(ref.get()); > > The more usual check is something like this: > > while (ref.get() != null) { Thread.sleep(20); } > > so that the test waits until the reference is cleared. OK, will update. > Is there a reason why the thread uses a custom channel group? I assume it can use the default channel group, would reduce the code a bit if you want. There is for example this check attemptRead = invokeDirect || !port.isFixedThreadPool(); and if attemptRead is true then a direct read is attempted without the handler. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From huizhe.wang at oracle.com Wed Jun 27 16:34:14 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 27 Jun 2018 09:34:14 -0700 Subject: RFR(JDK11/NIO) 8205058 throw CharacterCodingException --> Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> <5B29CADD.7090200@oracle.com> <5B311CC2.2060005@oracle.com> <5B316FB6.7020308@oracle.com> <5B31C605.5070001@oracle.com> <5B327ACF.3080804@oracle.com> <9522704a-d23f-bb43-9fb9-360222330a2a@oracle.com> <5B3298B9.2020603@oracle.com> <858f7537-2e0c-7c38-9c54-06014793540a@oracle.com> Message-ID: <5B33BC86.4000708@oracle.com> Thanks all! Glad to be able to get this done right before the deadline :-) -Joe On 6/27/18, 12:44 AM, Alan Bateman wrote: > On 26/06/2018 20:49, Joe Wang wrote: >> : >> >> Removed the null check. The internal impl and test guarantees it's >> not null indeed: >> http://cr.openjdk.java.net/~joehw/jdk11/8205058/webrev04/ > Looks good. > > -Alan From brian.burkhalter at oracle.com Wed Jun 27 18:12:57 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 27 Jun 2018 11:12:57 -0700 Subject: 8202252: (aio) Closed AsynchronousSocketChannel keeps completion handler alive In-Reply-To: <95001e70-61dc-d005-b92b-f5f97d3d9e2f@oracle.com> References: <1B6BE9A5-C7F6-481C-8BC2-8881073E2583@oracle.com> <95001e70-61dc-d005-b92b-f5f97d3d9e2f@oracle.com> Message-ID: On Jun 27, 2018, at 8:25 AM, Alan Bateman wrote: > The change to set connectHandler to null needs more checking, I can't immediately say if this if right. The only lines in the (new) code where connectHandler is used are: implConnect() 340: set connectHandler instance variable from parameter passed in finishConnect() 275: assign a local variable ?handler? from connectHandler instance variable 276: set connectHandler to null > The test does this: > > System.gc(); > assertNull(ref.get()); > > The more usual check is something like this: > > while (ref.get() != null) { Thread.sleep(20); } > > so that the test waits until the reference is cleared. Updated: http://cr.openjdk.java.net/~bpb/8202252/webrev.01/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: