From david.lloyd at redhat.com Tue Apr 29 16:42:53 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 29 Apr 2008 18:42:53 -0500 Subject: Scope of OpenJDK NIO Message-ID: <4817B27D.2040206@redhat.com> So exactly what is the scope of this list/project? Is it simply to implement JSR-203 within OpenJDK, or is it generalized, ongoing NIO development within OpenJDK? - DML From Alan.Bateman at Sun.COM Tue Apr 29 18:10:13 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 02:10:13 +0100 Subject: Welcome and introductions Message-ID: <4817C6F5.5000809@sun.com> Welcome to the nio-dev mailing list. This mailing list is setup so that it can be used for all communication related to the development of the next phase of the New I/O APIs. The folks from Sun that you will likely see chatter from are: $myself - specification, implementation, and unit tests. Iris Clark (Core Libraries group). Iris will be contributing the implementation of the new buffer classes. Rajendra Gutupalli (Quality group). Rajendra has been testing the file system API and has also co-developed a sample file system provider that treats the contents of zip and JAR file as file systems. Umashankar Ningaiah (Quality group). Umashankar has been testing the file change notification API and the asynchronous I/O API. Shantaram Pande (Quality group). Shantaram mostly works in the networking area but has been helping out by testing the new multicast support and the updates to the socket-channel APIs. Arkadiy Sutchilin (Conformance group). Arkadiy is working on the conformance tests that will eventually become part of the TCK for Java SE 7. In time I hope we can attract some non-Sun contributers. The general plan is that the implementation will develop and bake here until it is ready to go into jdk7. There are a few areas (such as the updates to the socket channel APIs) that are relatively mature and we hope to push these into jdk7 soon to make it easier to continue bug fixing and other work on the existing NIO implementation. That's it for now. -Alan. From Alan.Bateman at Sun.COM Tue Apr 29 18:45:59 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 02:45:59 +0100 Subject: Scope of OpenJDK NIO In-Reply-To: <4817B27D.2040206@redhat.com> References: <4817B27D.2040206@redhat.com> Message-ID: <4817CF57.9050805@sun.com> David M. Lloyd wrote: > So exactly what is the scope of this list/project? Is it simply to > implement JSR-203 within OpenJDK, or is it generalized, ongoing NIO > development within OpenJDK? > The project is primarily to develop the implementation of the next phase of New I/O APIs. There will be ongoing bug fixing and development of the existing NIO implementation on core-libs-dev. When non-trivial API updates arise then it may work best to pull them in here. Do you have something specific in mind? -Alan. From dmlloyd at flurg.com Tue Apr 29 16:17:17 2008 From: dmlloyd at flurg.com (David M. Lloyd) Date: Tue, 29 Apr 2008 18:17:17 -0500 Subject: Scope of OpenJDK NIO Message-ID: <4817AC7D.4050007@flurg.com> So exactly what is the scope of this list/project? Is it simply to implement JSR-203 within OpenJDK, or is it generalized, ongoing NIO development within OpenJDK? - DML From david.lloyd at redhat.com Wed Apr 30 07:23:53 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 30 Apr 2008 09:23:53 -0500 Subject: Scope of OpenJDK NIO In-Reply-To: <4817CF57.9050805@sun.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> Message-ID: <481880F9.4030704@redhat.com> On 04/29/2008 08:45 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> So exactly what is the scope of this list/project? Is it simply to >> implement JSR-203 within OpenJDK, or is it generalized, ongoing NIO >> development within OpenJDK? >> > The project is primarily to develop the implementation of the next phase > of New I/O APIs. There will be ongoing bug fixing and development of the > existing NIO implementation on core-libs-dev. When non-trivial API > updates arise then it may work best to pull them in here. Do you have > something specific in mind? Well, just the usual thoughts that come up whenever I deal with NIO: - DatagramChannels don't have a way to recover the destination address of an incoming datagram (iow, how do we know whether we just received a broadcast packet?) - unconnected DatagramChannels don't support scatter/gather There's a few other things that come up now and then, but I don't recall them at the moment. But perhaps now when they do come up, I shall forward them here. :-) - DML From Alan.Bateman at Sun.COM Wed Apr 30 10:14:18 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 18:14:18 +0100 Subject: AsynchronousFileChannel#lock() method throws NullPointerException In-Reply-To: <481821F9.1080403@sun.com> References: <481821F9.1080403@sun.com> Message-ID: <4818A8EA.2090006@sun.com> Umashankar wrote: > Hi Alan, > I noticed that on Solaris and Linux, > AsynchronousFileChannel.lock() method throws NullPointerException, if > the channel is closed asynchronously > while the lock is still being acquired. Thanks Umashankar; you are correct. The bug is specific to platforms where the "portable" implementation of AsynchronousFileChannel is used. It's been on my list to fix for some time but I neglected to list it in the known bugs list. FWIW, it's the same issue that is 6429910 and a few others. The failure mode varies and the reason it's been around for so long is because it hasn't been feasible to completely fix it on Windows. The AsynchronousFileChannel on Windows is a different implementation and so doesn't run into this issue. I'll put time into this after JavaOne as we need the asynchronous close to be very reliable. -Alan. From david.lloyd at redhat.com Wed Apr 30 10:45:47 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 30 Apr 2008 12:45:47 -0500 Subject: Scope of OpenJDK NIO In-Reply-To: <481880F9.4030704@redhat.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> Message-ID: <4818B04B.6070305@redhat.com> On 04/30/2008 09:23 AM, David M. Lloyd wrote: > On 04/29/2008 08:45 PM, Alan Bateman wrote: >> David M. Lloyd wrote: >>> So exactly what is the scope of this list/project? Is it simply to >>> implement JSR-203 within OpenJDK, or is it generalized, ongoing NIO >>> development within OpenJDK? >>> >> The project is primarily to develop the implementation of the next >> phase of New I/O APIs. There will be ongoing bug fixing and >> development of the existing NIO implementation on core-libs-dev. When >> non-trivial API updates arise then it may work best to pull them in >> here. Do you have something specific in mind? > > Well, just the usual thoughts that come up whenever I deal with NIO: > > - DatagramChannels don't have a way to recover the destination address > of an incoming datagram (iow, how do we know whether we just received a > broadcast packet?) > - unconnected DatagramChannels don't support scatter/gather Here's more: - Selector doesn't implement Closable, despite have a "void close() throws IOException" method - Other classes, like ZipFile, also have this same deficiency - DML From Alan.Bateman at Sun.COM Wed Apr 30 11:39:32 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 19:39:32 +0100 Subject: Scope of OpenJDK NIO In-Reply-To: <481880F9.4030704@redhat.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> Message-ID: <4818BCE4.8030104@sun.com> David M. Lloyd wrote: > : > Well, just the usual thoughts that come up whenever I deal with NIO: > > - DatagramChannels don't have a way to recover the destination address > of an incoming datagram (iow, how do we know whether we just received > a broadcast packet?) This one has a long history, mostly because it wasn't feasible to implement everywhere. It's a bit of niche requirement but the one example that does come up every so often is the DHCP that needs to know the interface that a datagram is received on. Michael McMahon (network group) put work into this last year with a view to adding support in jdk7. Here's a relatively recent mail from him on this topic: http://mail.openjdk.java.net/pipermail/net-dev/2008-February/000140.html So do you use broadcast datagrams in some application? Just curious, as perhaps the new multicast support will be interesting. > - unconnected DatagramChannels don't support scatter/gather This is a no-brainer that we should do. I'm not aware that anyone has ever asked for this but I will create an RFE for this today (I'll send the bugID). -Alan. From Alan.Bateman at Sun.COM Wed Apr 30 13:20:57 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 21:20:57 +0100 Subject: Scope of OpenJDK NIO In-Reply-To: <4818B04B.6070305@redhat.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> <4818B04B.6070305@redhat.com> Message-ID: <4818D4A9.9000200@sun.com> David M. Lloyd wrote: > : > Here's more: > > - Selector doesn't implement Closable, despite have a "void close() > throws IOException" method > - Other classes, like ZipFile, also have this same deficiency I don't think we have an RFE for Selector to implement Closeable but there is one for ZipFile: http://bugs.sun.com/view_bug.do?bug_id=6389768 As it has a low priority and doesn't have any votes then maybe you could send a patch to core-libs-dev to try to make it happen? -Alan. From david.lloyd at redhat.com Wed Apr 30 14:02:34 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 30 Apr 2008 16:02:34 -0500 Subject: Scope of OpenJDK NIO In-Reply-To: <4818D4A9.9000200@sun.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> <4818B04B.6070305@redhat.com> <4818D4A9.9000200@sun.com> Message-ID: <4818DE6A.2030008@redhat.com> On 04/30/2008 03:20 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> : >> Here's more: >> >> - Selector doesn't implement Closable, despite have a "void close() >> throws IOException" method >> - Other classes, like ZipFile, also have this same deficiency > I don't think we have an RFE for Selector to implement Closeable but > there is one for ZipFile: > http://bugs.sun.com/view_bug.do?bug_id=6389768 > As it has a low priority and doesn't have any votes then maybe you could > send a patch to core-libs-dev to try to make it happen? Sounds good. However, since these classes are part of the JDK, doesn't there need to be a JSR to make it happen? Just sending a patch and getting it applied seems too easy to be true. :-) - DML From david.lloyd at redhat.com Wed Apr 30 14:24:54 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 30 Apr 2008 16:24:54 -0500 Subject: Scope of OpenJDK NIO In-Reply-To: <4818BCE4.8030104@sun.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> <4818BCE4.8030104@sun.com> Message-ID: <4818E3A6.1010408@redhat.com> On 04/30/2008 01:39 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> : >> Well, just the usual thoughts that come up whenever I deal with NIO: >> >> - DatagramChannels don't have a way to recover the destination address >> of an incoming datagram (iow, how do we know whether we just received >> a broadcast packet?) > This one has a long history, mostly because it wasn't feasible to > implement everywhere. It's a bit of niche requirement but the one > example that does come up every so often is the DHCP that needs to know > the interface that a datagram is received on. Michael McMahon (network > group) put work into this last year with a view to adding support in > jdk7. Here's a relatively recent mail from him on this topic: > http://mail.openjdk.java.net/pipermail/net-dev/2008-February/000140.html > > So do you use broadcast datagrams in some application? Just curious, as > perhaps the new multicast support will be interesting. Not in my current work - I needed it for some project years ago though. I don't recall the specifics but I think I solved it by using JNI. The new multicast support is definitely interesting. >> - unconnected DatagramChannels don't support scatter/gather > This is a no-brainer that we should do. I'm not aware that anyone has > ever asked for this but I will create an RFE for this today (I'll send > the bugID). Excellent. - DML From Alan.Bateman at Sun.COM Wed Apr 30 15:12:20 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 23:12:20 +0100 Subject: Scope of OpenJDK NIO In-Reply-To: <4818DE6A.2030008@redhat.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> <4818B04B.6070305@redhat.com> <4818D4A9.9000200@sun.com> <4818DE6A.2030008@redhat.com> Message-ID: <4818EEC4.2080104@sun.com> David M. Lloyd wrote: > : > Sounds good. However, since these classes are part of the JDK, > doesn't there need to be a JSR to make it happen? Just sending a > patch and getting it applied seems too easy to be true. :-) Sending mailing to core-libs-dev is a good start. There is process associated with API changes so starting with something small and non-controversial is a good way to learn. Not all the processes are externalized yet but they will be in time. As regards JSRs - a major release has an umbrella JSR that covers all the component JSRs and other API changes in that release. If you download the JSR-270 spec for example, you will see that it specifies the release contents and lists the API updates that were done as part of the ongoing maintenance. -Alan. From Alan.Bateman at Sun.COM Wed Apr 30 15:41:12 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 30 Apr 2008 23:41:12 +0100 Subject: Cancellation test failures In-Reply-To: <4816D3FB.4020109@sun.com> References: <4816D3FB.4020109@sun.com> Message-ID: <4818F588.8010208@sun.com> Arkadiy Sutchilin wrote: > Hi Alan, > > I understand that you are extremelly busy now, so I do not expect > quick reply. > > My AsynchronousServerSocketChannel.accept() test suite should include > a couple of tests for IoFuture.cancel(boolean) method - for both > "graceful" and "forceful" cancellation. > > The problems are the following: > > The IoFuture spec says: > > (1) "This method otherwise behaves exactly as specified by the Future > interface." > > The Future spec, in turn, says: > > (2) "After this method returns, subsequent calls to isDone() will > always return true." > > (3) "Subsequent calls to isCancelled() will always return true if > this method returned true." > > ---------- > > The test scenario: > -- initial state: AsychronousServerSocketChannel is open and bound to > a local port; > -- invoke accept(attacment, handler); > -- ensure that accept has returned IoFuture in pending state; > -- invoke IoFuture.cancel(boolean) > -- analyze returned value and post-conditions. > > ---------- > > A "graceful" cancellation - cancel(false) - constantly returns false. > isDone() returns false - OOPS - that contradicts (2). > isCancelled() returns false - OKAY. > Trying to connect - OKAY, all the channels are open and functional, > completion handler has been invoked. > > Graceful cancellation is not supported, cancel(false) is ignored and > isDone() == false seems reasonable, while contradics (2) > > ---------- > > A "forceful" cancellation - cancel(true) - constantly returns true. > isDone() returns true - OKAY > isCancelled() returns false - OOPS - that contradicts (3) > The channel is closed - that could be expected. > > Forceful cancellation actually closes the channel and isCancelled() == > false seems reasonable, while contradicts (3) > > ---------- > > IMHO, the implementation's behavior seems reasonable and consistent, > while "inherited" Future assertions look inappropriate. That would be > great to document at least the following: > > Override (2): > IoFuture.cancel(???) == false && IoFuture.isDone() == false > combination is permissible and indicates, that cancel(???) had no > effect and left the operation in pending stste; > > Override (3): > in case of forceful cancellation > IoFuture.cancel(true) == true && IoFuture.isCancelled() == false > combination is permissible and indicates, that cancel(true) has closed > the channel. > > I have not tested other asynchronous operations yet, so my proposals > may appear too narrow and inconsistent with a big picture. Please feel > free to ignore these ones and look for more appropriate solution. Great mail and I'm happy you have caught this. The simplest approach is for IoFuture#cancel to specify how isDone and isCancelled should behave for the case that cancel(false) is a no-op, and cancel(true) closes the channel. That would intentionally violate Future's contract for cancel but since we are deviating anyway it may not be too bad. I'll need to get input from Doug and Mike (co-conspirators for this area) to get opinions. Will get back to you soon. -Alan. From Alan.Bateman at Sun.COM Wed Apr 30 19:03:46 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 01 May 2008 03:03:46 +0100 Subject: Scope of OpenJDK NIO In-Reply-To: <4818BCE4.8030104@sun.com> References: <4817B27D.2040206@redhat.com> <4817CF57.9050805@sun.com> <481880F9.4030704@redhat.com> <4818BCE4.8030104@sun.com> Message-ID: <48192502.5000105@sun.com> Alan Bateman wrote: > : >> - unconnected DatagramChannels don't support scatter/gather > This is a no-brainer that we should do. I'm not aware that anyone has > ever asked for this but I will create an RFE for this today (I'll send > the bugID). This is now in the bug database as 6696470. -Alan.