From daniel.latremoliere at gmail.com Sun Jul 1 08:31:59 2012 From: daniel.latremoliere at gmail.com (=?ISO-8859-1?Q?Daniel_Latr=E9moli=E8re?=) Date: Sun, 01 Jul 2012 17:31:59 +0200 Subject: Flushing ZipFS Message-ID: <4FF06D6F.1000608@gmail.com> When using JDK7 ZipFS provider, I have seen that the Zip file is not updated even many hours after writing new files in Zip (closing OutputStream of new entries, but not closing Zip instance of FileSystem). When closing application (and particularly Zip FileSystem), the new entries are added to the zip file. It seems ZipFS provider has a cache of files and that seem logical, but I don't see a method for flushing this cache (in a valid Zip file would be better). I would have expected to see a method flush() in java.nio.file.FileSystem, but it is not the case. Is there a clean solution (excepted closing and reopening regularly the FileSystem)? Thanks. From xueming.shen at oracle.com Sun Jul 1 10:44:00 2012 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 01 Jul 2012 10:44:00 -0700 Subject: Flushing ZipFS In-Reply-To: <4FF06D6F.1000608@gmail.com> References: <4FF06D6F.1000608@gmail.com> Message-ID: <4FF08C60.8020001@oracle.com> It was debated whether or not the sync() method of the ZipFileSystem should be exposed. Currently the sync() is only invoked when close() is being invoked with the assumption that zipfs will only be used via the java.nio.file API, which currently does not have a flush/sync method (flush/sync is more an implementation details of a file system). We might need to re-address this issue if zipFs is being used by its own in JDK8 (there are couple more issues need to be addressed as well). -Sherman On 7/1/2012 8:31 AM, Daniel Latr?moli?re wrote: > When using JDK7 ZipFS provider, I have seen that the Zip file is not > updated even many hours after writing new files in Zip (closing > OutputStream of new entries, but not closing Zip instance of > FileSystem). When closing application (and particularly Zip > FileSystem), the new entries are added to the zip file. > > It seems ZipFS provider has a cache of files and that seem logical, > but I don't see a method for flushing this cache (in a valid Zip file > would be better). I would have expected to see a method flush() in > java.nio.file.FileSystem, but it is not the case. Is there a clean > solution (excepted closing and reopening regularly the FileSystem)? > > Thanks. From daniel.latremoliere at gmail.com Sun Jul 1 13:05:40 2012 From: daniel.latremoliere at gmail.com (=?ISO-8859-1?Q?Daniel_Latr=E9moli=E8re?=) Date: Sun, 01 Jul 2012 22:05:40 +0200 Subject: Flushing ZipFS In-Reply-To: <4FF08C60.8020001@oracle.com> References: <4FF06D6F.1000608@gmail.com> <4FF08C60.8020001@oracle.com> Message-ID: <4FF0AD94.5010401@gmail.com> Le 01/07/2012 19:44, Xueming Shen a ?crit : > It was debated whether or not the sync() method of the ZipFileSystem > should be exposed. Currently > the sync() is only invoked when close() is being invoked with the > assumption that zipfs will only be > used via the java.nio.file API, which currently does not have a > flush/sync method (flush/sync is more > an implementation details of a file system). We might need to > re-address this issue if zipFs is being > used by its own in JDK8 (there are couple more issues need to be > addressed as well). > > -Sherman I think a sync()-like method is not strictly an implementation details. For my usage, I see this method as a feature for creating something like a savepoint in a ZIP file system. Clearly, this does not need exposure of sync() method, but probably exposure of a savepoint() method calling itself sync() method and possibly some others, like close() call sync(). But globally, I think now that I am asking for a bigger problem: methods for supporting disk coherency in Java NIO.2 FileSystem (journaling filesystem), which will probably be a real but much more complex question (and without fast response). Thank you for your detailed response [1]. Daniel. [1]: and better hack than closing and reopening: use JSR292 or Reflection, with setAccessible(true) to invoke private method sync() of ZipFileSystem. O:-) From ammar.ahmad.awan at gmail.com Tue Jul 3 23:44:52 2012 From: ammar.ahmad.awan at gmail.com (Ammar Ahmad Awan) Date: Wed, 4 Jul 2012 15:44:52 +0900 Subject: Java NIO on PVFS or Lustre Message-ID: Dear All, I am working on a research project and I want to evaluate Java NIO's performance on different file systems. As far as I know, Java NIO directly maps to the native file system calls but looking at the source code of NIO or Java I/O classes, I cannot find out where and how is the implementation checking platform specific optimizations. For example, there is a directory specific to Windows, Linux or Mac but I am not sure what will be the behavior of Java I/O operations when the underlying file system is not Windows or Linux or Solaris but it is Lustre or PVFS2 or any other parallel file system. Looking forward to hearing from the developers on this list. Kind Regards -- Ammar ---------------------------------------------------------------------- Masters Student Dept. Of Computer Engineering, Kyung Hee University ---------------------------------------------------------------------- From mik3hall at gmail.com Wed Jul 4 06:18:15 2012 From: mik3hall at gmail.com (Michael Hall) Date: Wed, 4 Jul 2012 08:18:15 -0500 Subject: sun.nio.fs.AbstractWatchKey In-Reply-To: References: Message-ID: <4DAB9A11-F9A7-4BBA-BFE0-10FB2AA1BF06@gmail.com> Nevermind. On Jul 4, 2012, at 8:10 AM, Michael Hall wrote: > I am looking at this class. It implements WatchKey which means it should have something for isValid(). It doesn't seem to? > > From chris.hegarty at oracle.com Sat Jul 7 00:13:19 2012 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sat, 7 Jul 2012 08:13:19 +0100 Subject: Request for review: 7179305: (fs) Method name sun.nio.fs.UnixPath.getPathForExecptionMessage is misspelled In-Reply-To: <4FF6E394.4000606@oracle.com> References: <4FF6E394.4000606@oracle.com> Message-ID: On 6 Jul 2012, at 14:09, Rob McKenna wrote: > http://cr.openjdk.java.net/~robm/7179305/webrev.01/ Looks fine Rob. -Chris. > > Thanks, > > -Rob From mik3hall at gmail.com Mon Jul 9 13:57:09 2012 From: mik3hall at gmail.com (Michael Hall) Date: Mon, 9 Jul 2012 15:57:09 -0500 Subject: ENTRY_MODIFY Message-ID: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> Would this mean? write/append i/o maybe attributes changed implementation defined From Alan.Bateman at oracle.com Wed Jul 11 02:07:04 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2012 10:07:04 +0100 Subject: ENTRY_MODIFY In-Reply-To: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> Message-ID: <4FFD4238.40209@oracle.com> On 09/07/2012 21:57, Michael Hall wrote: > Would this mean? > write/append i/o > maybe attributes changed > implementation defined > It means the file content but the means by which this is detected is very implementation specific. For example one could poll the file modification time to detect that something has changed or you could get events from the platform to indicate that it has changed. A change to a file attribute may also trigger an ENTRY_MODIFY event, again this is implementation specific. -Alan From Alan.Bateman at oracle.com Wed Jul 11 02:19:16 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2012 10:19:16 +0100 Subject: Java NIO on PVFS or Lustre In-Reply-To: References: Message-ID: <4FFD4514.40506@oracle.com> On 04/07/2012 07:44, Ammar Ahmad Awan wrote: > Dear All, > > I am working on a research project and I want to evaluate Java NIO's > performance on different file systems. As far as I know, Java NIO > directly maps to the native file system calls but looking at the > source code of NIO or Java I/O classes, I cannot find out where and > how is the implementation checking platform specific optimizations. > For example, there is a directory specific to Windows, Linux or Mac > but I am not sure what will be the behavior of Java I/O operations > when the underlying file system is not Windows or Linux or Solaris but > it is Lustre or PVFS2 or any other parallel file system. > The implementation is just using syscalls or library functions to access the file system, it doesn't do anything special for Lustre or other file systems. So it may be that your tests turn out to be tests for the file system implementation/driver rather than for the JDK. If you are publishing benchmarks or results then please send a link as I'm sure folks here would be interested. -Alan From zhong.j.yu at gmail.com Wed Jul 11 11:05:56 2012 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Wed, 11 Jul 2012 13:05:56 -0500 Subject: ENTRY_MODIFY In-Reply-To: <4FFD4238.40209@oracle.com> References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> Message-ID: One odd thing on windows: if a file is created/deleted under a dir, a ENTRY_MODIFY event on the dir is raised. Writing a cross platform file monitoring app is a challenge. On Wed, Jul 11, 2012 at 4:07 AM, Alan Bateman wrote: > On 09/07/2012 21:57, Michael Hall wrote: >> >> Would this mean? >> write/append i/o >> maybe attributes changed >> implementation defined >> > It means the file content but the means by which this is detected is very > implementation specific. For example one could poll the file modification > time to detect that something has changed or you could get events from the > platform to indicate that it has changed. A change to a file attribute may > also trigger an ENTRY_MODIFY event, again this is implementation specific. > > -Alan From Alan.Bateman at oracle.com Wed Jul 11 11:09:52 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2012 19:09:52 +0100 Subject: ENTRY_MODIFY In-Reply-To: References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> Message-ID: <4FFDC170.6010705@oracle.com> On 11/07/2012 19:05, Zhong Yu wrote: > One odd thing on windows: if a file is created/deleted under a dir, a > ENTRY_MODIFY event on the dir is raised. > > Writing a cross platform file monitoring app is a challenge. > Presumably you also get the CREATE and DELETE events too. -Alan From zhong.j.yu at gmail.com Wed Jul 11 11:17:33 2012 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Wed, 11 Jul 2012 13:17:33 -0500 Subject: ENTRY_MODIFY In-Reply-To: <4FFDC170.6010705@oracle.com> References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> <4FFDC170.6010705@oracle.com> Message-ID: On Wed, Jul 11, 2012 at 1:09 PM, Alan Bateman wrote: > On 11/07/2012 19:05, Zhong Yu wrote: >> >> One odd thing on windows: if a file is created/deleted under a dir, a >> ENTRY_MODIFY event on the dir is raised. >> >> Writing a cross platform file monitoring app is a challenge. >> > Presumably you also get the CREATE and DELETE events too. Yes. And one odd thing on linux: edit a file with gedit, then save, a ENTRY_CREATE event on the file is raised. Since the API is very platform dependent, I wasn't even sure if these are bugs, or tolerable behaviors. From Alan.Bateman at oracle.com Wed Jul 11 11:45:03 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2012 19:45:03 +0100 Subject: ENTRY_MODIFY In-Reply-To: References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> <4FFDC170.6010705@oracle.com> Message-ID: <4FFDC9AF.3060004@oracle.com> On 11/07/2012 19:17, Zhong Yu wrote: > : > > And one odd thing on linux: edit a file with gedit, then save, a > ENTRY_CREATE event on the file is raised. > > Since the API is very platform dependent, I wasn't even sure if these > are bugs, or tolerable behaviors. Presumably gedit opens a new file when saving. -Alan. From mik3hall at gmail.com Wed Jul 11 11:57:00 2012 From: mik3hall at gmail.com (Michael Hall) Date: Wed, 11 Jul 2012 13:57:00 -0500 Subject: ENTRY_MODIFY In-Reply-To: <4FFD4238.40209@oracle.com> References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> Message-ID: <6E903D0F-C079-4190-9AE4-09B1062EE984@gmail.com> On Jul 11, 2012, at 4:07 AM, Alan Bateman wrote: > A change to a file attribute may also trigger an ENTRY_MODIFY event, This was mainly what I was wondering about. If the flag was intended to only mean data change or data and/or attribute change. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120711/adff2027/attachment.html From zhong.j.yu at gmail.com Wed Jul 11 12:00:54 2012 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Wed, 11 Jul 2012 14:00:54 -0500 Subject: ENTRY_MODIFY In-Reply-To: <4FFDC9AF.3060004@oracle.com> References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> <4FFDC170.6010705@oracle.com> <4FFDC9AF.3060004@oracle.com> Message-ID: On Wed, Jul 11, 2012 at 1:45 PM, Alan Bateman wrote: > On 11/07/2012 19:17, Zhong Yu wrote: >> >> : >> >> And one odd thing on linux: edit a file with gedit, then save, a >> ENTRY_CREATE event on the file is raised. >> >> Since the API is very platform dependent, I wasn't even sure if these >> are bugs, or tolerable behaviors. > > Presumably gedit opens a new file when saving. but there's no ENTRY_DELETE prior to that. From Alan.Bateman at oracle.com Thu Jul 12 04:41:03 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 12 Jul 2012 12:41:03 +0100 Subject: ENTRY_MODIFY In-Reply-To: References: <1084F5E6-3BC2-41BF-AACA-016DAF71B58C@gmail.com> <4FFD4238.40209@oracle.com> <4FFDC170.6010705@oracle.com> <4FFDC9AF.3060004@oracle.com> Message-ID: <4FFEB7CF.3080203@oracle.com> On 11/07/2012 20:00, Zhong Yu wrote: > On Wed, Jul 11, 2012 at 1:45 PM, Alan Bateman wrote: >> On 11/07/2012 19:17, Zhong Yu wrote: >>> : >>> >>> And one odd thing on linux: edit a file with gedit, then save, a >>> ENTRY_CREATE event on the file is raised. >>> >>> Since the API is very platform dependent, I wasn't even sure if these >>> are bugs, or tolerable behaviors. >> Presumably gedit opens a new file when saving. > but there's no ENTRY_DELETE prior to that. You may need to use strace to see what gedit does, it may be creating a backup file (I've seen other editors do that). -Alan. From chris.hegarty at oracle.com Sun Jul 15 14:52:44 2012 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 15 Jul 2012 22:52:44 +0100 Subject: Code Review Request: 7183800: TEST_BUG: Update tests to run on Ubuntu 12.04 (localhost is 127.0.1.1) In-Reply-To: <500311E9.4070809@oracle.com> References: <500311E9.4070809@oracle.com> Message-ID: <50033BAC.1020205@oracle.com> Kurchi, I don't have any problem with the source changes ( they make sense ), but I don't see why they are necessary. The listening socket is bound to the wildcard address, so sending to the local address should work. I guess this is some kind of (default configuration error/issue). Anyway, your test changes are fine. Adding the third test to the problem list makes sense ( as you say, it will require a more involved fix ), but you should create a new separate bug number for it and add it to the comment in the ProblemList. It may be a little confusing to see this bug number in the comment when the bug database will show it as fixed. -Chris. On 15/07/12 19:54, Kurchi Subhra Hazra wrote: > Hi, > > Some tests in our test repository assume that InetAddress.getLocalHost() > will return 127.0.0.1. However, with > linux systems running Ubuntu 12.04 now returning 127.0.1.1 as localhost, > these tests are failing. I have changed two > of the NIO tests to remove their dependency on > InetAddress.getLocalHost(). A third test has been added to > the ProblemList.txt since the changes required for it are more involved. > > Bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7183800 > Webrev: http://cr.openjdk.java.net/~khazra/7183800/webrev.00/ > > > Thanks, > Kurchi > From Alan.Bateman at oracle.com Mon Jul 16 03:56:39 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jul 2012 11:56:39 +0100 Subject: Code Review Request: 7183800: TEST_BUG: Update tests to run on Ubuntu 12.04 (localhost is 127.0.1.1) In-Reply-To: <500311E9.4070809@oracle.com> References: <500311E9.4070809@oracle.com> Message-ID: <5003F367.20708@oracle.com> On 15/07/2012 19:54, Kurchi Subhra Hazra wrote: > Hi, > > Some tests in our test repository assume that > InetAddress.getLocalHost() will return 127.0.0.1. However, with > linux systems running Ubuntu 12.04 now returning 127.0.1.1 as > localhost, these tests are failing. I have changed two > of the NIO tests to remove their dependency on > InetAddress.getLocalHost(). A third test has been added to > the ProblemList.txt since the changes required for it are more involved. > > Bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7183800 > Webrev: http://cr.openjdk.java.net/~khazra/7183800/webrev.00/ > As Chris pointed out, these tests bind to the wildcard address and you need a specific address to setup the address association, otherwise it's highly platform/implementation specific as to whether it will do as intended. So I think the proposed changes will cause problems where you can't connect to 0.0.0.0 and so I think we need to look for another solution. Given that none of the TCP tests fail then it suggests it mean that something else is going on, maybe something in the Ubuntu 12 configuration. Do these tests run if IPv6 is disabled? We've had issues with some distributions where multicast tests fail and these have always come down to iptables blocking IPv6 multicast packets causing the tests to fail. I just wonder if we have something similar here. -Alan From Alan.Bateman at oracle.com Mon Jul 16 10:39:08 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jul 2012 18:39:08 +0100 Subject: 6633549 (dc) Include-mode filtering of IPv6 sources does not block datagrams on Linux Message-ID: <500451BC.7040601@oracle.com> This is one I tracked down a long time ago but didn't push the changes for some reason. On Linux we have disabled support for include-mode multicasting with IPv6 addresses (this is allowed by the spec). The original issue was because the values for MCAST_{JOIN,LEAVE}_SOURCE_GROUP were incorrect, an issue that only exists when compiling on older systems that don't have the definitions in the include files. The fix is simply to get rid of the definitions and change joinOrDrop6 so that it now supports include-mode filtering. I've also updated one of the tests to assert that UOE cannot be thrown on Linux. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/6633549/webrev/ -Alan From kurchi.subhra.hazra at oracle.com Mon Jul 16 13:12:15 2012 From: kurchi.subhra.hazra at oracle.com (Kurchi Hazra) Date: Mon, 16 Jul 2012 13:12:15 -0700 Subject: Code Review Request: 7183800: TEST_BUG: Update tests to run on Ubuntu 12.04 (localhost is 127.0.1.1) In-Reply-To: <5003F367.20708@oracle.com> References: <500311E9.4070809@oracle.com> <5003F367.20708@oracle.com> Message-ID: <5004759F.7060200@oracle.com> I just tried running these tests with -Djava.net.preferIPv4Stack set to true, but they still fail. I remember working on some of these failures in networking where ipv6 multicast was buggy, but absolutely nothing made those tests work. However, in this case, if we make sure that the address that the sender is sending the packets to and the address that the listener is binding to is the same, the tests are passing. The fix in this webrev is not working for Windows since the tests try to connect to 0.0.0.0. So we have to think of some other solution. - Kurchi On 7/16/2012 3:56 AM, Alan Bateman wrote: > On 15/07/2012 19:54, Kurchi Subhra Hazra wrote: >> Hi, >> >> Some tests in our test repository assume that >> InetAddress.getLocalHost() will return 127.0.0.1. However, with >> linux systems running Ubuntu 12.04 now returning 127.0.1.1 as >> localhost, these tests are failing. I have changed two >> of the NIO tests to remove their dependency on >> InetAddress.getLocalHost(). A third test has been added to >> the ProblemList.txt since the changes required for it are more involved. >> >> Bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7183800 >> Webrev: http://cr.openjdk.java.net/~khazra/7183800/webrev.00/ >> > As Chris pointed out, these tests bind to the wildcard address and you > need a specific address to setup the address association, otherwise > it's highly platform/implementation specific as to whether it will do > as intended. So I think the proposed changes will cause problems where > you can't connect to 0.0.0.0 and so I think we need to look for > another solution. > > Given that none of the TCP tests fail then it suggests it mean that > something else is going on, maybe something in the Ubuntu 12 > configuration. Do these tests run if IPv6 is disabled? We've had > issues with some distributions where multicast tests fail and these > have always come down to iptables blocking IPv6 multicast packets > causing the tests to fail. I just wonder if we have something similar > here. > > -Alan > -- -Kurchi From chris.hegarty at oracle.com Tue Jul 17 06:48:31 2012 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 17 Jul 2012 14:48:31 +0100 Subject: 6633549 (dc) Include-mode filtering of IPv6 sources does not block datagrams on Linux In-Reply-To: <500451BC.7040601@oracle.com> References: <500451BC.7040601@oracle.com> Message-ID: <50056D2F.6030309@oracle.com> Looks fine to me. -Chris. On 16/07/2012 18:39, Alan Bateman wrote: > > This is one I tracked down a long time ago but didn't push the changes > for some reason. On Linux we have disabled support for include-mode > multicasting with IPv6 addresses (this is allowed by the spec). The > original issue was because the values for > MCAST_{JOIN,LEAVE}_SOURCE_GROUP were incorrect, an issue that only > exists when compiling on older systems that don't have the definitions > in the include files. > > The fix is simply to get rid of the definitions and change joinOrDrop6 > so that it now supports include-mode filtering. I've also updated one of > the tests to assert that UOE cannot be thrown on Linux. > > The webrev with the changes is here: > > http://cr.openjdk.java.net/~alanb/6633549/webrev/ > > -Alan From rob.mckenna at oracle.com Wed Jul 18 06:36:24 2012 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 18 Jul 2012 14:36:24 +0100 Subject: review request: 7184932: Remove the temporary Selector usage in the NIO socket adapters Message-ID: <5006BBD8.3040205@oracle.com> Hi folks, Looking for a review for the above. This fix replaces temporary selectors with a poll method in SocketAdaptor. http://cr.openjdk.java.net/~robm/7184932/webrev.01/ -Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120718/a679af16/attachment.html From Alan.Bateman at oracle.com Thu Jul 19 03:27:25 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 19 Jul 2012 11:27:25 +0100 Subject: review request: 7184932: Remove the temporary Selector usage in the NIO socket adapters In-Reply-To: <5006BBD8.3040205@oracle.com> References: <5006BBD8.3040205@oracle.com> Message-ID: <5007E10D.8060801@oracle.com> On 18/07/2012 14:36, Rob McKenna wrote: > Hi folks, > > Looking for a review for the above. This fix replaces temporary > selectors with a poll method in SocketAdaptor. > > http://cr.openjdk.java.net/~robm/7184932/webrev.01/ > > > -Rob Thanks for taking this one (as background to others, the issue here is applications with hundreds of threads using the socket adapters means that there are a hundreds of temporary Selectors cached, there are also issues with applications that create short-lived threads doing timed operations using the socket adapters). One thing that doesn't seem right is the return from the Windows implementation of Java_sun_nio_ch_Net_poll. It's return the value from select where it should return the equivalent of the revents and so needs to use FD_ISSET to check the rd/wr/ex values. In the test then the main method doesn't need to catch IOException, it would be better to just let it throw it. It would also be good to make sure that the listener and all other sockets are closed even for the case that the test fails. A few other comments on the test is that AcceptorThread.listener can be final. Also, are you missing a test for ServerSocket.setSoTimeout? -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120719/aeabf938/attachment.html From mik3hall at gmail.com Wed Jul 25 16:13:54 2012 From: mik3hall at gmail.com (Michael Hall) Date: Wed, 25 Jul 2012 18:13:54 -0500 Subject: WatchService re-register events appended or replaced Message-ID: <702042BE-A874-41D8-AAC9-F1C749021462@gmail.com> This isn't clear to me from the javadoc which I thought might make it worth raising the question even if I could track down the answer myself. When you re-register a path you should get the same WatchKey back. Does it keep it's old events and get the new ones added? I assume re-register amounts to an event reset but would feel safer in this assumption with some confirmation? From Alan.Bateman at oracle.com Wed Jul 25 21:11:51 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Jul 2012 05:11:51 +0100 Subject: WatchService re-register events appended or replaced In-Reply-To: <702042BE-A874-41D8-AAC9-F1C749021462@gmail.com> References: <702042BE-A874-41D8-AAC9-F1C749021462@gmail.com> Message-ID: <5010C387.2090601@oracle.com> On 26/07/2012 00:13, Michael Hall wrote: > This isn't clear to me from the javadoc which I thought might make it worth raising the question even if I could track down the answer myself. > When you re-register a path you should get the same WatchKey back. Does it keep it's old events and get the new ones added? > I assume re-register amounts to an event reset but would feel safer in this assumption with some confirmation? > I think this is all covered in the javadoc (register method): "If the file system object identified by this object is currently registered with the watch service then the watch key, representing that registration, is returned after changing the event set or modifiers to those specified by the |events| and |modifiers| parameters. Changing the event set does not cause pending events for the object to be discarded. " So calling register on an object that is already registered does change the event set but it doesn't discard events that are waiting to be retrieved. Hopefully this helps. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120726/f2a0529a/attachment.html From mik3hall at gmail.com Thu Jul 26 03:05:23 2012 From: mik3hall at gmail.com (Michael Hall) Date: Thu, 26 Jul 2012 05:05:23 -0500 Subject: WatchService re-register events appended or replaced In-Reply-To: <5010C387.2090601@oracle.com> References: <702042BE-A874-41D8-AAC9-F1C749021462@gmail.com> <5010C387.2090601@oracle.com> Message-ID: On Jul 25, 2012, at 11:11 PM, Alan Bateman wrote: > On 26/07/2012 00:13, Michael Hall wrote: >> >> This isn't clear to me from the javadoc which I thought might make it worth raising the question even if I could track down the answer myself. >> When you re-register a path you should get the same WatchKey back. Does it keep it's old events and get the new ones added? >> I assume re-register amounts to an event reset but would feel safer in this assumption with some confirmation? >> > I think this is all covered in the javadoc (register method): > > "If the file system object identified by this object is currently registered with the watch service then the watch key, representing that registration, is returned after changing the event set or modifiers to those specified by the events and modifiers parameters. Changing the event set does not cause pending events for the object to be discarded. " > > So calling register on an object that is already registered does change the event set but it doesn't discard events that are waiting to be retrieved. Hopefully this helps. > > -Alan Alan, That does clear it up. It took a little while while to find it though. It is not mentioned in the WatchService or Path javadoc but the Watchable. It was about the only way the test/java/nio/file/WatchService/Basic testEvents() method was going to run. Which I guess makes that a good test. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120726/177550ca/attachment.html From david.lloyd at redhat.com Thu Jul 26 11:44:11 2012 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 26 Jul 2012 13:44:11 -0500 Subject: FileChannel incompatibility introduced between 6 and 7 (and possible compiler bug) Message-ID: <50118FFB.2090200@redhat.com> Ran into a pretty ugly little issue just now. My NIO framework has a FileChannel subclass that it uses for certain tasks. This works fine on 7, but on 6 I get errors like this: java.lang.NoClassDefFoundError: java/nio/channels/SeekableByteChannel I found this odd, initially, because I never reference this interface. I noticed that in 7 this interface was introduced and (sensibly) retroactively added as a superinterface of FileChannel. Logically this seems like it should not be a problem; it just means that in 7 we should have a couple new bridge methods on FileChannel (and we do): public java.nio.channels.SeekableByteChannel truncate(long) throws java.io.IOException; Code: 0: aload_0 1: lload_1 2: invokevirtual #95 // Method truncate:(J)Ljava/nio/channels/FileChannel; 5: areturn public java.nio.channels.SeekableByteChannel position(long) throws java.io.IOException; Code: 0: aload_0 1: lload_1 2: invokevirtual #94 // Method position:(J)Ljava/nio/channels/FileChannel; 5: areturn Note the invokevirtual, which is specifically relevant here. These methods are obviously necessary as FileChannel covariantly overrides the new interface methods. However, the trouble started when I compiled my subclass and noticed, to my surprise, the very same bridge methods are on it! Thus there's a static linkage from my class directly to SeekableByteChannel, despite there being no reference to it in the source code at all. Now what bugs me about this is that if my class did not have these bridge methods, it would still work just fine, since the superclass is using invokevirtual to access the covariantly overridden method! I could understand this behavior if it were using invokespecial, but it's not, or if I were actually directly implementing SeekableByteChannel on my class, which I'm not; the end result is what appears to be a needless break in compatibility. Now I do have some options available - I can make mocks of the Java 7 classes I need to compile against, and then compile against 6, or I can add a build-time step to strip these methods from the class. But this strikes me as being a somewhat serious, if highly specific, problem. Is this a compiler bug, or is it an irreparable backwards-compatibility mistake in NIO? -- - DML From Alan.Bateman at oracle.com Thu Jul 26 13:57:10 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Jul 2012 21:57:10 +0100 Subject: FileChannel incompatibility introduced between 6 and 7 (and possible compiler bug) In-Reply-To: <50118FFB.2090200@redhat.com> References: <50118FFB.2090200@redhat.com> Message-ID: <5011AF26.1060804@oracle.com> On 26/07/2012 19:44, David M. Lloyd wrote: > Ran into a pretty ugly little issue just now. My NIO framework has a > FileChannel subclass that it uses for certain tasks. This works fine > on 7, but on 6 I get errors like this: > > java.lang.NoClassDefFoundError: java/nio/channels/SeekableByteChannel > It sounds like you are compiling with 7 and running with 6, in which case you need to make sure to specify - bootclasspath when specifying -source and -target. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120726/e78574ff/attachment.html From david.lloyd at redhat.com Thu Jul 26 14:35:09 2012 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 26 Jul 2012 16:35:09 -0500 Subject: FileChannel incompatibility introduced between 6 and 7 (and possible compiler bug) In-Reply-To: <5011AF26.1060804@oracle.com> References: <50118FFB.2090200@redhat.com> <5011AF26.1060804@oracle.com> Message-ID: <5011B80D.5070602@redhat.com> On 07/26/2012 03:57 PM, Alan Bateman wrote: > On 26/07/2012 19:44, David M. Lloyd wrote: >> Ran into a pretty ugly little issue just now. My NIO framework has a >> FileChannel subclass that it uses for certain tasks. This works fine >> on 7, but on 6 I get errors like this: >> >> java.lang.NoClassDefFoundError: java/nio/channels/SeekableByteChannel >> > It sounds like you are compiling with 7 and running with 6, in which > case you need to make sure to specify - bootclasspath when specifying > -source and -target. May not be an option for me, as I need to compile against certain classes only present in 7. I'll probably have to stub them to make it work. We shall see how it goes. -- - DML From mik3hall at gmail.com Sun Jul 29 18:05:38 2012 From: mik3hall at gmail.com (Michael Hall) Date: Sun, 29 Jul 2012 20:05:38 -0500 Subject: trz update - nio.2 MacFileSystemProvider added kqueue WatchService Message-ID: I have added an implementation of a kqueue based WatchService implementation to my OS X specific nio.2 code. At this time this code will run a slightly modified version of the openjdk WatchService Basic test code. (See known_issues.txt for more info). This provides a platform specific non-polling native solution to replace the generic polling WatchService currently being provided in Java 7 for OS X. I intend to provide one more WatchService based on FSEvents for OS X. See [1[ for mention of kqueue vs. fsevents use. Example usage to run the modified WatchService Basic test with the kqueue WatchService java -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider -cp lib/macnio2.jar Basic This adds a WatchService to the currently supported Mac nio.2 file attributes. Included there are attributes for? o Finder o Launch Services o Cocoa NSFileManager o xattr These provide access to platform specific file meta information not available otherwise with java. Download - http://www195.pair.com/mik3hall/trz.zip [1] https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html#//apple_ref/doc/uid/TP40005289-CH5-SW2 From Alan.Bateman at oracle.com Sun Jul 29 20:41:31 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 30 Jul 2012 04:41:31 +0100 Subject: Code Review Request For 7185340 In-Reply-To: <5014B3B0.1010307@oracle.com> References: <5014B3B0.1010307@oracle.com> Message-ID: <5016026B.1040309@oracle.com> On 29/07/2012 04:53, Dan Xu wrote: > Hi Alan, > > Please review the fix to bug 7185340. The webrev can be found at, > http://cr.openjdk.java.net/~dxu/7185340/webrev.01/. > > The fix is to increase the -XX:MaxDirectMemorySize parameter from 64M > to 75M. Bug 7176485 allows the temporary buffer cache to grow to IOV > MAX. Therefore, in Windows each thread can have up to 16 direct > buffers in the cache, which causes the addition of this jvm parameter > to run this testcase. > > Thanks, > > -Dan Thanks Dan, this looks fine to me and looks like you've worked out the maximum amount of direct memory that this test will use. I can push this for you. -Alan From Alan.Bateman at oracle.com Sun Jul 29 20:54:26 2012 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 30 Jul 2012 04:54:26 +0100 Subject: trz update - nio.2 MacFileSystemProvider added kqueue WatchService In-Reply-To: References: Message-ID: <50160572.9080500@oracle.com> On 30/07/2012 02:05, Michael Hall wrote: > I have added an implementation of a kqueue based WatchService implementation to my OS X specific nio.2 code. > > At this time this code will run a slightly modified version of the openjdk WatchService Basic test code. (See known_issues.txt for more info). > > This provides a platform specific non-polling native solution to replace the generic polling WatchService currently being provided in Java 7 for OS X. > > I intend to provide one more WatchService based on FSEvents for OS X. See [1[ for mention of kqueue vs. fsevents use. > It looks like you have made good progress. I have not tried your file system provider yet but is the WatchService implementation something that you would be willing to contribute to OpenJDK? I don't know if you have tried to run the unit tests that in the test/java/nio/file/WatchService directory but it would be interesting to see if it passes these tests. -Alan From david.holmes at oracle.com Sun Jul 29 23:17:45 2012 From: david.holmes at oracle.com (David Holmes) Date: Mon, 30 Jul 2012 16:17:45 +1000 Subject: Code Review Request For 7185340 In-Reply-To: <50160B8D.4020904@oracle.com> References: <5014B3B0.1010307@oracle.com> <5016026B.1040309@oracle.com> <50160B8D.4020904@oracle.com> Message-ID: <50162709.4070407@oracle.com> Hi Dan, On 30/07/2012 2:20 PM, Dan Xu wrote: > Thank you, Alan. I see the change is already pushed into the repository. > I am going to move the bug status to 10-Fix Delivered. But I don't know > what values I need put into "Commit To Fix In Build", "Fixed In Build", > and "Integrated In Build". In glisten, I see the latest build under > /java/re/jdk/8/promoted/all is b49. Shall I put b49 to those fields? While Alan is likely sleeping ... :) Change the bug to "Fix Available". It is marked "Fix Delivered" after the TL repository integrates. b49 is already promoted and I would not expect TL to integrate again before the next promotion (b50), so b51 is a reasonable guesstimate for both "commit to fix in build" and "fixed in build". RE will fill in "Integrated in build" when the bug is moved to Fix Delivered after integration. It is "Integrated in build" that is the important value for tracking purposes. Cheers, David Holmes > Thanks! > > - Dan From mik3hall at gmail.com Mon Jul 30 03:26:46 2012 From: mik3hall at gmail.com (Michael Hall) Date: Mon, 30 Jul 2012 05:26:46 -0500 Subject: trz update - nio.2 MacFileSystemProvider added kqueue WatchService In-Reply-To: <50160572.9080500@oracle.com> References: <50160572.9080500@oracle.com> Message-ID: On Jul 29, 2012, at 10:54 PM, Alan Bateman wrote: > On 30/07/2012 02:05, Michael Hall wrote: >> I have added an implementation of a kqueue based WatchService implementation to my OS X specific nio.2 code. >> >> At this time this code will run a slightly modified version of the openjdk WatchService Basic test code. (See known_issues.txt for more info). >> >> This provides a platform specific non-polling native solution to replace the generic polling WatchService currently being provided in Java 7 for OS X. >> >> I intend to provide one more WatchService based on FSEvents for OS X. See [1[ for mention of kqueue vs. fsevents use. >> > It looks like you have made good progress. I have not tried your file system provider yet but is the WatchService implementation something that you would be willing to contribute to OpenJDK? I don't know if you have tried to run the unit tests that in the test/java/nio/file/WatchService directory but it would be interesting to see if it passes these tests. > > -Alan The WatchService is slightly different in it's implementation from the current java WatchServices, so I'm not sure you would want that. Of course I would be happy to see it included but having just got it to where it passes a few unit tests I'm not that confident it's ready for that. The openjdk WatchService directory has a Basic test class which it does as of just now, pass. Which is why I decided it was ready to be considered more or less a release worth telling anyone about. I was going to try some of the other unit test classes. I added the LotsOfEvents test but mine doesn't seem to get anywhere with that. No debugging yet. A little more about this is in the project known_issues.txt file. From dan.xu at oracle.com Sun Jul 29 21:20:29 2012 From: dan.xu at oracle.com (Dan Xu) Date: Mon, 30 Jul 2012 04:20:29 -0000 Subject: Code Review Request For 7185340 In-Reply-To: <5016026B.1040309@oracle.com> References: <5014B3B0.1010307@oracle.com> <5016026B.1040309@oracle.com> Message-ID: <50160B8D.4020904@oracle.com> On 07/29/2012 08:41 PM, Alan Bateman wrote: > On 29/07/2012 04:53, Dan Xu wrote: >> Hi Alan, >> >> Please review the fix to bug 7185340. The webrev can be found at, >> http://cr.openjdk.java.net/~dxu/7185340/webrev.01/. >> >> The fix is to increase the -XX:MaxDirectMemorySize parameter from 64M >> to 75M. Bug 7176485 allows the temporary buffer cache to grow to IOV >> MAX. Therefore, in Windows each thread can have up to 16 direct >> buffers in the cache, which causes the addition of this jvm parameter >> to run this testcase. >> >> Thanks, >> >> -Dan > Thanks Dan, this looks fine to me and looks like you've worked out the > maximum amount of direct memory that this test will use. I can push > this for you. > > -Alan Thank you, Alan. I see the change is already pushed into the repository. I am going to move the bug status to 10-Fix Delivered. But I don't know what values I need put into "Commit To Fix In Build", "Fixed In Build", and "Integrated In Build". In glisten, I see the latest build under /java/re/jdk/8/promoted/all is b49. Shall I put b49 to those fields? Thanks! - Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120730/ec76a1d9/attachment.html From dan.xu at oracle.com Mon Jul 30 14:22:41 2012 From: dan.xu at oracle.com (Dan Xu) Date: Mon, 30 Jul 2012 21:22:41 -0000 Subject: Code Review Request For 7185340 In-Reply-To: <50162709.4070407@oracle.com> References: <5014B3B0.1010307@oracle.com> <5016026B.1040309@oracle.com> <50160B8D.4020904@oracle.com> <50162709.4070407@oracle.com> Message-ID: <5016FB21.6070406@oracle.com> On 07/29/2012 11:17 PM, David Holmes wrote: > Hi Dan, > > On 30/07/2012 2:20 PM, Dan Xu wrote: >> Thank you, Alan. I see the change is already pushed into the repository. >> I am going to move the bug status to 10-Fix Delivered. But I don't know >> what values I need put into "Commit To Fix In Build", "Fixed In Build", >> and "Integrated In Build". In glisten, I see the latest build under >> /java/re/jdk/8/promoted/all is b49. Shall I put b49 to those fields? > > While Alan is likely sleeping ... :) > > Change the bug to "Fix Available". It is marked "Fix Delivered" after > the TL repository integrates. b49 is already promoted and I would not > expect TL to integrate again before the next promotion (b50), so b51 > is a reasonable guesstimate for both "commit to fix in build" and > "fixed in build". RE will fill in "Integrated in build" when the bug > is moved to Fix Delivered after integration. It is "Integrated in > build" that is the important value for tracking purposes. > > Cheers, > David Holmes > >> Thanks! >> >> - Dan Thank you, David. I have moved the bug to 8-Fix Available. - Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120730/a9b892f8/attachment.html