From dingxmin at linux.vnet.ibm.com Sun Feb 3 23:57:22 2013 From: dingxmin at linux.vnet.ibm.com (Frank Ding) Date: Mon, 04 Feb 2013 15:57:22 +0800 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <51097A70.2040104@oracle.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> Message-ID: <510F69E2.7020905@linux.vnet.ibm.com> Hi Alan, Thanks for your comments on the patch and the test case. I did more investigation on the synchronization policy in Windows selector. Currently, closeLock is used in 3 methods of WindowsSelectorImpl, which are implClose, putEventOps, implRegister. There are several vars guarded by closeLock in these 3 methods such as pollWrapper, channelArray, totalChannels, fdMap and some others. Methods implClose, implRegister and implDereg are called by SelectorImpl which are all guarded by publicKeys. The contention arises in this case between putEventOps and implDereg. putEventOps is called by SelectionKey.interestOps where only closeLock is acquired and var pollWrapper is affected. As a conclusion, I incline to keep the sync block as it is. As of the test case, I did change some by absorbing your comments: 1. Class name and test case folder 2. No System.exit 3. No try/catch at the end 4. Better termination in main thread 5. Selector and SocketChannels are closed. Please review the revised test case again http://cr.openjdk.java.net/~dingxmin/6429204/webrev.01 Thanks && Best regards, Frank On 1/31/2013 3:54 AM, Alan Bateman wrote: > On 30/01/2013 07:21, Frank Ding wrote: >> Hi guys, >> There is a bug in nio selector on Windows which is reported by bug >> 4863822 (closed now) and 6429204 (open). >> >> The issue happens on Windows, where two threads, one doing a >> channel registration/deregistration with a selector and the other >> setting interestOps on one of the channels registered to the selector. >> The thread doing select() goes eventually to >> WindowsSelectorImpl.implDereg() that rearranges SelectionKeyImpls in >> array channelArray and then adjusts entries in pollWrapper. Since >> implDereg method does not synchronize rearrangement of channelArray >> and accessing pollWrapper, the other thread calling interestOps has a >> possibility of updating ops on the already deleted fd in the poll array. >> >> The problem can be manifested by the test case in the webrev review >> [1]. The test loops a large number of times in thread 1 trying to >> change interest set for a SelectionKey. The other thread(main >> thread) should be able to sense the change and notify thread 1 within >> a short period of time. I admit that the test is not deterministic, >> which means it has a chance to pass with current JDK. I think it >> would be better having a bootstrapped class to reproduce the problem >> in a stable way but don't have any idea whether jtreg can achieve this. >> Solution to the problem is quite straightforward. See patch in >> [1]. Simply extending closeLock sync scope from implRegister, >> putEventOps, implClose to implDereg. I have concern on the scope of >> lock closeLock. In the patch I provided, I put >> synchronized(closeLock) within if statement. This resolves the issue >> and also minimizes the potential performance degradation. However, in >> class WindowsSelectorImpl, closeLock should protect other variables >> such as totalChannels, fdMap, keys and selectedKeys. So would it >> make more sense to increase scope to before deregister(ski) call? >> >> Thanks in advance for any comments and review offered. >> >> [1] http://cr.openjdk.java.net/~dingxmin/6429204/webrev.00/ > Yes, there is a problem here. Most of the original issue was fixed by > the changes in 5025260 but implDereg was left to 6429204. > > It needs a bit of study (to remember how the synchronization is > specified and implemented in this Selector) but I think the > synchronization block does need to go further to include down to (and > including) the fdMap.remove. > > The test will be a bit of clean-up. Tests for Selector go into > test/java/nio/channels/Selector. A possible name for the test is > RacyDeregister (we've avoiding putting bug IDs into the test names > here). jtreg tests aren't supposed to use System.exit so I think it > needs a better way to terminate. Also it the try/catch to throw > RuntimeException at the end isn't needed. Otherwise I think it just > needs clean-up and to make sure that the Selector and SocketChannels > are closed. > > -Alan. From Alan.Bateman at oracle.com Tue Feb 5 07:33:29 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 05 Feb 2013 15:33:29 +0000 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <510F69E2.7020905@linux.vnet.ibm.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> <510F69E2.7020905@linux.vnet.ibm.com> Message-ID: <51112649.4060101@oracle.com> On 04/02/2013 07:57, Frank Ding wrote: > Hi Alan, > Thanks for your comments on the patch and the test case. > I did more investigation on the synchronization policy in Windows > selector. > Currently, closeLock is used in 3 methods of WindowsSelectorImpl, > which are implClose, putEventOps, implRegister. There are several > vars guarded by closeLock in these 3 methods such as pollWrapper, > channelArray, totalChannels, fdMap and some others. Methods > implClose, implRegister and implDereg are called by SelectorImpl which > are all guarded by publicKeys. The contention arises in this case > between putEventOps and implDereg. putEventOps is called by > SelectionKey.interestOps where only closeLock is acquired and var > pollWrapper is affected. As a conclusion, I incline to keep the sync > block as it is. I think there is still an issue, albeit remote. Consider an asynchronous interstOps at around the time that a key is canceled and a thread is doing a select and processing the canceled key set. It's possible that the thread that called interestOps has already checked that the key is valid so we have implDereg and putEventsOps running concurrently. In that scenario it is possible for selection key's index to be set to -1 at around the time that the poll wrapper is updated at that index. So I think the synchronized block need to be extended to ski.setIndex(-1) and I think that putEventOps needs to be updated to check the index, ie: int index = ski.getIndex(); if (index == -1) throw new CancelledKeyException(); I don't think this scenario arises with your test but hopefully you agree that this scenario is still possible. On the test - thanks for the cleanups and moving it, it looks much better now. -Alan. From dingxmin at linux.vnet.ibm.com Wed Feb 6 22:42:43 2013 From: dingxmin at linux.vnet.ibm.com (Frank Ding) Date: Thu, 07 Feb 2013 14:42:43 +0800 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <51112649.4060101@oracle.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> <510F69E2.7020905@linux.vnet.ibm.com> <51112649.4060101@oracle.com> Message-ID: <51134CE3.5090203@linux.vnet.ibm.com> Hi Alan, Thank you for your careful review. I agree the scenario could happen. Would it be better if ski.setIndex(-1) in implDereg method and sk.getIndex() in putEventOps method are guarded by closeLock from JMM visibility point of perspective? The patch is available in the 02 patch below. http://cr.openjdk.java.net/~dingxmin/6429204/webrev.02/ Best regards, Frank On 2/5/2013 11:33 PM, Alan Bateman wrote: > On 04/02/2013 07:57, Frank Ding wrote: >> Hi Alan, >> Thanks for your comments on the patch and the test case. >> I did more investigation on the synchronization policy in Windows >> selector. >> Currently, closeLock is used in 3 methods of WindowsSelectorImpl, >> which are implClose, putEventOps, implRegister. There are several >> vars guarded by closeLock in these 3 methods such as pollWrapper, >> channelArray, totalChannels, fdMap and some others. Methods >> implClose, implRegister and implDereg are called by SelectorImpl >> which are all guarded by publicKeys. The contention arises in this >> case between putEventOps and implDereg. putEventOps is called by >> SelectionKey.interestOps where only closeLock is acquired and var >> pollWrapper is affected. As a conclusion, I incline to keep the sync >> block as it is. > I think there is still an issue, albeit remote. > > Consider an asynchronous interstOps at around the time that a key is > canceled and a thread is doing a select and processing the canceled > key set. It's possible that the thread that called interestOps has > already checked that the key is valid so we have implDereg and > putEventsOps running concurrently. In that scenario it is possible for > selection key's index to be set to -1 at around the time that the poll > wrapper is updated at that index. > > So I think the synchronized block need to be extended to > ski.setIndex(-1) and I think that putEventOps needs to be updated to > check the index, ie: > > int index = ski.getIndex(); > if (index == -1) > throw new CancelledKeyException(); > > I don't think this scenario arises with your test but hopefully you > agree that this scenario is still possible. > > On the test - thanks for the cleanups and moving it, it looks much > better now. > > -Alan. > From henry.jen at oracle.com Thu Feb 7 16:15:53 2013 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 07 Feb 2013 16:15:53 -0800 Subject: Early review on NIO Stream APIs Message-ID: <511443B9.2060505@oracle.com> Hi, While lambda still finalizing the final API for Stream, there are a couple of streamifaction APIs in nio area, which I would like to start circling around to get early feedbacks. Those APIs return a CloseableStream, other than that, there is not much directly connection to Stream definition. The specdiff and webrev is available at http://cr.openjdk.java.net/~henryjen/ccc/8006884.0/ Please include me in the reply as I don't necessary receiving emails from all aliases. Cheers, Henry From zhong.j.yu at gmail.com Thu Feb 7 18:53:04 2013 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 7 Feb 2013 20:53:04 -0600 Subject: Early review on NIO Stream APIs In-Reply-To: <511443B9.2060505@oracle.com> References: <511443B9.2060505@oracle.com> Message-ID: The find() method uses a BiPredicate matcher what about using the existing java.nio.file.PathMatcher interface? There are already some implementations of PathMatcher both in JDK and in the wild. PathMatcher doesn't accept the file attributes though, but that's a problem for most Path-based APIs - path alone often isn't enough and app has to look up attributes in a separate step. Zhong Yu On Thu, Feb 7, 2013 at 6:15 PM, Henry Jen wrote: > Hi, > > While lambda still finalizing the final API for Stream, there are a > couple of streamifaction APIs in nio area, which I would like to start > circling around to get early feedbacks. > > Those APIs return a CloseableStream, other than that, there is not much > directly connection to Stream definition. > > The specdiff and webrev is available at > > http://cr.openjdk.java.net/~henryjen/ccc/8006884.0/ > > Please include me in the reply as I don't necessary receiving emails > from all aliases. > > Cheers, > Henry From henry.jen at oracle.com Thu Feb 7 20:45:52 2013 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 7 Feb 2013 20:45:52 -0800 Subject: Early review on NIO Stream APIs In-Reply-To: References: <511443B9.2060505@oracle.com> Message-ID: <6BA0AD92-429C-4AC5-909D-57D53326ABA8@oracle.com> BiPredicate is a intentional choice, as during the walk process, the attribute is read and many times, filtering is based on file attributes. PathMatcher is basically a Predicate, developer can easily use that in a filter with walk, which return a Stream. Hope that helps. Cheers, Henry On Feb 7, 2013, at 6:53 PM, Zhong Yu wrote: > The find() method uses a > > BiPredicate matcher > > what about using the existing java.nio.file.PathMatcher interface? > There are already some implementations of PathMatcher both in JDK and > in the wild. > > PathMatcher doesn't accept the file attributes though, but that's a > problem for most Path-based APIs - path alone often isn't enough and > app has to look up attributes in a separate step. > > Zhong Yu > > On Thu, Feb 7, 2013 at 6:15 PM, Henry Jen wrote: >> Hi, >> >> While lambda still finalizing the final API for Stream, there are a >> couple of streamifaction APIs in nio area, which I would like to start >> circling around to get early feedbacks. >> >> Those APIs return a CloseableStream, other than that, there is not much >> directly connection to Stream definition. >> >> The specdiff and webrev is available at >> >> http://cr.openjdk.java.net/~henryjen/ccc/8006884.0/ >> >> Please include me in the reply as I don't necessary receiving emails >> from all aliases. >> >> Cheers, >> Henry From zhong.j.yu at gmail.com Thu Feb 7 21:10:30 2013 From: zhong.j.yu at gmail.com (Zhong Yu) Date: Thu, 7 Feb 2013 23:10:30 -0600 Subject: Early review on NIO Stream APIs In-Reply-To: <6BA0AD92-429C-4AC5-909D-57D53326ABA8@oracle.com> References: <511443B9.2060505@oracle.com> <6BA0AD92-429C-4AC5-909D-57D53326ABA8@oracle.com> Message-ID: On Thu, Feb 7, 2013 at 10:45 PM, Henry Jen wrote: > BiPredicate is a intentional choice, as during the walk process, the attribute is read and many times, filtering is based on file attributes. > > PathMatcher is basically a Predicate, developer can easily use that in a filter with walk, which return a Stream. I see. It's a pity that we don't have an abstraction for Path+Attributes, since often applications need both. > Hope that helps. > > Cheers, > Henry > > On Feb 7, 2013, at 6:53 PM, Zhong Yu wrote: > >> The find() method uses a >> >> BiPredicate matcher >> >> what about using the existing java.nio.file.PathMatcher interface? >> There are already some implementations of PathMatcher both in JDK and >> in the wild. >> >> PathMatcher doesn't accept the file attributes though, but that's a >> problem for most Path-based APIs - path alone often isn't enough and >> app has to look up attributes in a separate step. >> >> Zhong Yu >> >> On Thu, Feb 7, 2013 at 6:15 PM, Henry Jen wrote: >>> Hi, >>> >>> While lambda still finalizing the final API for Stream, there are a >>> couple of streamifaction APIs in nio area, which I would like to start >>> circling around to get early feedbacks. >>> >>> Those APIs return a CloseableStream, other than that, there is not much >>> directly connection to Stream definition. >>> >>> The specdiff and webrev is available at >>> >>> http://cr.openjdk.java.net/~henryjen/ccc/8006884.0/ >>> >>> Please include me in the reply as I don't necessary receiving emails >>> from all aliases. >>> >>> Cheers, >>> Henry > From Ulf.Zibis at CoSoCo.de Fri Feb 8 04:09:24 2013 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 08 Feb 2013 13:09:24 +0100 Subject: Early review on NIO Stream APIs In-Reply-To: <511443B9.2060505@oracle.com> References: <511443B9.2060505@oracle.com> Message-ID: <5114EAF4.8090308@CoSoCo.de> Am 08.02.2013 01:15, schrieb Henry Jen: > The specdiff and webrev is available at > > http://cr.openjdk.java.net/~henryjen/ccc/8006884.0/ > > Please include me in the reply as I don't necessary receiving emails > from all aliases. In Files.lines(...): After returned from this method, if an I/O error occurs reading ... should be: After returned from this method, if an I/O error occurs while reading ... In Files.walk(...): ... a given staring file... should be: ... a given starting file... Do those streams close automatically from finalize() method, if released to GC, e.g. by nulling the referencing variable? ...and if not, why? -Ulf From henry.jen at oracle.com Fri Feb 8 09:25:32 2013 From: henry.jen at oracle.com (Henry Jen) Date: Fri, 08 Feb 2013 09:25:32 -0800 Subject: Early review on NIO Stream APIs In-Reply-To: <5114EAF4.8090308@CoSoCo.de> References: <511443B9.2060505@oracle.com> <5114EAF4.8090308@CoSoCo.de> Message-ID: <5115350C.4050004@oracle.com> On 02/08/2013 04:09 AM, Ulf Zibis wrote: > Am 08.02.2013 01:15, schrieb Henry Jen: > In Files.lines(...): > After returned from this method, if an I/O error occurs reading ... > should be: > After returned from this method, if an I/O error occurs while reading ... > > In Files.walk(...): > ... a given staring file... > should be: > ... a given starting file... > Fixed, thanks. > Do those streams close automatically from finalize() method, if released > to GC, e.g. by nulling the referencing variable? ...and if not, why? > The stream doesn't do that, it's transparent to whatever the underlying Closeable resource's behavior. Cheers, Henry From Ulf.Zibis at CoSoCo.de Fri Feb 8 09:50:46 2013 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Fri, 08 Feb 2013 18:50:46 +0100 Subject: Early review on NIO Stream APIs In-Reply-To: <5115350C.4050004@oracle.com> References: <511443B9.2060505@oracle.com> <5114EAF4.8090308@CoSoCo.de> <5115350C.4050004@oracle.com> Message-ID: <51153AF6.9090305@CoSoCo.de> Am 08.02.2013 18:25, schrieb Henry Jen: > On 02/08/2013 04:09 AM, Ulf Zibis wrote: >> Do those streams close automatically from finalize() method, if released >> to GC, e.g. by nulling the referencing variable? ...and if not, why? > The stream doesn't do that, it's transparent to whatever the underlying > Closeable resource's behavior. :-( -Ulf From Alan.Bateman at oracle.com Fri Feb 8 11:40:23 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 08 Feb 2013 19:40:23 +0000 Subject: Early review on NIO Stream APIs In-Reply-To: <5114EAF4.8090308@CoSoCo.de> References: <511443B9.2060505@oracle.com> <5114EAF4.8090308@CoSoCo.de> Message-ID: <511554A7.901@oracle.com> On 08/02/2013 12:09, Ulf Zibis wrote: > : > > Do those streams close automatically from finalize() method, if > released to GC, e.g. by nulling the referencing variable? ...and if > not, why? > > -Ulf > We don't use finalizers in this area. Aside from the overhead, the other issue here is that file descriptors are a relatively scarce resource so if the application is not closing files or channels then you'll often run out of file descriptors long before you run out of memory. In any case, I think try-with-resources should be very useful here. -Alan. From Alan.Bateman at oracle.com Mon Feb 11 07:24:00 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Feb 2013 15:24:00 +0000 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <51134CE3.5090203@linux.vnet.ibm.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> <510F69E2.7020905@linux.vnet.ibm.com> <51112649.4060101@oracle.com> <51134CE3.5090203@linux.vnet.ibm.com> Message-ID: <51190D10.80402@oracle.com> On 07/02/2013 06:42, Frank Ding wrote: > Hi Alan, > Thank you for your careful review. I agree the scenario could happen. > Would it be better if ski.setIndex(-1) in implDereg method and > sk.getIndex() in putEventOps method are guarded by closeLock from JMM > visibility point of perspective? The patch is available in the 02 > patch below. > > http://cr.openjdk.java.net/~dingxmin/6429204/webrev.02/ > > Best regards, > Frank This is much better so thumbs up from me. -Alan. From dingxmin at linux.vnet.ibm.com Fri Feb 15 19:21:09 2013 From: dingxmin at linux.vnet.ibm.com (Frank Ding) Date: Sat, 16 Feb 2013 11:21:09 +0800 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <51190D10.80402@oracle.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> <510F69E2.7020905@linux.vnet.ibm.com> <51112649.4060101@oracle.com> <51134CE3.5090203@linux.vnet.ibm.com> <51190D10.80402@oracle.com> Message-ID: <511EFB25.3000205@linux.vnet.ibm.com> Hi Alan, Thank you. Shall I have my colleague push the change? Best regards, Frank On 2/11/2013 11:24 PM, Alan Bateman wrote: > On 07/02/2013 06:42, Frank Ding wrote: >> Hi Alan, >> Thank you for your careful review. I agree the scenario could happen. >> Would it be better if ski.setIndex(-1) in implDereg method and >> sk.getIndex() in putEventOps method are guarded by closeLock from JMM >> visibility point of perspective? The patch is available in the 02 >> patch below. >> >> http://cr.openjdk.java.net/~dingxmin/6429204/webrev.02/ >> >> Best regards, >> Frank > This is much better so thumbs up from me. > > -Alan. > From Alan.Bateman at oracle.com Sun Feb 17 23:31:49 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Feb 2013 07:31:49 +0000 Subject: Bug: SelectionKey.interestOps ignores interestOps on Windows In-Reply-To: <511EFB25.3000205@linux.vnet.ibm.com> References: <5108CA07.7080007@linux.vnet.ibm.com> <51097A70.2040104@oracle.com> <510F69E2.7020905@linux.vnet.ibm.com> <51112649.4060101@oracle.com> <51134CE3.5090203@linux.vnet.ibm.com> <51190D10.80402@oracle.com> <511EFB25.3000205@linux.vnet.ibm.com> Message-ID: <5121D8E5.5010509@oracle.com> On 16/02/2013 03:21, Frank Ding wrote: > Hi Alan, > Thank you. Shall I have my colleague push the change? > I forgot to ask you about that. Assume I will push it for you today. -Alan From zhouyx at linux.vnet.ibm.com Tue Feb 19 00:30:55 2013 From: zhouyx at linux.vnet.ibm.com (Sean Chou) Date: Tue, 19 Feb 2013 16:30:55 +0800 Subject: A race problem about select in a small time window In-Reply-To: <50F47455.1000307@oracle.com> References: <50D4863A.5000109@oracle.com> <50F47455.1000307@oracle.com> Message-ID: Hi Rob, Is there any progress ? On Tue, Jan 15, 2013 at 5:10 AM, Rob McKenna wrote: > Apologies folks, I managed to overlook this completely. Sean, its on my > radar and I'll get back to you soon. > > -Rob > > > On 21/12/12 15:54, Alan Bateman wrote: > >> >> I don't have cycles to look at this one (too much going on for M6) but >> Rob McKenna (cc'ed) might. >> >> On 17/12/2012 08:56, Sean Chou wrote: >> >>> Hello , >>> >>> This is the detail problem, there is a small time window in which a 3 >>> threads race makes select() always return 0 without blocking. >>> >>> I wrote a testcase(http://cr.openjdk.**java.net/~zhouyx/OJDK-714/** >>> webrev0.2/ < >>> http://cr.openjdk.java.net/%**7Ezhouyx/OJDK-714/webrev0.2/>) >>> which needs to modify the lib code to reproduce, because the time windows >>> is small. >>> >>> The reproduce scenario is described in follow, use Tx for thread x: >>> >>> 1. T1 (the user code) is selecting a channel(suppose C), it just returns >>> from native select function, and niolib select method is checking if the >>> returned channel is interested in the event, then 2 happens; >>> 2. T2 is closing channel C, it just set the open variable to false but >>> not yet closed the channel actually, and then 3 happens; >>> 3. T3 set the interedOps of the channel to 0. // 0 means the channel is >>> not interested in anything, the channel will be put into cancel list >>> normally. >>> >>> In this senario, T1 returns from select, and return 0 which means no >>> channel is selected(because the channel C returned from native invocation >>> has nothing insterested in, it is not returned to application). Then T1 >>> goes to invoke select again(usually in a loop, this is how select is >>> designed to be used). In normal case, select method checks if any channels >>> those should be cancelled and remove them from the set to be selected. >>> Then, goes to native select function. >>> >>> The problem is: select method first checks if the channel is closed, if >>> it is closed, select method doesn't put it into cancel list. >>> >>> In above senario, channel C is in close state, but not closed indeed, >>> and setInteredOps to 0(which means cancel). So select method doesn't put C >>> into cancel list(due to the problem) which means the native select set >>> still contains channel C . So the native select always return C and nio >>> select always return 0. Until the channel is finally closed. >>> >>> >>> The testcase: http://cr.openjdk.java.net/~**zhouyx/OJDK-714/webrev0.2/< >>> http://cr.openjdk.java.net/%**7Ezhouyx/OJDK-714/webrev0.2/ >>> > >>> >>> A working fix: http://cr.openjdk.java.net/~**zhouyx/OJDK-714/webrev_fix/< >>> http://cr.openjdk.java.net/%**7Ezhouyx/OJDK-714/webrev_fix/ >>> > >>> >>> >>> Please have a look. >>> >>> >>> >> > -- Best Regards, Sean Chou -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130219/c1b150a4/attachment.html From dan.xu at oracle.com Tue Feb 19 15:44:59 2013 From: dan.xu at oracle.com (Dan Xu) Date: Tue, 19 Feb 2013 15:44:59 -0800 Subject: Review Request - JDK-8006645: TEST_BUG: java/nio/file/Files/CopyAndMove.java failing intermittently (sol) Message-ID: <51240E7B.5040106@oracle.com> HiAll, CopyAndMove.java test has failed in theregression testintermittently. This is due to some potential time precision loss in file copy and move operationswhen it tries to copy the last-modified-time to the target file. The fixwill only verify the last-modified-time at time resolution of secondlevelto avoid the precision loss. I also update the java doc of Files.move method to clarify it. Thanks! webrev: http://cr.openjdk.java.net/~dxu/8006645/webrev.00/ -Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130219/1e912f6a/attachment.html From Alan.Bateman at oracle.com Tue Feb 19 22:49:38 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Feb 2013 06:49:38 +0000 Subject: Review Request - JDK-8006645: TEST_BUG: java/nio/file/Files/CopyAndMove.java failing intermittently (sol) In-Reply-To: <51240E7B.5040106@oracle.com> References: <51240E7B.5040106@oracle.com> Message-ID: <51247202.6020909@oracle.com> On 19/02/2013 23:44, Dan Xu wrote: > HiAll, > > CopyAndMove.java test has failed in theregression testintermittently. > This is due to some potential time precision loss in file copy and > move operationswhen it tries to copy the last-modified-time to the > target file. > > The fixwill only verify the last-modified-time at time resolution of > secondlevelto avoid the precision loss. I also update the java doc of > Files.move method to clarify it. Thanks! > > webrev: http://cr.openjdk.java.net/~dxu/8006645/webrev.00/ > > -Dan > > This looks good to me. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130220/772e2fb7/attachment.html From chris.hegarty at oracle.com Wed Feb 20 02:21:37 2013 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Feb 2013 10:21:37 +0000 Subject: Review Request - JDK-8006645: TEST_BUG: java/nio/file/Files/CopyAndMove.java failing intermittently (sol) In-Reply-To: <51240E7B.5040106@oracle.com> References: <51240E7B.5040106@oracle.com> Message-ID: <5124A3B1.3030207@oracle.com> Looks ok to me Dan. -Chris. On 19/02/2013 23:44, Dan Xu wrote: > HiAll, > > CopyAndMove.java test has failed in theregression testintermittently. > This is due to some potential time precision loss in file copy and move > operationswhen it tries to copy the last-modified-time to the target file. > > The fixwill only verify the last-modified-time at time resolution of > secondlevelto avoid the precision loss. I also update the java doc of > Files.move method to clarify it. Thanks! > > webrev: http://cr.openjdk.java.net/~dxu/8006645/webrev.00/ > > -Dan > > From akhil.arora at oracle.com Fri Feb 22 11:07:10 2013 From: akhil.arora at oracle.com (Akhil Arora) Date: Fri, 22 Feb 2013 11:07:10 -0800 Subject: access error with FileChannel.open(dir) on windows Message-ID: <5127C1DE.6040101@oracle.com> The following simple program throws access denied on windows7 for any directory path passed as the first arg. Works fine on linux, and works fine for args that are file. Verified that the exception occurs on a coworker's windows7 as well. Is this a bug or am I doing something wrong? Thanks $java open . Exception in thread "main" java.nio.file.AccessDeniedException: . at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:114) at java.nio.channels.FileChannel.open(FileChannel.java:287) at open.main(open.java:12) import java.io.IOException; import java.nio.channels.FileChannel; import java.nio.file.Paths; import java.nio.file.StandardOpenOption; import java.nio.file.OpenOption; import java.util.EnumSet; import java.util.Set; public class open { public static void main(String[] args) throws IOException { Set options = EnumSet.noneOf(StandardOpenOption.class); FileChannel fc = FileChannel.open(Paths.get(args[0]), options); } } From Alan.Bateman at oracle.com Fri Feb 22 11:19:40 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Feb 2013 19:19:40 +0000 Subject: access error with FileChannel.open(dir) on windows In-Reply-To: <5127C1DE.6040101@oracle.com> References: <5127C1DE.6040101@oracle.com> Message-ID: <5127C4CC.20409@oracle.com> On 22/02/2013 19:07, Akhil Arora wrote: > The following simple program throws access denied on windows7 for any > directory path passed as the first arg. Works fine on linux, and works > fine for args that are file. Verified that the exception occurs on a > coworker's windows7 as well. Is this a bug or am I doing something > wrong? Thanks Whether you can open a directory as a regular file is highly platform specific. In this case it's just the error coming from Windows. BTW: If you are just opening a FileChannel for reading then FileChannel.open(path) should do it. -Alan. From rob.mckenna at oracle.com Mon Feb 25 14:18:51 2013 From: rob.mckenna at oracle.com (Rob McKenna) Date: Mon, 25 Feb 2013 22:18:51 +0000 Subject: A race problem about select in a small time window In-Reply-To: References: <50D4863A.5000109@oracle.com> <50F47455.1000307@oracle.com> Message-ID: <512BE34B.2090105@oracle.com> Apologies for the delay Sean, Just to be clear: it appears we're neglecting to add the channel to the EPollArrayWrapper idle set as we're depending on ch.isOpen() to determine whether we need to or not. It makes sense to me that anything with u.events == 0 should indeed be put into the idle list regardless of whether it "isOpen". It will always be removed later by release, and both release and updateRegistrations are synced on updateList. I'll try to get ahold of Alan tomorrow just to run it past him (in any case, you'll need him to review this as I'm not a reviewer) and to figure out if I'm missing something with the ch.isOpen(). -Rob On 19/02/13 08:30, Sean Chou wrote: > Hi Rob, > > Is there any progress ? > > > On Tue, Jan 15, 2013 at 5:10 AM, Rob McKenna > wrote: > > Apologies folks, I managed to overlook this completely. Sean, its > on my radar and I'll get back to you soon. > > -Rob > > > On 21/12/12 15:54, Alan Bateman wrote: > > > I don't have cycles to look at this one (too much going on for > M6) but Rob McKenna (cc'ed) might. > > On 17/12/2012 08:56, Sean Chou wrote: > > Hello , > > This is the detail problem, there is a small time window > in which a 3 threads race makes select() always return 0 > without blocking. > > I wrote a > testcase(http://cr.openjdk.java.net/~zhouyx/OJDK-714/webrev0.2/ > > ) which > needs to modify the lib code to reproduce, because the > time windows is small. > > The reproduce scenario is described in follow, use Tx for > thread x: > > 1. T1 (the user code) is selecting a channel(suppose C), > it just returns from native select function, and niolib > select method is checking if the returned channel is > interested in the event, then 2 happens; > 2. T2 is closing channel C, it just set the open variable > to false but not yet closed the channel actually, and then > 3 happens; > 3. T3 set the interedOps of the channel to 0. // 0 means > the channel is not interested in anything, the channel > will be put into cancel list normally. > > In this senario, T1 returns from select, and return 0 > which means no channel is selected(because the channel C > returned from native invocation has nothing insterested > in, it is not returned to application). Then T1 goes to > invoke select again(usually in a loop, this is how select > is designed to be used). In normal case, select method > checks if any channels those should be cancelled and > remove them from the set to be selected. Then, goes to > native select function. > > The problem is: select method first checks if the channel > is closed, if it is closed, select method doesn't put it > into cancel list. > > In above senario, channel C is in close state, but not > closed indeed, and setInteredOps to 0(which means cancel). > So select method doesn't put C into cancel list(due to the > problem) which means the native select set still contains > channel C . So the native select always return C and nio > select always return 0. Until the channel is finally closed. > > > The testcase: > http://cr.openjdk.java.net/~zhouyx/OJDK-714/webrev0.2/ > > > > A working fix: > http://cr.openjdk.java.net/~zhouyx/OJDK-714/webrev_fix/ > > > > Please have a look. > > > > > > > > -- > Best Regards, > Sean Chou -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20130225/2fb88ad9/attachment.html From Alan.Bateman at oracle.com Thu Feb 28 06:31:38 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 28 Feb 2013 14:31:38 +0000 Subject: Request for review: 7190897 (fs) Files.isWritable method returns false when the path is writable (win). In-Reply-To: <512F4574.5090909@oracle.com> References: <512F4574.5090909@oracle.com> Message-ID: <512F6A4A.3080300@oracle.com> On 28/02/2013 11:54, Alexey Utkin wrote: > : > > The suggested fix: > http://cr.openjdk.java.net/~uta/openjdk-webrevs/JDK-7190897/webrev.00/ Thanks for sorting this out, this code should have used AccessCheck in the original implementation. A couple of comments: Did you consider having AccessCheckForFile take a parameter for the mapping so that it wouldn't be file-specific (and so could be renamed to AccessCheck)? checkFileAccess ignores the exception from AccessCheck whereas I should it should be translated to an IOException. Otherwise I think this is good. You don't have a test case but I can't think how this could be tested anyway as we already have tests for checkAccess and isWritable. -Alan. From Alan.Bateman at oracle.com Thu Feb 28 07:41:16 2013 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 28 Feb 2013 15:41:16 +0000 Subject: Request for review: 7190897 (fs) Files.isWritable method returns false when the path is writable (win). In-Reply-To: <512F7500.8010106@oracle.com> References: <512F4574.5090909@oracle.com> <512F6A4A.3080300@oracle.com> <512F7500.8010106@oracle.com> Message-ID: <512F7A9C.4030905@oracle.com> On 28/02/2013 15:17, Alexey Utkin wrote: > That is not single, but 4 additional parameters ( FILE_GENERIC_READ, > FILE_GENERIC_WRITE, FILE_GENERIC_EXECUTE, FILE_ALL_ACCESS) - that are > relatively complicate masks. That parameters have to be changed > consistently to avoid the problem (there is the analogy with > orthogonal basis in geometry If you understand what I mean). Now we > use the [AccessCheckForFile] just in [nio] package. We can extend the > implementation any time we need it. Okay, I can live with this but would be nice to get it to AccessCheck at most point. >> checkFileAccess ignores the exception from AccessCheck whereas I >> should it should be translated to an IOException. > That is by design. Any problem with the [checkFileAccess] need to be > converted to the [false] return value. At the end point - in the > [WindowsFileSystemProvider.checkAccess] function - the [false] return > value would be converted to the [AccessDeniedException] exception - > that is desired code flow. My point is that AccessCheck can fail for other reasons too and it would be good to get these reason into the exception so that it is not lost. It might have to AccessDeniedException if there aren't specific errors documented but at least the reason will be in the exception message to help someone figure it the issue. So I think it would be better to translate the exception rather than returning a boolean. >> >> Otherwise I think this is good. You don't have a test case but I >> can't think how this could be tested anyway as we already have tests >> for checkAccess and isWritable. > I have the test. It is attached to the bug as Netbeans project, but it > need manual security setup in security tab of the [demofile.txt] file > (as shown in attached screenshot). By changing the "Write" check box > on the [demofile.txt] file security dialog, test result have varying > accordingly. > Seems the web bug-db interface is not synchronized yet. Thanks, I guessed that an automated test would not be possible. -Alan. From alexey.utkin at oracle.com Thu Feb 28 07:20:33 2013 From: alexey.utkin at oracle.com (Alexey Utkin) Date: Thu, 28 Feb 2013 15:20:33 -0000 Subject: Request for review: 7190897 (fs) Files.isWritable method returns false when the path is writable (win). In-Reply-To: <512F6A4A.3080300@oracle.com> References: <512F4574.5090909@oracle.com> <512F6A4A.3080300@oracle.com> Message-ID: <512F7500.8010106@oracle.com> On 28.02.2013 18:31, Alan Bateman wrote: > On 28/02/2013 11:54, Alexey Utkin wrote: >> : >> >> The suggested fix: >> http://cr.openjdk.java.net/~uta/openjdk-webrevs/JDK-7190897/webrev.00/ > Thanks for sorting this out, this code should have used AccessCheck in > the original implementation. A couple of comments: > > Did you consider having AccessCheckForFile take a parameter for the > mapping so that it wouldn't be file-specific (and so could be renamed > to AccessCheck)? That is not single, but 4 additional parameters ( FILE_GENERIC_READ, FILE_GENERIC_WRITE, FILE_GENERIC_EXECUTE, FILE_ALL_ACCESS) - that are relatively complicate masks. That parameters have to be changed consistently to avoid the problem (there is the analogy with orthogonal basis in geometry If you understand what I mean). Now we use the [AccessCheckForFile] just in [nio] package. We can extend the implementation any time we need it. > checkFileAccess ignores the exception from AccessCheck whereas I > should it should be translated to an IOException. That is by design. Any problem with the [checkFileAccess] need to be converted to the [false] return value. At the end point - in the [WindowsFileSystemProvider.checkAccess] function - the [false] return value would be converted to the [AccessDeniedException] exception - that is desired code flow. > > Otherwise I think this is good. You don't have a test case but I can't > think how this could be tested anyway as we already have tests for > checkAccess and isWritable. I have the test. It is attached to the bug as Netbeans project, but it need manual security setup in security tab of the [demofile.txt] file (as shown in attached screenshot). By changing the "Write" check box on the [demofile.txt] file security dialog, test result have varying accordingly. Seems the web bug-db interface is not synchronized yet. > -Alan. From michael.hixson at gmail.com Wed Feb 13 03:24:34 2013 From: michael.hixson at gmail.com (Michael Hixson) Date: Wed, 13 Feb 2013 03:24:34 -0000 Subject: Paths.get("").normalize() throws AIOOBE on linux Message-ID: Hello, I notice that on Linux, the following program throws an exception: import java.nio.file.Paths; public class HelloWorld { public static void main(String[] args) { Paths.get("").normalize(); } } Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0 at sun.nio.fs.UnixPath.normalize(UnixPath.java:508) at HelloWorld.main(HelloWorld.java:4) Meanwhile on Windows, no exception is thrown. The following code prints "true": Path a = Paths.get(""); Path b = a.normalize(); System.out.println(a.equals(b)); Is this difference in behavior expected? I'm using 1.7u13 in both places. -Michael