From Alan.Bateman at oracle.com Sat Oct 2 02:16:32 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 02 Oct 2010 10:16:32 +0100 Subject: zip provider Message-ID: <4CA6F870.3070407@oracle.com> One of the items recently added to the list for jdk7 [1] is the inclusion of the zip provider so that is available "out of the box" in jdk7. Sherman is working on this now. He started with the demo zip provider that we had in the nio/nio/jdk repo and has re-worked it so that it now accesses the zip file directly (using a FileChannel, not with the java.util.zip.Zip* APIs). It's also read-write, using temporary storage to manage updates until the file system is closed. Overall it should be a lot better than the original demo provider. Hopefully we should have a webrev within a few days. -Alan. [1] http://openjdk.java.net/projects/jdk7/features/ From Alan.Bateman at oracle.com Sat Oct 2 02:21:13 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 02 Oct 2010 10:21:13 +0100 Subject: 6979526: (file) java/nio/file/FileStore/Basic.java fails if the same file system is mounted more than once Message-ID: <4CA6F989.4090205@oracle.com> I need a reviewer for a small change to the implementation of FileStore on Solaris/Linux. Currently the hashCode/equals methods are based on the device ID but that isn't sufficient when the same file system is mounted more than once. This causes one of our existing tests to fail in some environments. The fix changes the implementation of hashCode/equals to use both the device ID and the mount point. The webrev is here: http://cr.openjdk.java.net/~alanb/6979526/webrev/ Thanks, Alan. From Alan.Bateman at oracle.com Sat Oct 2 02:33:33 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 02 Oct 2010 10:33:33 +0100 Subject: WatchService - Exposing more of the Inotify Event Model In-Reply-To: References: <4C7FBBF3.7020501@oracle.com> Message-ID: <4CA6FC6D.1080306@oracle.com> fm wrote: > I've modified a few of the WatchService implementation classes to > allow for the registration of additional WatchEvent.Kind instances > that represent the native events of the underlying OS platform. I > believe the changes add flexibility to the WatchService API, with > minimal disruptive code changes to the existing implementation. There > should be a negligible impact on the existing performance. I have > altered the implementation of the LinuxWatchService to support the > additional event model; however, the SolarisWatchService and > WindowsWatchService implementations are effectively unchanged and > should work the same as before. Similar modifications could be made to > these WatchService implementations to also expose their native event > models. > > I am completely open to hearing comments on if this type of added > flexibility to the WatchService API is appropriate for the JDK 7 or > better left to an ex > ternal project. > > I will gladly provide the source code files for evaluation if there is > interest. Modified files are AbstractWatchKey.java, > AbstractPoller.java, LinuxWatchService.java, and one new file > LinuxWatchEventKind.java. > > Regards > Sorry for the delay getting back to you on this. It should be fine to send the patch here but we can't accept changes unless you are a contributor [1]. As regards the changes, I'm a bit surprised that you needed to change AbstractWatchKey as it doesn't know about specific events. For AbstractPoller then we probably need to reorganize to that it allows non-standard events through to be registered. -Alan. [1] http://openjdk.java.net/contribute/ From forax at univ-mlv.fr Sat Oct 2 03:22:37 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 02 Oct 2010 12:22:37 +0200 Subject: 6979526: (file) java/nio/file/FileStore/Basic.java fails if the same file system is mounted more than once In-Reply-To: <4CA6F989.4090205@oracle.com> References: <4CA6F989.4090205@oracle.com> Message-ID: <4CA707ED.8070805@univ-mlv.fr> Le 02/10/2010 11:21, Alan Bateman a ?crit : > > I need a reviewer for a small change to the implementation of > FileStore on Solaris/Linux. Currently the hashCode/equals methods are > based on the device ID but that isn't sufficient when the same file > system is mounted more than once. This causes one of our existing > tests to fail in some environments. The fix changes the implementation > of hashCode/equals to use both the device ID and the mount point. The > webrev is here: > http://cr.openjdk.java.net/~alanb/6979526/webrev/ > > Thanks, > Alan. Removing some codes is always a win ! Looks good. R?mi From Alan.Bateman at oracle.com Sat Oct 2 05:05:39 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 02 Oct 2010 13:05:39 +0100 Subject: 6979526: (file) java/nio/file/FileStore/Basic.java fails if the same file system is mounted more than once In-Reply-To: <4CA707ED.8070805@univ-mlv.fr> References: <4CA6F989.4090205@oracle.com> <4CA707ED.8070805@univ-mlv.fr> Message-ID: <4CA72013.8050905@oracle.com> R?mi Forax wrote: > : > Removing some codes is always a win ! > > Looks good. Thanks for the quick review. -Alan. From fudmail at gmail.com Sat Oct 2 10:33:10 2010 From: fudmail at gmail.com (fm) Date: Sat, 2 Oct 2010 10:33:10 -0700 Subject: WatchService - Exposing more of the Inotify Event Model In-Reply-To: <4CA6FC6D.1080306@oracle.com> References: <4C7FBBF3.7020501@oracle.com> <4CA6FC6D.1080306@oracle.com> Message-ID: Thanks for the response. I figured I'd sign up for the contributor status if there was any interest in the modifications I've described here. > As regards the changes, I'm a bit surprised that you needed to change > AbstractWatchKey as it doesn't know about specific events. I may be misunderstanding you here but the signalEvent(...) method in AbstractWatchKey does do "==" comparisons of the WatchEvent.Kind argument it receives against the hardcoded StandardWatchEventKind types (e.g. MODIFIED, OVERFLOW, and prior redundant events). My modification to this method retains exactly the same behavior as before; it just moves some of these comparisons into a local method called mapsTo(WatchEvent.Kind kind1, WatchEvent.Kind kind2). What this does is allow the current WatchService implementations to function as they currently do. But I could then override the mapsTo method in the LinuxWatchKey implementation to handle the new event kinds I introduced in the LinuxWatchEventKind class similarly (e.g., counting redundent events). Anyway, it is probably not very useful for me to describe these changes without providing the code somehow. I have the four modified classes in a zip file that I could post (attach?). However, I get the sense the preferred way to post the code is through some type a mercurial diff. Unfortunately, I've never used mercurial and need to get up to speed on using it in order to post the diff... if that is what is desired? On Sat, Oct 2, 2010 at 2:33 AM, Alan Bateman wrote: > fm wrote: >> >> I've modified a few of the WatchService implementation classes to >> allow for the registration of additional WatchEvent.Kind instances >> that represent the native events of the underlying OS platform. I >> believe the changes add flexibility to the WatchService API, with >> minimal disruptive code changes to the existing implementation. There >> should be a negligible impact on the existing performance. I have >> altered the implementation of the LinuxWatchService to support the >> additional event model; however, the SolarisWatchService and >> WindowsWatchService implementations are effectively unchanged and >> should work the same as before. Similar modifications could be made to >> these WatchService implementations to also expose their native event >> models. >> >> I am completely open to hearing comments on if this type of added >> flexibility to the WatchService API is appropriate for the JDK 7 or >> better left to an ex >> ternal project. >> >> I will gladly provide the source code files for evaluation if there is >> interest. Modified files are AbstractWatchKey.java, >> AbstractPoller.java, LinuxWatchService.java, and one new file >> LinuxWatchEventKind.java. >> >> Regards >> > > Sorry for the delay getting back to you on this. ?It should be fine to send > the patch here but we can't accept changes unless you are a contributor [1]. > > As regards the changes, I'm a bit surprised that you needed to change > AbstractWatchKey as it doesn't know about specific events. For > AbstractPoller then we probably need to reorganize to that it allows > non-standard events through to be registered. > > -Alan. > > [1] http://openjdk.java.net/contribute/ > From fudmail at gmail.com Sun Oct 3 23:23:23 2010 From: fudmail at gmail.com (fm) Date: Sun, 3 Oct 2010 23:23:23 -0700 Subject: Deleting symbolic links via the java.nio.file.Path API Message-ID: Something seems odd to me about the fact that the two following code snippets do not behave the same for symbolic links: if (somePath.exists()) { somePath.delete(); } and somePath.deleteIfExists(); Assume somePath is a java.nio.file.Path and represents a symbolic link on the file system. If somePath points to a nonexistent file on the file system (i.e., it is a broken symbolic link), then the first snippet of code will fail to call somePath.delete(). Therefore the broken symbolic link remains on the file system. However, the second snippet of code always results in the removal of the symbolic link from the filesystem (broken link or not). The difference is that exists() in the first snippet checks the existence of the symbolic link's target file and not the symbolic link file itself; The notion of exists in the second snippet applies to the symbolic link and not its target. Anyone have thoughts about this particular behavior of the exists() method? Should the exists() method behavior be more explicit by adding a LinkOption... paramenter? This would enable the call a call like somePath.exists(LinkOption.NOFOLLOW_LINK) to check the existence of the link itself (if somePath happens to be a symbolic link) and not the link's target. Regards. From xueming.shen at oracle.com Tue Oct 5 00:36:57 2010 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Oct 2010 00:36:57 -0700 Subject: zip filesystem provider Message-ID: <4CAAD599.4050407@oracle.com> Here is the webrev for nio filesystem provider for zip {1] http://cr.openjdk.java.net/~sherman/zipfs/webrev We are targeting for b114 for the first drop, if we can address some integration logistic issues. If not, it should be in b116. Since most of the code are so "young" (2-3 weeks), it would be appreciated if you guys can help review the code and/or try it out, any feedback is welcome. And don't be surprised if you find the webrev is being updated constantly:-) Demo.java shows how to use the NIO2 filesystem APIs to access a zip file via the zip filesystem provider, with couple lines of code you can copy, rename, move, list, walk a zip file just like access a normal file system. The most fun part of the past couple weeks is to convert my test case into this Demo:-) Myself is a little surprised how easy it is to use Alan's nio fs to access a zip file. I put a copy ofzipfs.jar on the cr website for your convenience, in case you want to try it out first. Just copy the zipfs.jar into the ext directory or add it into your classpath. -Sherman [1]NIO.2 filesystem provider for zip/jar archives -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20101005/15b55d29/attachment.html From Alan.Bateman at oracle.com Tue Oct 5 04:12:14 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 05 Oct 2010 12:12:14 +0100 Subject: 6987116: (so) test/java/nio/channels/SocketChannel/VectorIO.java failed on Solaris 11 Message-ID: <4CAB080E.6040106@oracle.com> I need a reviewer for a small update to test/java/nio/channels/SocketChannel/VectorIO.java. It's an old test that does a non-blocking write without considering the case that the write may return 0 because the socket buffer is full. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/6987116/webrev/ Thanks, Alan. From Ulf.Zibis at gmx.de Tue Oct 5 04:55:15 2010 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Tue, 05 Oct 2010 13:55:15 +0200 Subject: zip filesystem provider In-Reply-To: <4CAAD599.4050407@oracle.com> References: <4CAAD599.4050407@oracle.com> Message-ID: <4CAB1223.2090703@gmx.de> Sherman, thanks for that great work. Maybe you now could acknowledge: Bug ID: 6818737 Provide NIO channel access for zip/jar entry and look forward for: Bug ID: 6818736 Provide NIO channel access for resources from class Class I'm wondering how much work had to be done in java code. I would have expected, most should be provided by libzip. -Ulf Am 05.10.2010 09:36, schrieb Xueming Shen: > Here is the webrev for nio filesystem provider for zip {1] > > http://cr.openjdk.java.net/~sherman/zipfs/webrev > > We are targeting for b114 for the first drop, if we can address > some integration logistic issues. If not, it should be in b116. > > Since most of the code are so "young" (2-3 weeks), it would be > appreciated if you guys can help review the code and/or try > it out, any feedback is welcome. And don't be surprised if you > find the webrev is being updated constantly:-) > > Demo.java shows how to use the NIO2 filesystem APIs to access > a zip file via the zip filesystem provider, with couple lines > of code you can copy, rename, move, list, walk a zip file just like > access a normal file system. The most fun part of the past couple > weeks is to convert my test case into this Demo:-) Myself is > a little surprised how easy it is to use Alan's nio fs to access > a zip file. > > I put a copy ofzipfs.jar on the cr website for your convenience, > in case you want to try it out first. Just copy the zipfs.jar into > the ext directory or add it into your classpath. > > -Sherman > > [1]NIO.2 filesystem provider for zip/jar archives -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20101005/9974a539/attachment.html From forax at univ-mlv.fr Tue Oct 5 04:56:38 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 05 Oct 2010 13:56:38 +0200 Subject: 6987116: (so) test/java/nio/channels/SocketChannel/VectorIO.java failed on Solaris 11 In-Reply-To: <4CAB080E.6040106@oracle.com> References: <4CAB080E.6040106@oracle.com> Message-ID: <4CAB1276.9090105@univ-mlv.fr> Le 05/10/2010 13:12, Alan Bateman a ?crit : > > I need a reviewer for a small update to > test/java/nio/channels/SocketChannel/VectorIO.java. It's an old test > that does a non-blocking write without considering the case that the > write may return 0 because the socket buffer is full. The webrev with > the changes is here: > http://cr.openjdk.java.net/~alanb/6987116/webrev/ > > Thanks, > Alan. Looks good. Perhaps, you should initialize the pseudo random generator with a seed to make the test repeatable. cheers, R?mi From Alan.Bateman at oracle.com Tue Oct 5 07:10:32 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 05 Oct 2010 15:10:32 +0100 Subject: WatchService - Exposing more of the Inotify Event Model In-Reply-To: References: <4C7FBBF3.7020501@oracle.com> <4CA6FC6D.1080306@oracle.com> Message-ID: <4CAB31D8.2060506@oracle.com> fm wrote: > Thanks for the response. I figured I'd sign up for the contributor > status if there was any interest in the modifications I've described > here. > I can't gauge the interest but it may be best to focus on events that may be feasible to implement on more than platform. You could define them in ExtendedWatchEventKind a bit like the extensions we have for modifiers already. > : > I may be misunderstanding you here but the signalEvent(...) method in > AbstractWatchKey does do "==" comparisons of the WatchEvent.Kind > argument it receives against the hardcoded StandardWatchEventKind > types (e.g. MODIFIED, OVERFLOW, and prior redundant events). My > modification to this method retains exactly the same behavior as > before; it just moves some of these comparisons into a local method > called mapsTo(WatchEvent.Kind kind1, WatchEvent.Kind kind2). What this > does is allow the current WatchService implementations to function as > they currently do. But I could then override the mapsTo method in the > LinuxWatchKey implementation to handle the new event kinds I > introduced in the LinuxWatchEventKind class similarly (e.g., counting > redundent events). > Right, there is filtering in AbstractWatchKey and I'm just suggesting that this (shared) class shouldn't know about non-standard events. We'll have to modify the validation as currently it rejects non-standard events. -Alan. From Alan.Bateman at oracle.com Tue Oct 5 07:16:43 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 05 Oct 2010 15:16:43 +0100 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: References: Message-ID: <4CAB334B.4000706@oracle.com> fm wrote: > Something seems odd to me about the fact that the two following code > snippets do not behave the same for symbolic links: > > if (somePath.exists()) { somePath.delete(); } > > and > > somePath.deleteIfExists(); > > Assume somePath is a java.nio.file.Path and represents a symbolic link > on the file system. > > If somePath points to a nonexistent file on the file system (i.e., it > is a broken symbolic link), then the first snippet of code will fail > to call somePath.delete(). Therefore the broken symbolic link remains > on the file system. However, the second snippet of code always > results in the removal of the symbolic link from the filesystem > (broken link or not). > > The difference is that exists() in the first snippet checks the > existence of the symbolic link's target file and not the symbolic link > file itself; The notion of exists in the second snippet applies to the > symbolic link and not its target. > > Anyone have thoughts about this particular behavior of the exists() > method? Should the exists() method behavior be more explicit by adding > a LinkOption... paramenter? This would enable the call a call like > > somePath.exists(LinkOption.NOFOLLOW_LINK) > > to check the existence of the link itself (if somePath happens to be a > symbolic link) and not the link's target. > > Regards. > It's only the delete and moveTo methods that are specified to not follow sym links. It's not clear to me that changing the exists method is worth it. If someone really needs to then they can use readBasicFileAttributes specifying NOFOLLOW_LINKS. Furthermore, the exists method isn't really needed in this API as you've no guarantee that the file still exists when you go to access it (and from a security point of view it is usually better to remove the window between the check and the access). -Alan. From Alan.Bateman at oracle.com Tue Oct 5 07:45:27 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 05 Oct 2010 15:45:27 +0100 Subject: 6987116: (so) test/java/nio/channels/SocketChannel/VectorIO.java failed on Solaris 11 In-Reply-To: <4CAB1276.9090105@univ-mlv.fr> References: <4CAB080E.6040106@oracle.com> <4CAB1276.9090105@univ-mlv.fr> Message-ID: <4CAB3A07.7040103@oracle.com> R?mi Forax wrote: > : > > Perhaps, you should initialize the pseudo random generator with a seed > > to make the test repeatable. It can be argued both ways. I decided not to change it as the test is relatively simple. -Alan From fudmail at gmail.com Tue Oct 5 08:13:08 2010 From: fudmail at gmail.com (fm) Date: Tue, 5 Oct 2010 08:13:08 -0700 Subject: WatchService - Exposing more of the Inotify Event Model In-Reply-To: <4CAB31D8.2060506@oracle.com> References: <4C7FBBF3.7020501@oracle.com> <4CA6FC6D.1080306@oracle.com> <4CAB31D8.2060506@oracle.com> Message-ID: > I can't gauge the interest but it may be best to focus on events that may be > feasible to implement on more than platform. You could define them in > ExtendedWatchEventKind a bit like the extensions we have for modifiers > already. Thanks. I see your point and agree. At the risk of sounding like a desperate software peddler, how about this idea? A WatchEvent.Modifier implementation that enables you to register a regular file (as opposed to a directory) with a WatchService. E.g., someRegularFile.register(watchService, watchEvents, ExtendedWatchEventModifier.REGULAR_FILE); I believe with the current WatchService implementation you would need to filter any events received from a registered directory to look for your one file of interest. I tested this concept on Linux but I'm unsure if it would work with the native file system event facilities on Windows or Solaris. From forax at univ-mlv.fr Tue Oct 5 09:22:35 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 05 Oct 2010 18:22:35 +0200 Subject: 6987116: (so) test/java/nio/channels/SocketChannel/VectorIO.java failed on Solaris 11 In-Reply-To: <4CAB3A07.7040103@oracle.com> References: <4CAB080E.6040106@oracle.com> <4CAB1276.9090105@univ-mlv.fr> <4CAB3A07.7040103@oracle.com> Message-ID: <4CAB50CB.4000809@univ-mlv.fr> Le 05/10/2010 16:45, Alan Bateman a ?crit : > R?mi Forax wrote: >> : >> >> Perhaps, you should initialize the pseudo random generator with a seed >> >> to make the test repeatable. > It can be argued both ways. I decided not to change it as the test is > relatively simple. > > -Alan Ok. R?mi From xueming.shen at oracle.com Tue Oct 5 11:15:17 2010 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Oct 2010 11:15:17 -0700 Subject: zip filesystem provider In-Reply-To: <4CAB1223.2090703@gmx.de> References: <4CAAD599.4050407@oracle.com> <4CAB1223.2090703@gmx.de> Message-ID: <4CAB6B35.80902@oracle.com> On 10/05/2010 04:55 AM, Ulf Zibis wrote: > Sherman, > > thanks for that great work. > > Maybe you now could acknowledge: > Bug ID: 6818737 Provide NIO channel access for zip/jar entry > > and look forward for: > Bug ID: 6818736 Provide NIO channel access for resources from class > Class > > I'm wondering how much work had to be done in java code. I would have > expected, most should be provided by libzip. > You can now access the zip/jar entry by using the NIO channel already with the zip filesystem provider:-) See Demo.java #checkEqual(Path src, Path dst) SeekableByteChannel chSrc = src.newByteChannel(); OK, that is cheating:-) you just get a channel built on top of a stream. We need a pair of Inflater/DeflaterByteChannel and a little update at native, but probably don't need to touch the libzip (if not too ambitious) I guess I have some code somewhere, but need time to clean them out. -Sherman > -Ulf > > > Am 05.10.2010 09:36, schrieb Xueming Shen: >> Here is the webrev for nio filesystem provider for zip {1] >> >> http://cr.openjdk.java.net/~sherman/zipfs/webrev >> >> We are targeting for b114 for the first drop, if we can address >> some integration logistic issues. If not, it should be in b116. >> >> Since most of the code are so "young" (2-3 weeks), it would be >> appreciated if you guys can help review the code and/or try >> it out, any feedback is welcome. And don't be surprised if you >> find the webrev is being updated constantly:-) >> >> Demo.java shows how to use the NIO2 filesystem APIs to access >> a zip file via the zip filesystem provider, with couple lines >> of code you can copy, rename, move, list, walk a zip file just like >> access a normal file system. The most fun part of the past couple >> weeks is to convert my test case into this Demo:-) Myself is >> a little surprised how easy it is to use Alan's nio fs to access >> a zip file. >> >> I put a copy ofzipfs.jar on the cr website for your convenience, >> in case you want to try it out first. Just copy the zipfs.jar into >> the ext directory or add it into your classpath. >> >> -Sherman >> >> [1]NIO.2 filesystem provider for zip/jar archives -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20101005/6f1bfdbd/attachment.html From dmdabbs at gmail.com Tue Oct 5 11:55:25 2010 From: dmdabbs at gmail.com (David Dabbs) Date: Tue, 5 Oct 2010 13:55:25 -0500 Subject: zip filesystem provider In-Reply-To: <4CAB6B35.80902@oracle.com> References: <4CAAD599.4050407@oracle.com> <4CAB1223.2090703@gmx.de> <4CAB6B35.80902@oracle.com> Message-ID: <009301cb64be$e06a7c70$a13f7550$@com> >It still appears the memory/buffer handling code in Inflater.c can be changed to improve >the performance in use cases similar to above mentioned scenario. The proposed change >is to copy no more than the size of the "user's buffer". It's an de-compression, so you can't >consume more bytes than you can write out in most cases. The attached test case shows >the performance of gzip-ing the un-compressed rt.jar by using different buffer size. >Good find! Do you think this will make it into jdk 6u23 FCS (whenever that will be)? Thanks, David From xueming.shen at oracle.com Tue Oct 5 12:36:56 2010 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Oct 2010 12:36:56 -0700 Subject: zip filesystem provider In-Reply-To: <009301cb64be$e06a7c70$a13f7550$@com> References: <4CAAD599.4050407@oracle.com> <4CAB1223.2090703@gmx.de> <4CAB6B35.80902@oracle.com> <009301cb64be$e06a7c70$a13f7550$@com> Message-ID: <4CAB7E58.7080103@oracle.com> On 10/05/2010 11:55 AM, David Dabbs wrote: > Do you think this will make it into jdk 6u23 FCS (whenever that will be)? Unlikely. The plan is to have this one baked in JDK7 for couple builds before get into any previous releases, as we usually do. I believe the cutoff for "non-stopper" bug fix for 6u23 is within one week or so. But never say never:-) if it is categoried as "stopper" later, it might still be possible to get it in last minute. -Sherman From fudmail at gmail.com Fri Oct 8 22:32:10 2010 From: fudmail at gmail.com (fm) Date: Fri, 8 Oct 2010 22:32:10 -0700 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: <4CAB334B.4000706@oracle.com> References: <4CAB334B.4000706@oracle.com> Message-ID: > If someone really needs to then they can use readBasicFileAttributes specifying > NOFOLLOW_LINKS. Right. But it is currently not possible to check that a symbolic link exists without the possibility of an IOException being thrown by the readBasicFileAttributes method. > It's not clear to me that changing the exists method is worth it. The only downside to changing the method (from what I can see) would be adding slightly to the conceptual complexity of the method (i.e., there would be one more optional argument for the programmer to have to think about or ignore). My 2 cents is that the exists, notExists, and checkAccess methods should have the following signatures: void checkAccess(AccessMode[] modes, LinkOption... options) boolean exists(LinkOption... options) boolean notExists(LinkOption... options) This would add convenience by enabling the programmer to avoid accessing file attributes (and handling an IOException) just to determine if something exists on the file system that corresponds to a java.nio.file.Path instance. From Alan.Bateman at oracle.com Sat Oct 9 11:51:05 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 09 Oct 2010 19:51:05 +0100 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: References: <4CAB334B.4000706@oracle.com> Message-ID: <4CB0B999.1030909@oracle.com> fm wrote: > : > > My 2 cents is that the exists, notExists, and checkAccess methods > should have the following signatures: > > void checkAccess(AccessMode[] modes, LinkOption... options) > > boolean exists(LinkOption... options) > > boolean notExists(LinkOption... options) > > This would add convenience by enabling the programmer to avoid > accessing file attributes (and handling an IOException) just to > determine if something exists on the file system that corresponds to a > java.nio.file.Path instance. > There may be a case where you want to check if a file exists without following links, but I'm not sure that it really make sense for checkAccess. What does it mean for a sym link to be writable or executable for example. In the native world, there isn't an laccess so it's not something that native code can readily do either (it can simulate it but there may be subtle differences). One thing I should say is that checkAccess does need to be re-examined (we have a bug tracking this) so I'll add it to the list of things to look at (for the exists case anyway). -Alan From Alan.Bateman at oracle.com Mon Oct 11 07:50:46 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Oct 2010 15:50:46 +0100 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements Message-ID: <4CB32446.7050706@oracle.com> A long standing gripe for many is that direct buffers are page aligned with the result that the memory usage can be significantly higher than expected. We've never documented the alignment and I'm not aware of anyone that depends on it (if you do then speak up now!). The patches here change the implementation so these buffers aren't page aligned by default. A new VM option is introduced to force page alignment if really needed. Mapped buffers will of course continue to be page aligned. A related issue is that the addition of the management interface for buffer pools subtly changed the semantics of the MaxDirectMemorySize option. That option used to set a limit on the total capacity of all direct buffers, whereas now it limits the total memory size. This is bug 6743526, and the webrev includes the changes to restore the original semantics (which is course is only interesting when these buffers are page aligned). The webrev with the changes is here. I'm only looking for a reviewer for the changes to the jdk repo as I will be going to hotspot-runtime-dev for the hotspot change. http://cr.openjdk.java.net/~alanb/4837564/ Thanks, Alan. From fudmail at gmail.com Mon Oct 11 08:07:33 2010 From: fudmail at gmail.com (fm) Date: Mon, 11 Oct 2010 08:07:33 -0700 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: <4CB0B999.1030909@oracle.com> References: <4CAB334B.4000706@oracle.com> <4CB0B999.1030909@oracle.com> Message-ID: Sorry. I was being careless when I threw the checkAccess method into my conversation...I didn't think that one through. You are right, checkAccess does not seem to make sense for the symbolic links themselves (as opposed to their targets). However, I believe my concern with respect to checking the existence of symbolic links with the exists method is still reasonable. Just to provide another example. The following code throws a FileAlreadyExistsException when somePath corresponds to a broken symbolic link. try { if (!somePath.exists()) // or use somePath.notExists() somePath.createFile(); } catch (FileAlreadyExistsException e) { System.err.println("File exists!?!"); } catch (IOException e) { } Being able to call something like "somePath.exists(LinkOption.NOFOLLOW_LINKS)" would be helpful. On Sat, Oct 9, 2010 at 11:51 AM, Alan Bateman wrote: > fm wrote: >> >> : >> >> My 2 cents is that the exists, notExists, and checkAccess methods >> should have the following signatures: >> >> void checkAccess(AccessMode[] modes, LinkOption... options) >> >> boolean exists(LinkOption... options) >> >> boolean notExists(LinkOption... options) >> >> This would add convenience by enabling the programmer to avoid >> accessing file attributes (and handling an IOException) just to >> determine if something exists on the file system that corresponds to a >> java.nio.file.Path instance. >> > > There may be a case where you want to check if a file exists without > following links, but I'm not sure that it really make sense for checkAccess. > What does it mean for a sym link to be writable or executable for example. > In the native world, there isn't an laccess so it's not something that > native code can readily do either (it can simulate it but there may be > subtle differences). > > One thing I should say is that checkAccess does need to be re-examined (we > have a bug tracking this) so I'll add it to the list of things to look at > (for the exists case anyway). > > -Alan > From Ulf.Zibis at gmx.de Mon Oct 11 08:25:46 2010 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Mon, 11 Oct 2010 17:25:46 +0200 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements In-Reply-To: <4CB32446.7050706@oracle.com> References: <4CB32446.7050706@oracle.com> Message-ID: <4CB32C7A.4080908@gmx.de> Hi, maybe you remember, that I have discussed some changes with Martin Buchholz related to aligning. See thead "Swap should be better done native?" from around april. -Ulf Am 11.10.2010 16:50, schrieb Alan Bateman: > > A long standing gripe for many is that direct buffers are page aligned with the result that the > memory usage can be significantly higher than expected. We've never documented the alignment and > I'm not aware of anyone that depends on it (if you do then speak up now!). The patches here > change the implementation so these buffers aren't page aligned by default. A new VM option is > introduced to force page alignment if really needed. Mapped buffers will of course continue to be > page aligned. > > A related issue is that the addition of the management interface for buffer pools subtly changed > the semantics of the MaxDirectMemorySize option. That option used to set a limit on the total > capacity of all direct buffers, whereas now it limits the total memory size. This is bug 6743526, > and the webrev includes the changes to restore the original semantics (which is course is only > interesting when these buffers are page aligned). > > The webrev with the changes is here. I'm only looking for a reviewer for the changes to the jdk > repo as I will be going to hotspot-runtime-dev for the hotspot change. > http://cr.openjdk.java.net/~alanb/4837564/ > > Thanks, > Alan. > From Alan.Bateman at oracle.com Mon Oct 11 08:31:23 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Oct 2010 16:31:23 +0100 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements In-Reply-To: <4CB32C7A.4080908@gmx.de> References: <4CB32446.7050706@oracle.com> <4CB32C7A.4080908@gmx.de> Message-ID: <4CB32DCB.2040103@oracle.com> Ulf Zibis wrote: > Hi, > > maybe you remember, that I have discussed some changes with Martin > Buchholz related to aligning. See thead "Swap should be better done > native?" from around april. > > -Ulf I don't recall any discussion about the alignment of direct buffers. I suspect you are instead thinking about this: http://hg.openjdk.java.net/jdk7/jdk7/jdk/rev/ce0ba8da0bd1 -Alan. From Ulf.Zibis at gmx.de Mon Oct 11 08:59:45 2010 From: Ulf.Zibis at gmx.de (Ulf Zibis) Date: Mon, 11 Oct 2010 17:59:45 +0200 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements In-Reply-To: <4CB32DCB.2040103@oracle.com> References: <4CB32446.7050706@oracle.com> <4CB32C7A.4080908@gmx.de> <4CB32DCB.2040103@oracle.com> Message-ID: <4CB33471.6080508@gmx.de> Am 11.10.2010 17:31, schrieb Alan Bateman: > Ulf Zibis wrote: >> Hi, >> >> maybe you remember, that I have discussed some changes with Martin Buchholz related to aligning. >> See thead "Swap should be better done native?" from around april. >> >> -Ulf > I don't recall any discussion about the alignment of direct buffers. I suspect you are instead > thinking about this: > http://hg.openjdk.java.net/jdk7/jdk7/jdk/rev/ce0ba8da0bd1 > > -Alan. > This is only one of the changes. Ok, I will come back one day, when I can provide more details. -Ulf From martijnverburg at gmail.com Mon Oct 11 09:36:44 2010 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 11 Oct 2010 17:36:44 +0100 Subject: Questions on the NIO.2 API Message-ID: Hi all, A bit of background first :). Along with my co-author (Ben Evans) I'm at the early stages of writing a book covering modern Java developer topics. This will include the NIO.2 API changes for Java 7 (which we think are an important step forward for Java). I contacted Alan a whole back and he suggested I submit any issues/thoughts we had with the API here. In particular we're focussing on what a fairly new developer to Java might find awkward or difficult (we're testing our samples with lots of grads/undergrads). Apologies if some of this has been discussed before - I try to scan the archives for answers in most cases, but sometimes my research skills fail me :(. The first example we take our readers through is the relationships between FileSystem, Path, FileStore and pals. The listing below simply takes the user through a fairly patsy example of copying a file over (inefficiently I might add, but I'm trying to explore some of the core classes for now). Feel free to skip the code and jump to the bottom, the code is there just for completeness. private?void?listing2_1()?{ ??????????? FileSystem fileSystem = FileSystems.getDefault(); ??????????? Path timeSheetFile = fileSystem.getPath("C:/projects/timesheet.txt"); ??????????? Path backupDir = fileSystem.getPath("H:/projects/"); ????????????try { ????????????????? timeSheetFile.checkAccess(AccessMode.READ); ????????????????? backupDir.checkAccess(AccessMode.WRITE); ????????????????? Path backupFile = fileSystem.getPath("H:/projects/timesheet.txt"); ????????????????? FileStore backupFileStore = backupDir.getFileStore(); ??????????????????if?(!backupFileStore.type().equals("NTFS")) { ????????????????????????throw?new?IOException("Suspicious store type"); ????????????????? } ????????????????? CopyOption copyOptions = StandardCopyOption.REPLACE_EXISTING; ????????????????? timeSheetFile.copyTo(backupFile, copyOptions); ????????????????? System.out.println(backupFile.toAbsolutePath()); ??????????? } catch?(IOException e) { ????????????????? e.printStackTrace(); ??????????? } } So some things I thought were really cool when using this API for the first time: * FileSystems.getDefault() - Great time saving shortcut, easy for new developers * Being able to check access modes directly on a Path * The concept of a FileStore - fantastic * CopyOptions etc - I'll comment on in another example, but I liked this, again easy for developers. Things I wondered about (which there are probably good answers for!): * There seems to be no direct way to query the FileSystem or Path whether you're dealing with a directory as opposed to a file (or other type of Path), something like isDir(). This is something a new developer is likely to 'expect' in an API (perhaps in a specific implementation of a Path or FileSystem). * getPath(String URI) doesn't throw any Exceptions if the underlying file doesn't exist. Only when you try to utilise the non-existent Path does it throw an Exception. Now I assume this is a deliberate design choice (Path's shouldn't be tied to physical resources until needed) but I was wondering if would be sensible to add something like a getPath(String URL, boolean bind) throws NoSuchFileException method so that developers could have the check made immediately (if bind == true). ---------- Is this sort of feedback useful to this list? Or is it just going to waste your valuable time? I've worked on a number of open source projects, so I know what it can be like :). Disclaimer: I'm not a low-level API designer by any stretch of the imagination, so if it's the case that you only want bug reports as opposed to ill-thought out suggestions then I understand! Cheers, Martijn (@java7developer - twitter) From Alan.Bateman at oracle.com Mon Oct 11 10:09:05 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Oct 2010 18:09:05 +0100 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: References: <4CAB334B.4000706@oracle.com> <4CB0B999.1030909@oracle.com> Message-ID: <4CB344B1.9000201@oracle.com> fm wrote: > Sorry. I was being careless when I threw the checkAccess method into > my conversation...I didn't think that one through. You are right, > checkAccess does not seem to make sense for the symbolic links > themselves (as opposed to their targets). > > However, I believe my concern with respect to checking the existence > of symbolic links with the exists method is still reasonable. > > Just to provide another example. The following code throws a > FileAlreadyExistsException when somePath corresponds to a broken > symbolic link. > > try { > if (!somePath.exists()) // or use somePath.notExists() > somePath.createFile(); > } > catch (FileAlreadyExistsException e) { > System.err.println("File exists!?!"); > } > catch (IOException e) { } > > Being able to call something like > "somePath.exists(LinkOption.NOFOLLOW_LINKS)" would be helpful > Yes, this could help for this case but the above may be still with FailAlreadyExistsException if there are other threads (or other processes) competing to create the file. For this specific case then the following might be closer to what you want: file.newOutputStream(CREATE).close(); -Alan From Alan.Bateman at oracle.com Mon Oct 11 11:17:40 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Oct 2010 19:17:40 +0100 Subject: Questions on the NIO.2 API In-Reply-To: References: Message-ID: <4CB354C4.4010605@oracle.com> Martijn Verburg wrote: > : > > Things I wondered about (which there are probably good answers for!): > > * There seems to be no direct way to query the FileSystem or Path > whether you're dealing with a directory as opposed to a file (or other > type of Path), something like isDir(). This is something a new > developer is likely to 'expect' in an API (perhaps in a specific > implementation of a Path or FileSystem). > Yes, we need to do something about this. The awkward issue has always been that isDirectory (or any isXXX method to test the file type) can fail for various reasons and so technically the answer isn't a boolean but rather one of three answers (it's a directory, it's not a directory, or it failed). For now, the workaround is to do something like this: BasicFileAttributes attrs = Attributes.readBasicFileAttributes(file); if (attrs.isDirectory()) { : } but of course when you go to access the file then it might not be a directory because someone/something replaced the file in the mean-time. > * getPath(String URI) doesn't throw any Exceptions if the underlying > file doesn't exist. Only when you try to utilise the non-existent > Path does it throw an Exception. Now I assume this is a deliberate > design choice (Path's shouldn't be tied to physical resources until > needed) but I was wondering if would be sensible to add something like > a getPath(String URL, boolean bind) throws NoSuchFileException method > so that developers could have the check made immediately (if bind == > true). > A side comment, but the parameter to getPath is a path string (not a URI, at least not in the case of the default provider). So is checking if the file exists when you create the Path really useful? There's no guarantee that the file will still exist when you do to access it later. If you really want to verify that you've got a path to a real file then you can test it with the exists method, or if you want an exception, use checkAccess. > > Is this sort of feedback useful to this list? > Yes, definitely. I'm particularly interested, as I'm sure are others, to hear how you get on with the samples that you are giving the students. These students will be the developers that will hopefully be using the platform when they graduate and move into jobs. -Alan. From martijnverburg at gmail.com Tue Oct 12 09:35:56 2010 From: martijnverburg at gmail.com (Martijn Verburg) Date: Tue, 12 Oct 2010 17:35:56 +0100 Subject: Questions on the NIO.2 API In-Reply-To: <4CB354C4.4010605@oracle.com> References: <4CB354C4.4010605@oracle.com> Message-ID: Hi Alan/All, >> Things I wondered about (which there are probably good answers for!): >> >> * There seems to be no direct way to query the FileSystem or Path >> whether you're dealing with a directory as opposed to a file (or other >> type of Path), something like isDir(). ?This is something a new >> developer is likely to 'expect' in an API (perhaps in a specific >> implementation of a Path or FileSystem). >> > Yes, we need to do something about this. The awkward issue has always been > that isDirectory (or any isXXX method to test the file type) can fail for > various reasons and so technically the answer isn't a boolean but rather one > of three answers (it's a directory, it's not a directory, or it failed). For > now, the workaround is to do something like this: > > BasicFileAttributes attrs = Attributes.readBasicFileAttributes(file); > if (attrs.isDirectory()) { > ? : > } > > but of course when you go to access the file then it might not be a > directory because someone/something replaced the file in the mean-time. Understood, I had also thought that there might be a number of cases where it might throw something like an UnsupportedOperationException or something similar (your 'failed' case) as certain implementations of Path might not have the concept of a directory (or other XXX that a developer might naively think is common to all Path like systems). Interesting stuff! I'll keep a note of the workaround in the mean time. >> * getPath(String URI) doesn't throw any Exceptions if the underlying >> file doesn't exist. ?Only when you try to utilise the non-existent >> Path does it throw an Exception. Now I assume this is a deliberate >> design choice (Path's shouldn't be tied to physical resources until >> needed) but I was wondering if would be sensible to add something like >> a getPath(String URL, boolean bind) throws NoSuchFileException method >> so that developers could have the check made immediately (if bind == >> true). >> > > A side comment, but the parameter to getPath is a path string (not a URI, at > least not in the case of the default provider). Yep sorry for the red-herring there, my day job consists of URIs at the moment, got them on my brain :) > So is checking if the file exists when you create the Path really useful? > There's no guarantee that the file will still exist when you do to access it > later. ?If you really want to verify that you've got a path to a real file > then you can test it with the exists method, or if you want an exception, > use checkAccess. That's fair and pretty much what I expected was the design decision, after all your Java code doesn't have sole control over that file! That's actually a good teaching point for developers that don't always have to design code at this sort of level. A little bit like the database and transactions around that analogy. I'll make a special tip to highlight this in the material. >> Is this sort of feedback useful to this list? > > Yes, definitely. I'm particularly interested, as I'm sure are others, to > hear how you get on with the samples that you are giving the students. These > students will be the developers that will hopefully be using the platform > when they graduate and move into jobs. Cool, definitely didn't want to annoy this list (let me know as soon as I do!). I've got quite a number of further examples, but I'll group them into meaningful posts and drop feed them out so I'm not flooding this list. I think the Async I/O stuff is the real mind boggler for newer developers, it can certainly be tricky to explain with simple yet meaningful examples. Thanks again all. Martijn (@java7developer - twitter) From Alan.Bateman at oracle.com Wed Oct 13 07:08:11 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Oct 2010 15:08:11 +0100 Subject: Path.checkAccess (6873681) Message-ID: <4CB5BD4B.3050203@oracle.com> One of methods that needs a bit of re-work is Path's checkAccess method. It's an awkward operation because you either have the requested access or the test fails for any number of reasons. When it fails then you can't reliably determine if it failed because you don't have the requested access, or because you didn't have effective permissions to test the access. A couple of folks have suggested adding the equivalent of File's canXXX methods and there is some merit in that except that a simple boolean doesn't convey the reason when it fails. There is also the security (and performance) argument to simply remove the method as it isn't really needed and you are almost always better to just attempt access to the file and not have a window between the time of the check to the time of the usage. In any case, we need to do something with checkAccess as the exception makes it difficult to use. One possible approach is to change checkAccess so that it doesn't throw an IOException but instead returns an Access or some object that represents the result of the check, eg: interface Access { boolean isAllowed(); IOException denyReason(); } That would allow code such as the following: if (file.checkAccess(READ,WRITE).isAllowed()) { } -or- Access acc = file.checkAccess(EXECUTE); if (acc.isAllowed()) { : } else { IOException ioe = acc.denyReason(); : } This approach make it trivial to implement methods such as exception-less isReadable, isWritable, isExecutable if they are needed. I'm interested to hear if there are folks using checkAccess, whether the IOException has been a problem, and whether a solution along the lines here would be usable. Thanks, -Alan. [1] http://download.java.net/jdk7/docs/api/java/nio/file/Path.html#checkAccess(java.nio.file.AccessMode...) From forax at univ-mlv.fr Wed Oct 13 07:58:08 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 13 Oct 2010 16:58:08 +0200 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5BD4B.3050203@oracle.com> References: <4CB5BD4B.3050203@oracle.com> Message-ID: <4CB5C900.2030903@univ-mlv.fr> Usually when a method can fail for different reasons we use different exceptions. checkAccess already returns different kind of IOException so I don't see your point. In my opinion, you should update the doc to says that checkAccess is useless and also add a link to Attributes. R?mi Le 13/10/2010 16:08, Alan Bateman a ?crit : > > One of methods that needs a bit of re-work is Path's checkAccess > method. It's an awkward operation because you either have the > requested access or the test fails for any number of reasons. When it > fails then you can't reliably determine if it failed because you don't > have the requested access, or because you didn't have effective > permissions to test the access. A couple of folks have suggested > adding the equivalent of File's canXXX methods and there is some merit > in that except that a simple boolean doesn't convey the reason when it > fails. There is also the security (and performance) argument to simply > remove the method as it isn't really needed and you are almost always > better to just attempt access to the file and not have a window > between the time of the check to the time of the usage. In any case, > we need to do something with checkAccess as the exception makes it > difficult to use. > > One possible approach is to change checkAccess so that it doesn't > throw an IOException but instead returns an Access or some object that > represents the result of the check, eg: > > interface Access { > boolean isAllowed(); > IOException denyReason(); > } > > That would allow code such as the following: > > if (file.checkAccess(READ,WRITE).isAllowed()) { > } > > -or- > > Access acc = file.checkAccess(EXECUTE); > if (acc.isAllowed()) { > : > } else { > IOException ioe = acc.denyReason(); > : > } > > This approach make it trivial to implement methods such as > exception-less isReadable, isWritable, isExecutable if they are needed. > > I'm interested to hear if there are folks using checkAccess, whether > the IOException has been a problem, and whether a solution along the > lines here would be usable. > > Thanks, > > -Alan. > > > [1] > http://download.java.net/jdk7/docs/api/java/nio/file/Path.html#checkAccess(java.nio.file.AccessMode...) > > > From Alan.Bateman at oracle.com Wed Oct 13 09:32:45 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Oct 2010 17:32:45 +0100 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5C900.2030903@univ-mlv.fr> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> Message-ID: <4CB5DF2D.40103@oracle.com> R?mi Forax wrote: > Usually when a method can fail for different reasons we use different > exceptions. > checkAccess already returns different kind of IOException so I don't > see your point. It would be great to get rid of checkAccess but we have to provide some means for those that think they need it. The issue with the current method is that it's not easy to use, especially where you want to take action where you have the requested access. -Alan. From forax at univ-mlv.fr Wed Oct 13 10:15:40 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 13 Oct 2010 19:15:40 +0200 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5DF2D.40103@oracle.com> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> Message-ID: <4CB5E93C.8000102@univ-mlv.fr> Le 13/10/2010 18:32, Alan Bateman a ?crit : > R?mi Forax wrote: >> Usually when a method can fail for different reasons we use different >> exceptions. >> checkAccess already returns different kind of IOException so I don't >> see your point. > It would be great to get rid of checkAccess but we have to provide > some means for those that think they need it. The issue with the > current method is that it's not easy to use, especially where you want > to take action where you have the requested access. > > -Alan. The other solution is to move checkAccess in BasicFileAttributes or in Attributes. If you decide to move checkAccess, I think that isHidden should follow. R?mi From Alan.Bateman at oracle.com Wed Oct 13 10:27:44 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Oct 2010 18:27:44 +0100 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5E93C.8000102@univ-mlv.fr> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> Message-ID: <4CB5EC10.1050902@oracle.com> R?mi Forax wrote: > : > The other solution is to move checkAccess in BasicFileAttributes or in > Attributes. > If you decide to move checkAccess, I think that isHidden should follow. The effective access relates to the file content rather than the attributes so I don't think this would work too well. The isHidden is also not related to the basic attributes (it's a function of the file name on Unix for example). -Alan From fudmail at gmail.com Wed Oct 13 11:07:36 2010 From: fudmail at gmail.com (fm) Date: Wed, 13 Oct 2010 11:07:36 -0700 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5EC10.1050902@oracle.com> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> <4CB5EC10.1050902@oracle.com> Message-ID: I think Alan is on to something here. I think good programming warrants checking a file for access (prior to doing so) if there is a "reasonable" chance that accessing the file may fail due to some permission issue. Of course, any code actually accessing a file will still need to catch (or declare to throw ) the various IOExceptions that may occur. On a related subject, it has been brought up a few times that convenience methods such as checkAccess(...) or exists() don't guarantee that access or existence to a file hasn't changed between the time of the check and the time that the file is actually accessed. I think this risk will be understood by programmers and does not invalidate the usefulness of these convenience methods. If I can handle an alternative case because I did a check first as opposed to reacting to it by catching the exception, then this can often be a good thing. The IOException hierarchy will always be there as a last resort (or safety net) to handling file access that goes wrong for some reason. On Wed, Oct 13, 2010 at 10:27 AM, Alan Bateman wrote: > R?mi Forax wrote: >> >> : >> The other solution is to move checkAccess in BasicFileAttributes or in >> Attributes. >> If you decide to move checkAccess, I think that isHidden should follow. > > The effective access relates to the file content rather than the attributes > so I don't think this would work too well. The isHidden is also not related > to the basic attributes (it's a function of the file name on Unix for > example). > > -Alan > From forax at univ-mlv.fr Wed Oct 13 11:17:59 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 13 Oct 2010 20:17:59 +0200 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5EC10.1050902@oracle.com> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> <4CB5EC10.1050902@oracle.com> Message-ID: <4CB5F7D7.9070301@univ-mlv.fr> Le 13/10/2010 19:27, Alan Bateman a ?crit : > R?mi Forax wrote: >> : >> The other solution is to move checkAccess in BasicFileAttributes or >> in Attributes. >> If you decide to move checkAccess, I think that isHidden should follow. > The effective access relates to the file content rather than the > attributes so I don't think this would work too well. The isHidden is > also not related to the basic attributes (it's a function of the file > name on Unix for example). > > -Alan If you put them in Attributes, they can take a Path as parameter. Or perhaps they should be in Paths. R?mi From forax at univ-mlv.fr Wed Oct 13 11:26:37 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 13 Oct 2010 20:26:37 +0200 Subject: Path.checkAccess (6873681) In-Reply-To: References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> <4CB5EC10.1050902@oracle.com> Message-ID: <4CB5F9DD.9040609@univ-mlv.fr> Le 13/10/2010 20:07, fm a ?crit : > I think Alan is on to something here. I think good programming > warrants checking a file for access (prior to doing so) if there is a > "reasonable" chance that accessing the file may fail due to some > permission issue. Of course, any code actually accessing a file will > still need to catch (or declare to throw ) the various IOExceptions > that may occur. > > On a related subject, it has been brought up a few times that > convenience methods such as checkAccess(...) or exists() don't > guarantee that access or existence to a file hasn't changed between > the time of the check and the time that the file is actually accessed. > I think this risk will be understood by programmers and does not > invalidate the usefulness of these convenience methods. If I can > handle an alternative case because I did a check first as opposed to > reacting to it by catching the exception, then this can often be a > good thing. The IOException hierarchy will always be there as a last > resort (or safety net) to handling file access that goes wrong for > some reason. > Without checkAccess the code is simpler ! This is, in my opinion, a strong argument for not using checkAccess. before by example opening a file. R?mi > On Wed, Oct 13, 2010 at 10:27 AM, Alan Bateman wrote: > >> R?mi Forax wrote: >> >>> : >>> The other solution is to move checkAccess in BasicFileAttributes or in >>> Attributes. >>> If you decide to move checkAccess, I think that isHidden should follow. >>> >> The effective access relates to the file content rather than the attributes >> so I don't think this would work too well. The isHidden is also not related >> to the basic attributes (it's a function of the file name on Unix for >> example). >> >> -Alan >> >> From mthornton at optrak.co.uk Wed Oct 13 11:39:22 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Wed, 13 Oct 2010 19:39:22 +0100 Subject: Path.checkAccess (6873681) In-Reply-To: References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> <4CB5EC10.1050902@oracle.com> Message-ID: <4CB5FCDA.5040906@optrak.co.uk> On 13/10/2010 19:07, fm wrote: > On a related subject, it has been brought up a few times that > convenience methods such as checkAccess(...) or exists() don't > guarantee that access or existence to a file hasn't changed between > the time of the check and the time that the file is actually accessed. > I think this risk will be understood by programmers and does not > invalidate the usefulness of these convenience methods. If I can > handle an alternative case because I did a check first as opposed to > reacting to it by catching the exception, then this can often be a > good thing. The IOException hierarchy will always be there as a last > resort (or safety net) to handling file access that goes wrong for > some reason. In my experience the main reason for using check methods has been because you couldn't programmatically determine the reason for failure from the IOException (or some methods didn't throw an exception at all --- File.delete). Given a useful exception heirarchy I would far prefer to try the operation first and then handle the exception if one arises. Mark Thornton From sean.coffey at oracle.com Thu Oct 14 02:19:27 2010 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Thu, 14 Oct 2010 10:19:27 +0100 Subject: Code review request for 6974104: TEST: sun/nio/ch/6645197.java should be fixed in 1.5.0u25b05 and jdk6 workspace Message-ID: <4CB6CB1F.1060008@oracle.com> I'd like to add a testcase for 6645197 to the jdk 7 workspace. 6645197 was recently fixed in jdk6 and this is the same testcase being forward ported. webrev : http://cr.openjdk.java.net/~coffeys/6974104/webrev.00/ Regards, Sean. From Alan.Bateman at oracle.com Thu Oct 14 03:45:44 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Oct 2010 11:45:44 +0100 Subject: Path.checkAccess (6873681) In-Reply-To: <4CB5FCDA.5040906@optrak.co.uk> References: <4CB5BD4B.3050203@oracle.com> <4CB5C900.2030903@univ-mlv.fr> <4CB5DF2D.40103@oracle.com> <4CB5E93C.8000102@univ-mlv.fr> <4CB5EC10.1050902@oracle.com> <4CB5FCDA.5040906@optrak.co.uk> Message-ID: <4CB6DF58.6010708@oracle.com> Mark Thornton wrote: > In my experience the main reason for using check methods has been > because you couldn't programmatically determine the reason for failure > from the IOException (or some methods didn't throw an exception at all > --- File.delete). Given a useful exception heirarchy I would far > prefer to try the operation first and then handle the exception if one > arises. A while back I attempted to find out where the typical usages of canXXX were and in many cases, it was because the file operations lacked useful exceptions. As we've addressed that one, then the need for most of these usages should go away (as intended). That should make the application more secure and also improve performance as each of those canXXX methods required checking the file. Other usages included GUI apps where the result of canXXX influence how the file was displayed (in grey if the file is read-only for example). Using the existing checkAccess for this type of usage is a bit awkward, but it does suggest there are usages where we still require some support. -Alan From chris.hegarty at oracle.com Thu Oct 14 04:41:16 2010 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Oct 2010 12:41:16 +0100 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements In-Reply-To: <4CB32446.7050706@oracle.com> References: <4CB32446.7050706@oracle.com> Message-ID: <4CB6EC5C.9020401@oracle.com> Seems fine, some minor comments: // The "-XX:+PageAlignDirectMemory" option can be used to force // *buffer*, allocated by ByteBuffer.allocateDirect, to be page aligned. typo? buffer -> buffers Could we cache the value of VM.isDirectMemoryPageAligned() as a private static final rather than always invoking isDirectMemoryPageAligned from the constructor? Dito for Bits.pageSize? Or is it possible for these classes to be loaded before these values are initialized? Also, is there an issue with the publication of pageAlignDirectMemory? I can't see that it is every referenced within a sync block or from a volatile. int size = Math.max(1, cap + (pa ? ps : 0)); Are you concerned with overflow when page alignment is on? I'm just trying to understand what this is for, and what will happen if (Integer.MAX_VALUE - 1) -Chris. Alan Bateman wrote: > > A long standing gripe for many is that direct buffers are page aligned > with the result that the memory usage can be significantly higher than > expected. We've never documented the alignment and I'm not aware of > anyone that depends on it (if you do then speak up now!). The patches > here change the implementation so these buffers aren't page aligned by > default. A new VM option is introduced to force page alignment if > really needed. Mapped buffers will of course continue to be page aligned. > > A related issue is that the addition of the management interface for > buffer pools subtly changed the semantics of the MaxDirectMemorySize > option. That option used to set a limit on the total capacity of all > direct buffers, whereas now it limits the total memory size. This is bug > 6743526, and the webrev includes the changes to restore the original > semantics (which is course is only interesting when these buffers are > page aligned). > > The webrev with the changes is here. I'm only looking for a reviewer for > the changes to the jdk repo as I will be going to hotspot-runtime-dev > for the hotspot change. > http://cr.openjdk.java.net/~alanb/4837564/ > > Thanks, > Alan. From Alan.Bateman at oracle.com Thu Oct 14 05:17:08 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Oct 2010 13:17:08 +0100 Subject: 4837564: (bf) Please make DirectByteBuffer performance enhancements In-Reply-To: <4CB6EC5C.9020401@oracle.com> References: <4CB32446.7050706@oracle.com> <4CB6EC5C.9020401@oracle.com> Message-ID: <4CB6F4C4.4030502@oracle.com> Chris Hegarty wrote: > Seems fine, some minor comments: > > // The "-XX:+PageAlignDirectMemory" option can be used to force > // *buffer*, allocated by ByteBuffer.allocateDirect, to be page aligned. > > typo? buffer -> buffers Thanks, I'll fix that. > > > Could we cache the value of VM.isDirectMemoryPageAligned() as a > private static final rather than always invoking > isDirectMemoryPageAligned from the constructor? Dito for > Bits.pageSize? Or is it possible for these classes to be loaded before > these values are initialized? > > Also, is there an issue with the publication of pageAlignDirectMemory? > I can't see that it is every referenced within a sync block or from a > volatile. It's probably not worth caching the result of VM.isDirectMemoryPageAligned as the compiler will inline it. I'm not too concerned about the initialization because this runs as part of the initialization of the system classes, run as part of the VM initialiation. > > int size = Math.max(1, cap + (pa ? ps : 0)); > Are you concerned with overflow when page alignment is on? I'm just > trying to understand what this is for, and what will happen if > (Integer.MAX_VALUE - 1) Hmm, that's a good point. I don't think this has been noticed before. With these changes then we'd still have this issue if running with -XX:+PageAlignDirectMemory. Okay, I'll fix that - thanks for spotting it. -Alan. From Alan.Bateman at oracle.com Thu Oct 14 08:11:16 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Oct 2010 16:11:16 +0100 Subject: Code review request for 6974104: TEST: sun/nio/ch/6645197.java should be fixed in 1.5.0u25b05 and jdk6 workspace In-Reply-To: <4CB6CB1F.1060008@oracle.com> References: <4CB6CB1F.1060008@oracle.com> Message-ID: <4CB71D94.8070907@oracle.com> Se?n Coffey wrote: > I'd like to add a testcase for 6645197 to the jdk 7 workspace. > > 6645197 was recently fixed in jdk6 and this is the same testcase > being forward ported. > > webrev : http://cr.openjdk.java.net/~coffeys/6974104/webrev.00/ > > Regards, > Sean. Thanks for contributing a test for this. I have a couple of comments: I would it better to name this something like TemporarySelector? Does the test need to run in othervm mode? I see you specify a max heap of 5MB so I'm wondering if you were able to duplicate it with any heap size. I remember when Sherman fixed this bug originally and I don't think he could create a reliable test at the time. If you do change it to run in samevm mode the server socket channel and the accepted channel will need to be closed so that the test doesn't complete leaving channels open. It would be good if it duplicated in less than 20s - related to the samevm comment we've been trying to get the test times down. I don't think the acceptor thread is needed as you just need a loopback connection to run this test. You'll see examples in several of the other tests if it helps. -Alan From sean.coffey at oracle.com Mon Oct 18 04:08:58 2010 From: sean.coffey at oracle.com (=?ISO-8859-1?Q?Se=E1n_Coffey?=) Date: Mon, 18 Oct 2010 12:08:58 +0100 Subject: Code review request for 6974104: TEST: sun/nio/ch/6645197.java should be fixed in 1.5.0u25b05 and jdk6 workspace In-Reply-To: <4CB71D94.8070907@oracle.com> References: <4CB6CB1F.1060008@oracle.com> <4CB71D94.8070907@oracle.com> Message-ID: <4CBC2ACA.4030305@oracle.com> Alan, comments inline. http://cr.openjdk.java.net/~coffeys/6974104/webrev.01/ On 14/10/2010 16:11, Alan Bateman wrote: > Se?n Coffey wrote: >> I'd like to add a testcase for 6645197 to the jdk 7 workspace. >> >> 6645197 was recently fixed in jdk6 and this is the same testcase >> being forward ported. >> >> webrev : http://cr.openjdk.java.net/~coffeys/6974104/webrev.00/ >> >> Regards, >> Sean. > Thanks for contributing a test for this. I have a couple of comments: > > I would it better to name this something like TemporarySelector? Done. > > Does the test need to run in othervm mode? I see you specify a max > heap of 5MB so I'm wondering if you were able to duplicate it with any > heap size. I remember when Sherman fixed this bug originally and I > don't think he could create a reliable test at the time. If you do > change it to run in samevm mode the server socket channel and the > accepted channel will need to be closed so that the test doesn't > complete leaving channels open. Yes - jtreg doesn't appear to allow the use of Xmx flags when run in samevm mode. The reduced heap size is necessary to make sure the GC cycles are run good and early in the testcase. I didn't see the failure occur at 64MB heap size when socket timeout was for 20 seconds. > > It would be good if it duplicated in less than 20s - related to the > samevm comment we've been trying to get the test times down. Socket timeout of 5 second sees testcase not failing. Socket timeout of 10 seconds works. I've changed it to 10 seconds for performance. > > I don't think the acceptor thread is needed as you just need a > loopback connection to run this test. You'll see examples in several > of the other tests if it helps. You're right here. I'm not too used to networking testcases. Changes made as suggested. http://cr.openjdk.java.net/~coffeys/6974104/webrev.01/ regards, Sean. > > -Alan > > > > From Alan.Bateman at oracle.com Mon Oct 18 08:01:48 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Oct 2010 16:01:48 +0100 Subject: Code review request for 6974104: TEST: sun/nio/ch/6645197.java should be fixed in 1.5.0u25b05 and jdk6 workspace In-Reply-To: <4CBC2ACA.4030305@oracle.com> References: <4CB6CB1F.1060008@oracle.com> <4CB71D94.8070907@oracle.com> <4CBC2ACA.4030305@oracle.com> Message-ID: <4CBC615C.1020200@oracle.com> Se?n Coffey wrote: > Alan, > > comments inline. > > http://cr.openjdk.java.net/~coffeys/6974104/webrev.01/ The updated test looks okay to me - thanks again for doing this. -Alan. From Alan.Bateman at oracle.com Thu Oct 21 04:45:39 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Oct 2010 12:45:39 +0100 Subject: 6816049: (bf) MappedByteBuffer.force() method does not flush data correctly Message-ID: <4CC027E3.4010500@oracle.com> MappedByteBuffer.force doesn't currently work as expected on Windows because that operating system doesn't provider the equivalent of msync with MS_SYNC semantics. It's currently implemented with FlushViewOfFile but that is essentially msync(MS_ASYNC) and so doesn't wait until the dirty pages have been written to the disk. We've been back and forth with Microsoft on this a few times with a view to encouraging them to add support in a future release. In the mean-time, the only way to address this is to keep a handle to the original file and use FlushFileBuffers. This isn't completely satisfactory but preferable to having force complete prematurely. To keep things simple, the changes here duplicate the file handle when creating a file mapping so that it may be used subsequently by the force method. In other words, on Windows, each mapped buffer will have its own handle to the original file. This allows the file channel to be "closed" after the mapping is obtained, although the underlying file will remain open until the channel, and all mapped buffers obtained from it, are closed. The solution is preferable to changing the close mechanism and keeping track of each of the mapped buffers. The side effects to keeping the file open aren't as bad as it might appear and the only real issue that I can think of is where someone continues to use the channel after mapping a region, some I/O operation on the file blocks because of a network/other issue, and the asynchronous close doesn't preempt it. On balance it's probably not worth worrying about this case as the I/O operation will complete in a timely manner with the error anyway. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/6816049/webrev/ Thanks, -Alan. From Alan.Bateman at oracle.com Thu Oct 21 05:00:27 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Oct 2010 13:00:27 +0100 Subject: 6993267: TEST_BUG: java/nio/file/Path/InterruptCopy.java fails intermittently (win) Message-ID: <4CC02B5B.6050500@oracle.com> This test exercises copyTo with an implementation specific option that allows the copy to be interrupted. It fails intermittently on some of our faster Windows machines because the copy completes before it can be interrupted. The follow webrev fixes the test to make it more robust and handle the case where the copy completes quickly. http://cr.openjdk.java.net/~alanb/6993267/webrev/ Thanks, Alan. From forax at univ-mlv.fr Thu Oct 21 05:37:21 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 21 Oct 2010 14:37:21 +0200 Subject: 6993267: TEST_BUG: java/nio/file/Path/InterruptCopy.java fails intermittently (win) In-Reply-To: <4CC02B5B.6050500@oracle.com> References: <4CC02B5B.6050500@oracle.com> Message-ID: <4CC03401.7050004@univ-mlv.fr> Le 21/10/2010 14:00, Alan Bateman a ?crit : > > This test exercises copyTo with an implementation specific option that > allows the copy to be interrupted. It fails intermittently on some of > our faster Windows machines because the copy completes before it can > be interrupted. The follow webrev fixes the test to make it more > robust and handle the case where the copy completes quickly. > http://cr.openjdk.java.net/~alanb/6993267/webrev/ > > Thanks, > Alan. Hi Alan, if wakeup task is interrupted, you should throw an assertion error, it should never happen. so line 102 should be } catch (InterruptedException e) { throw (AssertionError)new Assertionerror().initCause(e); } Otherwise, it's Ok. R?mi From Alan.Bateman at oracle.com Thu Oct 21 05:46:56 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Oct 2010 13:46:56 +0100 Subject: 6993267: TEST_BUG: java/nio/file/Path/InterruptCopy.java fails intermittently (win) In-Reply-To: <4CC03401.7050004@univ-mlv.fr> References: <4CC02B5B.6050500@oracle.com> <4CC03401.7050004@univ-mlv.fr> Message-ID: <4CC03640.2090102@oracle.com> R?mi Forax wrote: > : > Hi Alan, > if wakeup task is interrupted, you should throw an assertion error, > it should never happen. > > so line 102 should be > > } catch (InterruptedException e) { > throw (AssertionError)new Assertionerror().initCause(e); > } > > > Otherwise, it's Ok. > > R?mi > The wakeup task interrupts the main thread, so if the copy (on the main thread) is really fast then the main thread may be blocked in wakeup.get when it is interrupted. So I think I have it right, and it's easy to demonstrate by copying a very small file. -Alan. From forax at univ-mlv.fr Thu Oct 21 06:08:51 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 21 Oct 2010 15:08:51 +0200 Subject: 6993267: TEST_BUG: java/nio/file/Path/InterruptCopy.java fails intermittently (win) In-Reply-To: <4CC03640.2090102@oracle.com> References: <4CC02B5B.6050500@oracle.com> <4CC03401.7050004@univ-mlv.fr> <4CC03640.2090102@oracle.com> Message-ID: <4CC03B63.1080401@univ-mlv.fr> Le 21/10/2010 14:46, Alan Bateman a ?crit : > R?mi Forax wrote: >> : >> Hi Alan, >> if wakeup task is interrupted, you should throw an assertion error, >> it should never happen. >> >> so line 102 should be >> >> } catch (InterruptedException e) { >> throw (AssertionError)new Assertionerror().initCause(e); >> } >> >> >> Otherwise, it's Ok. >> >> R?mi >> > The wakeup task interrupts the main thread, so if the copy (on the > main thread) is really fast then the main thread may be blocked in > wakeup.get when it is interrupted. So I think I have it right, and > it's easy to demonstrate by copying a very small file. > > -Alan. Ok, got it. Yes, you're right. R?mi From Alan.Bateman at oracle.com Thu Oct 21 12:00:11 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Oct 2010 20:00:11 +0100 Subject: anyone using AsynchronousDatagramChannel? Message-ID: <4CC08DBB.9000904@oracle.com> We're thinking of removing AsynchronousDatagramChannel (it provides an asynchronous channel for datagram-oriented sockets). This channel was included in the API for consistency reasons. It's not needed for scalability reasons. The implementation of this channel isn't up to scratch (it's a prototype implementation), the spec needs a bit of work, and there is testing work to do it. There are many other more important things to get done and there's no reason why we can't bring it back, say in jdk8. So I'm interested to know if anyone is actually using it? I think it's only been mentioned once on this list. Thanks, -Alan. From chris.hegarty at oracle.com Fri Oct 22 03:01:54 2010 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Oct 2010 11:01:54 +0100 Subject: 6816049: (bf) MappedByteBuffer.force() method does not flush data correctly In-Reply-To: <4CC027E3.4010500@oracle.com> References: <4CC027E3.4010500@oracle.com> Message-ID: <4CC16112.9090004@oracle.com> Looks fine, some minor comments: The Direct-X-Buffer constructors, not for memory-mapped buffers, can simply invoke 4 arg super constructor, instead of passing a null fd, right? Thought, this was always the case. The indentation of L55 (2nd line of args to force0) of the windows version of MappedByteBuffer.c seems wrong (too much). Or maybe this is webrev. -Chris. Alan Bateman wrote: > > MappedByteBuffer.force doesn't currently work as expected on Windows > because that operating system doesn't provider the equivalent of msync > with MS_SYNC semantics. It's currently implemented with FlushViewOfFile > but that is essentially msync(MS_ASYNC) and so doesn't wait until the > dirty pages have been written to the disk. We've been back and forth > with Microsoft on this a few times with a view to encouraging them to > add support in a future release. In the mean-time, the only way to > address this is to keep a handle to the original file and use > FlushFileBuffers. This isn't completely satisfactory but preferable to > having force complete prematurely. > > To keep things simple, the changes here duplicate the file handle when > creating a file mapping so that it may be used subsequently by the force > method. In other words, on Windows, each mapped buffer will have its own > handle to the original file. This allows the file channel to be "closed" > after the mapping is obtained, although the underlying file will remain > open until the channel, and all mapped buffers obtained from it, are > closed. The solution is preferable to changing the close mechanism and > keeping track of each of the mapped buffers. The side effects to keeping > the file open aren't as bad as it might appear and the only real issue > that I can think of is where someone continues to use the channel after > mapping a region, some I/O operation on the file blocks because of a > network/other issue, and the asynchronous close doesn't preempt it. On > balance it's probably not worth worrying about this case as the I/O > operation will complete in a timely manner with the error anyway. > > The webrev with the changes is here: > http://cr.openjdk.java.net/~alanb/6816049/webrev/ > > Thanks, > -Alan. From Alan.Bateman at oracle.com Fri Oct 22 06:08:34 2010 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Oct 2010 14:08:34 +0100 Subject: 6816049: (bf) MappedByteBuffer.force() method does not flush data correctly In-Reply-To: <4CC16112.9090004@oracle.com> References: <4CC027E3.4010500@oracle.com> <4CC16112.9090004@oracle.com> Message-ID: <4CC18CD2.9090107@oracle.com> Chris Hegarty wrote: > Looks fine, some minor comments: > > The Direct-X-Buffer constructors, not for memory-mapped buffers, can > simply invoke 4 arg super constructor, instead of passing a null fd, > right? Thought, this was always the case. Good idea, that would be cleaner. > > The indentation of L55 (2nd line of args to force0) of the windows > version of MappedByteBuffer.c seems wrong (too much). Or maybe this is > webrev. Windows and 8 space tabbing more likely. I've updated the webrev, and thanks for reviewing. -Alan.