From Alan.Bateman at oracle.com Sat Jan 8 11:17:26 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 08 Jan 2011 19:17:26 +0000 Subject: 7002957: (fc) FileChannel.transferTo fails to load libsendfile on Solaris 64-bit Message-ID: <4D28B846.7060401@oracle.com> This one came up on this mailing list a few weeks ago. The reason that sendfile isn't used is because this code attempts to load /usr/lib/libsendfile.so.1 but it should be loading from /usr/lib64 when running in 64-bit. There's no reason now to load this library via dlopen so I've changed the code so that we link against libsendfile. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/7002957/webrev/ Thanks, Alan. From chris.hegarty at oracle.com Sun Jan 9 01:42:57 2011 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 09 Jan 2011 09:42:57 +0000 Subject: 7002957: (fc) FileChannel.transferTo fails to load libsendfile on Solaris 64-bit In-Reply-To: <4D28B846.7060401@oracle.com> References: <4D28B846.7060401@oracle.com> Message-ID: <4D298321.8050704@oracle.com> If I understand the changes correctly, you explicitly state the xxx64 version of the function (and type) for the Solaris 32 bit compilation, but the regular sendfilev for _LP64. Is this a minor optimization? I think it would be simpler to just use sendfilev64 for both. Or would this be incorrect? -Chris. On 01/ 8/11 07:17 PM, Alan Bateman wrote: > > This one came up on this mailing list a few weeks ago. The reason that > sendfile isn't used is because this code attempts to load > /usr/lib/libsendfile.so.1 but it should be loading from /usr/lib64 when > running in 64-bit. There's no reason now to load this library via dlopen > so I've changed the code so that we link against libsendfile. The webrev > with the changes is here: > http://cr.openjdk.java.net/~alanb/7002957/webrev/ > > Thanks, > Alan. From Alan.Bateman at oracle.com Sun Jan 9 09:53:16 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 09 Jan 2011 17:53:16 +0000 Subject: 7002957: (fc) FileChannel.transferTo fails to load libsendfile on Solaris 64-bit In-Reply-To: <4D298321.8050704@oracle.com> References: <4D28B846.7060401@oracle.com> <4D298321.8050704@oracle.com> Message-ID: <4D29F60C.40804@oracle.com> Chris Hegarty wrote: > If I understand the changes correctly, you explicitly state the xxx64 > version of the function (and type) for the Solaris 32 bit compilation, > but the regular sendfilev for _LP64. Is this a minor optimization? I > think it would be simpler to just use sendfilev64 for both. Or would > this be incorrect? > > -Chris. When you compile for 64-bit then it supports large files/offsets by default and so sendfilev and sendfilev64 are the same thing. With 32-bit apps then you have to use the transitional interfaces (xxx64). I've cleaned it up slightly to only use sendfilev64 as you suggested and this will compile to the same thing as before. The main overall change is to link against libsendfile rather than using dlopen to attempt to load libsendfile. -Alan From chris.hegarty at oracle.com Sun Jan 9 12:03:49 2011 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 09 Jan 2011 20:03:49 +0000 Subject: 7002957: (fc) FileChannel.transferTo fails to load libsendfile on Solaris 64-bit In-Reply-To: <4D29F60C.40804@oracle.com> References: <4D28B846.7060401@oracle.com> <4D298321.8050704@oracle.com> <4D29F60C.40804@oracle.com> Message-ID: <4D2A14A5.6040403@oracle.com> On 01/ 9/11 05:53 PM, Alan Bateman wrote: > ... > When you compile for 64-bit then it supports large files/offsets by > default and so sendfilev and sendfilev64 are the same thing. With 32-bit > apps then you have to use the transitional interfaces (xxx64). I've > cleaned it up slightly to only use sendfilev64 as you suggested and this > will compile to the same thing as before. The main overall change is to Thanks, to me this looks simpler. > link against libsendfile rather than using dlopen to attempt to load > libsendfile. Yes, this part looks fine. -Chris. > > -Alan From Alan.Bateman at oracle.com Mon Jan 10 07:03:06 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Jan 2011 15:03:06 +0000 Subject: WatchService In-Reply-To: <4D0A79EE.5000002@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> Message-ID: <4D2B1FAA.40906@oracle.com> Wolfgang Baltes wrote: > > Hi Alan, > > I took the WatchDir sample code and modified it as follows: > - The directory parsing and event loop runs in a separate thread, > which allows me to delete files and directories programmatically. > - Added a few extra console outputs to better catch what is going on. > - In the event loop, added a section to catch the particular condition > in which we are interested. > > The bottom line of what I found is that the behavior changes depending > on whether the deleted subdirectory is empty or not. To reproduce: > > 1: Change the code to create a parent directory to your liking. This > is in line 241: final Path dir = Paths.get("c:\\watchtest"); > > 2: Run the code without other modification. It will create the before > mentioned directory, plus a subdirectory, plus two files in that > subdirectory, then waits for something to happen. > > 3: Manually delete "subdir". The output is: > Scanning c:\watchtest ... > register: c:\watchtest > register: c:\watchtest\subdir > Done with registering directories. > ENTRY_DELETE: c:\watchtest\subdir > c:\watchtest\subdir does not exist... ... but key is valid. > ENTRY_MODIFY: c:\watchtest\subdir\file1.txt > ENTRY_MODIFY: c:\watchtest\subdir\file2.txt > > 4: Kill the thread and start over. > > 5: Manually delete first the two files in subdir, and then subdir. The > output is: > Scanning c:\watchtest ... > register: c:\watchtest > register: c:\watchtest\subdir > Done with registering directories. > ENTRY_DELETE: c:\watchtest\subdir\file1.txt > ENTRY_MODIFY: c:\watchtest\subdir > ENTRY_DELETE: c:\watchtest\subdir\file2.txt > ENTRY_MODIFY: c:\watchtest\subdir > Removing watch key for c:\watchtest\subdir. > ENTRY_DELETE: c:\watchtest\subdir > > 6: You can kill the watch thread by deleting directory "watchtest". > > 7: You can remove the comments from lines 265 to 273 to automate step 5. > > For me it is important to make step 3 above work correctly. This is > what a user would do when a directory needs to be removed manually. > > I hope this helps. Please let me know how I can be of further help. > Wolfgang. Continuing the thread from December 16. I followed the instructions above and duplicated the issue you are seeing on Windows Server 2008 R2. I couldn't duplicate it on Windows XP. I need to debug it further but it looks like we are just aren't getting any notification from Windows to indicate that watch on subdir is no longer valid. At step 3 the event you are seeing is for directory watchtest to say that an entry has been deleted. I'll send mail once I've had time to debug it further. -Alan. From wolfgang.baltes at laposte.net Mon Jan 10 09:49:32 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Mon, 10 Jan 2011 09:49:32 -0800 Subject: WatchService In-Reply-To: <4D2B1FAA.40906@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> Message-ID: <4D2B46AC.9020001@laposte.net> On 2011-01-10 07:03, Alan Bateman wrote: > > Wolfgang Baltes wrote: >> >> Hi Alan, >> >> I took the WatchDir sample code and modified it as follows: >> - The directory parsing and event loop runs in a separate thread, >> which allows me to delete files and directories programmatically. >> - Added a few extra console outputs to better catch what is going on. >> - In the event loop, added a section to catch the particular >> condition in which we are interested. >> >> The bottom line of what I found is that the behavior changes >> depending on whether the deleted subdirectory is empty or not. To >> reproduce: >> >> 1: Change the code to create a parent directory to your liking. This >> is in line 241: final Path dir = Paths.get("c:\\watchtest"); >> >> 2: Run the code without other modification. It will create the before >> mentioned directory, plus a subdirectory, plus two files in that >> subdirectory, then waits for something to happen. >> >> 3: Manually delete "subdir". The output is: >> Scanning c:\watchtest ... >> register: c:\watchtest >> register: c:\watchtest\subdir >> Done with registering directories. >> ENTRY_DELETE: c:\watchtest\subdir >> c:\watchtest\subdir does not exist... ... but key is valid. >> ENTRY_MODIFY: c:\watchtest\subdir\file1.txt >> ENTRY_MODIFY: c:\watchtest\subdir\file2.txt >> >> 4: Kill the thread and start over. >> >> 5: Manually delete first the two files in subdir, and then subdir. >> The output is: >> Scanning c:\watchtest ... >> register: c:\watchtest >> register: c:\watchtest\subdir >> Done with registering directories. >> ENTRY_DELETE: c:\watchtest\subdir\file1.txt >> ENTRY_MODIFY: c:\watchtest\subdir >> ENTRY_DELETE: c:\watchtest\subdir\file2.txt >> ENTRY_MODIFY: c:\watchtest\subdir >> Removing watch key for c:\watchtest\subdir. >> ENTRY_DELETE: c:\watchtest\subdir >> >> 6: You can kill the watch thread by deleting directory "watchtest". >> >> 7: You can remove the comments from lines 265 to 273 to automate step 5. >> >> For me it is important to make step 3 above work correctly. This is >> what a user would do when a directory needs to be removed manually. >> >> I hope this helps. Please let me know how I can be of further help. >> Wolfgang. > Continuing the thread from December 16. I followed the instructions > above and duplicated the issue you are seeing on Windows Server 2008 > R2. I couldn't duplicate it on Windows XP. I need to debug it further > but it looks like we are just aren't getting any notification from > Windows to indicate that watch on subdir is no longer valid. At step 3 > the event you are seeing is for directory watchtest to say that an > entry has been deleted. I'll send mail once I've had time to debug it > further. > > -Alan. > > Hi Alan, thanks for the update. I look forward to hearing more from you. As a reminder, my system is Windows 7 64 bit, running Java 32 bit. I would like to take this opportunity to provide some feedback on some of the NIO2 classes. As I am using them quite a lot in the program I am working on, I have a short wish list. I have seen emails talking about some of the items, but did not see a conclusion: 1: WatchEvent.context(): it would be nice to get a complete path, not just a relative path. I can always build the relative path from a complete path, but the other way around forces me to store the parent path and do some extra work. /Complete path/ could be the concatenation of original or real (as in toRealPath) parent path that was used to obtain the WatchKey, and the name of the directory or file which caused the event. The original parent would probably be better because it maintains the information that was put into the system and one can always convert it to the real path. 2: WatchService class: provide a isClosed() method. This is convenient when multiple threads need to share this knowledge. It is simpler than having to catch the ClosedWatchServiceException in one thread and then to propagate it to other threads. All other methods that throw this exception have some side-effect. 3: Path class: 3.1: It would be nice to have a getExtension() method. 3.2: Same for getNameWithoutExtension() method. 4: Path or Paths class: provide a static method that returns an empty Path array. See Item 43 in Joshua Bloch, Effective Java, Second Edition, page 201. I use this systematically, and am frustrated that I have to redefine empty arrays again and again, instead of having them in the base library. 5: It would be nice to have some predefined Path instances, such as the "." path (= current path), and possibly a "null" path. The first acts as an identity element (as in mathematical groups), because its effect is neutral to all relevant Path operations. The second is for use in Path collections which do not accept null. I understand that this is a little bit tricky if you want this instance to have special behavior in regard to some methods. Another comment is relative to the documentation: almost all NIO methods throw IOException. That's a pretty large group of exceptions, many of which can probably appear at any time, but only a few of which have particular meaning in a specific context. For example, the watch events appear much faster than the file system finishes writing a file. The documentation says nothing about which exception I should expect if I try to access the file to soon (FileSystemException). I think that providing this type of information would help a lot of people. Thanks, Wolfgang. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110110/a852dc00/attachment.html From Alan.Bateman at oracle.com Mon Jan 10 10:36:42 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 10 Jan 2011 18:36:42 +0000 Subject: WatchService In-Reply-To: <4D2B46AC.9020001@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> Message-ID: <4D2B51BA.10000@oracle.com> Wolfgang Baltes wrote: > : > 1: WatchEvent.context(): it would be nice to get a complete path, not > just a relative path. I can always build the relative path from a > complete path, but the other way around forces me to store the parent > path and do some extra work. /Complete path/ could be the > concatenation of original or real (as in toRealPath) parent path that > was used to obtain the WatchKey, and the name of the directory or file > which caused the event. The original parent would probably be better > because it maintains the information that was put into the system and > one can always convert it to the real path. One of the things that has come up a few times is that WatchKey doesn't define a method to get the original Watchable. We'd need to set some expectations around that as there are implementations where the watch is on the inode and isn't tied to the original path but that aside, it would mean that you can resolve the file name against the original watch and that would give you the path that you want (and avoid needing to maintain the mapping yourself). > > 2: WatchService class: provide a isClosed() method. This is convenient > when multiple threads need to share this knowledge. It is simpler than > having to catch the ClosedWatchServiceException in one thread and then > to propagate it to other threads. All other methods that throw this > exception have some side-effect. > > 3: Path class: > 3.1: It would be nice to have a getExtension() method. > 3.2: Same for getNameWithoutExtension() method. > > 4: Path or Paths class: provide a static method that returns an empty > Path array. See Item 43 in Joshua Bloch, Effective Java, Second > Edition, page 201. I use this systematically, and am frustrated that I > have to redefine empty arrays again and again, instead of having them > in the base library. I can't think of anywhere in the API where we return null instead of an empty array or empty collection, but there are a couple of methods (getRoot, getName and getParent specifically) that return null when the path doesn't have the component - is that what you are asking about? > > 5: It would be nice to have some predefined Path instances, such as > the "." path (= current path), and possibly a "null" path. The first > acts as an identity element (as in mathematical groups), because its > effect is neutral to all relevant Path operations. The second is for > use in Path collections which do not accept null. I understand that > this is a little bit tricky if you want this instance to have special > behavior in regard to some methods. > > Another comment is relative to the documentation: almost all NIO > methods throw IOException. That's a pretty large group of exceptions, > many of which can probably appear at any time, but only a few of which > have particular meaning in a specific context. For example, the watch > events appear much faster than the file system finishes writing a > file. The documentation says nothing about which exception I should > expect if I try to access the file to soon (FileSystemException). I > think that providing this type of information would help a lot of people. The API defines several specific exceptions for specific classes where it may be useful to handle/recover. These are marked as "optional" and you'll see them in the throws list of several methods (the reason they are "optional" is that it's not always possible for implementations to distinguish specific errors from other errors). For this specific case, I guess you are looking for additional wording to make it clear that when you receive an event to indicate that a directory entry has been modified that there is no guarantee that whatever was changing the file has completed - is that right? -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110110/8d32b9af/attachment-0001.html From wolfgang.baltes at laposte.net Mon Jan 10 13:48:24 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Mon, 10 Jan 2011 13:48:24 -0800 Subject: WatchService In-Reply-To: <4D2B51BA.10000@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> Message-ID: <4D2B7EA8.2040708@laposte.net> On 2011-01-10 10:36, Alan Bateman wrote: > Wolfgang Baltes wrote: >> : >> 1: WatchEvent.context(): it would be nice to get a complete path, not >> just a relative path. I can always build the relative path from a >> complete path, but the other way around forces me to store the parent >> path and do some extra work. /Complete path/ could be the >> concatenation of original or real (as in toRealPath) parent path that >> was used to obtain the WatchKey, and the name of the directory or >> file which caused the event. The original parent would probably be >> better because it maintains the information that was put into the >> system and one can always convert it to the real path. > One of the things that has come up a few times is that WatchKey > doesn't define a method to get the original Watchable. We'd need to > set some expectations around that as there are implementations where > the watch is on the inode and isn't tied to the original path but that > aside, it would mean that you can resolve the file name against the > original watch and that would give you the path that you want (and > avoid needing to maintain the mapping yourself). > This would be fine with me. >> >> 2: WatchService class: provide a isClosed() method. This is >> convenient when multiple threads need to share this knowledge. It is >> simpler than having to catch the ClosedWatchServiceException in one >> thread and then to propagate it to other threads. All other methods >> that throw this exception have some side-effect. >> >> 3: Path class: >> 3.1: It would be nice to have a getExtension() method. >> 3.2: Same for getNameWithoutExtension() method. >> >> 4: Path or Paths class: provide a static method that returns an empty >> Path array. See Item 43 in Joshua Bloch, Effective Java, Second >> Edition, page 201. I use this systematically, and am frustrated that >> I have to redefine empty arrays again and again, instead of having >> them in the base library. > I can't think of anywhere in the API where we return null instead of > an empty array or empty collection, but there are a couple of methods > (getRoot, getName and getParent specifically) that return null when > the path doesn't have the component - is that what you are asking about? > My comments in item 4 were not specific to the NIO2 API, but are more general. For example, if somebody writes some code where he would return an empty array, then he has to define this array. If you write a collection of packages, you end up defining this array again and again (unless you have some "core" package). The idea here is to offer this feature in the base library, so that others don't have to create it. It may not be used in the NIO2 library itself. Of the NIO2 classes I have worked with, Path is the only one where this is likely to be of some help. The other class would be WatchKey, but this is more likely to be used in collections, for which there are already "empty" versions. However, related to item 5 below, instead of returning null from getRoot and getParent, one could return the EMPTY_PATH path (= Path with zero elements; but not root), which could then be used in a normal execution path, when null would require special code. But then, I am not sure this is even possible with the current implementation as the root component "/" already has some characteristics of an empty path (zero length and null name) and this may already be exploited by your code, and it does not have a neutral behavior in most methods. >> 5: It would be nice to have some predefined Path instances, such as >> the "." path (= current path), and possibly a "null" path. The first >> acts as an identity element (as in mathematical groups), because its >> effect is neutral to all relevant Path operations. The second is for >> use in Path collections which do not accept null. I understand that >> this is a little bit tricky if you want this instance to have special >> behavior in regard to some methods. >> >> Another comment is relative to the documentation: almost all NIO >> methods throw IOException. That's a pretty large group of exceptions, >> many of which can probably appear at any time, but only a few of >> which have particular meaning in a specific context. For example, the >> watch events appear much faster than the file system finishes writing >> a file. The documentation says nothing about which exception I should >> expect if I try to access the file to soon (FileSystemException). I >> think that providing this type of information would help a lot of people. > The API defines several specific exceptions for specific classes where > it may be useful to handle/recover. These are marked as "optional" and > you'll see them in the throws list of several methods (the reason they > are "optional" is that it's not always possible for implementations to > distinguish specific errors from other errors). > > For this specific case, I guess you are looking for additional wording > to make it clear that when you receive an event to indicate that a > directory entry has been modified that there is no guarantee that > whatever was changing the file has completed - is that right? > Yes! For example, the Javadoc for the WatchService says that "When an event is reported to indicate that a file in a watched directory has been modified then there is no guarantee that the program (or programs) that have modified the file have completed." However, there is no hint what to look for as an indicator to assure/wait for completion. When I started to work with the WatchService, I had to figure out which I/O exceptions were specific to this case, and then ended up catching FileSystemException as shown in this example where I try to retrieve a path's attributes: BasicFileAttributes attrs = null; for (;;) { try { attrs = Attributes.readBasicFileAttributes(path, LinkOption.NOFOLLOW_LINKS); return attrs; } catch (final FileSystemException fse) { if (path.notExists()) { throw fse; } } } I chose to catch FileSystemException (instead of IOException) because it seems to be broad enough to catch the cause for non-completion and then try to recover unless the path nonExists(), but not too broad to cause endless loops. All other possible exceptions are propagated to the calling code which can then decide whether there is a possibility to recover at a higher level. In summary, the above mentioned Javadoc could/should alert the programmer to expect FileSystemException (and maybe others) to indicate non-completion. And yes, I have noticed that some classes use very specific exceptions to indicate specific error conditions. > > -Alan Wolfgang. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110110/5929c47b/attachment.html From Alan.Bateman at oracle.com Tue Jan 11 04:12:28 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jan 2011 12:12:28 +0000 Subject: WatchService In-Reply-To: <4D2B7EA8.2040708@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> Message-ID: <4D2C492C.4060401@oracle.com> Wolfgang Baltes wrote: > My comments in item 4 were not specific to the NIO2 API, but are more > general. For example, if somebody writes some code where he would > return an empty array, then he has to define this array. If you write > a collection of packages, you end up defining this array again and > again (unless you have some "core" package). The idea here is to offer > this feature in the base library, so that others don't have to create > it. It may not be used in the NIO2 library itself. Of the NIO2 classes > I have worked with, Path is the only one where this is likely to be of > some help. The other class would be WatchKey, but this is more likely > to be used in collections, for which there are already "empty" versions. > > However, related to item 5 below, instead of returning null from > getRoot and getParent, one could return the EMPTY_PATH path (= Path > with zero elements; but not root), which could then be used in a > normal execution path, when null would require special code. But then, > I am not sure this is even possible with the current implementation as > the root component "/" already has some characteristics of an empty > path (zero length and null name) and this may already be exploited by > your code, and it does not have a neutral behavior in most methods. I don't think we have any assumptions in the code and a root directory ("/" or "C:\" or "\\server\share" for example) is just considered a path. The issue with an empty path is that there would be one per provider and it would create a couple of anomalies in API that would have to be worked out. However, it is worth going through again. > Yes! For example, the Javadoc for the WatchService says that "When an > event is reported to indicate that a file in a watched directory has > been modified then there is no guarantee that the program (or > programs) that have modified the file have completed." However, there > is no hint what to look for as an indicator to assure/wait for > completion. The next statement is "Care should be taken to coordinate access with other programs that may be updating the file" and it gives a suggestion to use file locking. Another suggestion that we could include is to atomically move files into place. Ideally we would have an event like CLOSE_AFTER_MODIFY (or something like that) but that isn't implementable by polling or isn't something that is widely supported by the native event notification mechanisms either. So I've been looking further at the issue that you originally reported and I'm wondering if you've only tried to delete the file tree with something other than Windows Explorer. I suspect the issue is that Windows Explorer is trying to move the directories into the Trash directory and it's getting a sharing violation (because Windows doesn't allow you to delete a directory if it contains a sub-directory that is being watched). -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110111/3a529493/attachment.html From Alan.Bateman at oracle.com Tue Jan 11 13:12:43 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jan 2011 21:12:43 +0000 Subject: Bringing all file operations together into one place Message-ID: <4D2CC7CB.7090301@oracle.com> One of the things that I've been looking at recently is the file system API and to see how it might evolve in the future, post jdk7. One confusing thing is that the file operations are currently split over several classes. At it stands, if a file operation requires new support from the provider implementation then it is added to the Path class, if it is independent of the provider it can be defined by the Files class, file attribute helper methods are defined by the Attributes class. This needs to be cleaned up so that it's clear and obvious where to add new file operations in the future. To that end, I have an update to the API [1] that moves all the file operations into the Files class and removes the Attributes class. For the most part, the methods defined by Files just delegate to the FileSystem associated with the Path or are implemented in terms of other methods. The alternative moves everything to Path but that is much less satisfactory and results in a huge class of path and file operations. Another advantage of having Path only define path methods is that it opens the possibility of look at interfaces again, esp. with extension methods potentially coming in 8. There are a couple of other small updates too. The readXXXFileAttributes methods to read attributes in bulk have been replaced with a single readAttributes method that takes a type token to specify the required attributes - this is something that Joe Darcy suggested. There are also a few new convenience methods for simple/common cases. Feedback welcome! -Alan [1] http://openjdk.java.net/projects/nio/javadoc From david.lloyd at redhat.com Tue Jan 11 13:20:57 2011 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 11 Jan 2011 15:20:57 -0600 Subject: Bringing all file operations together into one place In-Reply-To: <4D2CC7CB.7090301@oracle.com> References: <4D2CC7CB.7090301@oracle.com> Message-ID: <4D2CC9B9.2040806@redhat.com> On 01/11/2011 03:12 PM, Alan Bateman wrote: > [...] To that end, I have an update to the API [1] [...] > > [1] http://openjdk.java.net/projects/nio/javadoc Looks like there's no CSS or something? I only get raw HTML so it's a bit tough to read :) -- - DML From Alan.Bateman at oracle.com Tue Jan 11 13:28:39 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Jan 2011 21:28:39 +0000 Subject: Bringing all file operations together into one place In-Reply-To: <4D2CC9B9.2040806@redhat.com> References: <4D2CC7CB.7090301@oracle.com> <4D2CC9B9.2040806@redhat.com> Message-ID: <4D2CCB87.6030104@oracle.com> David M. Lloyd wrote: > : > Looks like there's no CSS or something? I only get raw HTML so it's a > bit tough to read :) There was a big javadoc update in b123 that included a new CSS. I just checked with Safari and Firefox and it works for me. Reload? -Alan. From david.lloyd at redhat.com Tue Jan 11 13:52:56 2011 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 11 Jan 2011 15:52:56 -0600 Subject: Bringing all file operations together into one place In-Reply-To: <4D2CCB87.6030104@oracle.com> References: <4D2CC7CB.7090301@oracle.com> <4D2CC9B9.2040806@redhat.com> <4D2CCB87.6030104@oracle.com> Message-ID: <4D2CD138.80501@redhat.com> On 01/11/2011 03:28 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> : >> Looks like there's no CSS or something? I only get raw HTML so it's a >> bit tough to read :) > There was a big javadoc update in b123 that included a new CSS. I just > checked with Safari and Firefox and it works for me. Reload? > > -Alan. OK, for some reason Chrome was caching or something. Got it working in any case. -- - DML From wolfgang.baltes at laposte.net Tue Jan 11 14:41:54 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Tue, 11 Jan 2011 14:41:54 -0800 Subject: WatchService In-Reply-To: <4D2C492C.4060401@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> Message-ID: <4D2CDCB2.8050607@laposte.net> On 2011-01-11 04:12, Alan Bateman wrote: > So I've been looking further at the issue that you originally reported > and I'm wondering if you've only tried to delete the file tree with > something other than Windows Explorer. I suspect the issue is that > Windows Explorer is trying to move the directories into the Trash > directory and it's getting a sharing violation (because Windows > doesn't allow you to delete a directory if it contains a sub-directory > that is being watched). > > -Alan. > Alan, I have run the same code as sent before and repeated step 3, now using the Windows Command Prompt instead of Windows Explorer. The bottom line result is that the deletion of the directory is now reported correctly. But read on to see some of the details. To reproduce: - Open a Windows Command Prompt window. - Make c:\watchtest the current directory. - Execute "rmdir /s /q subdir". The console output is: Scanning C:\watchtest ... register: C:\watchtest register: C:\watchtest\subdir Done with registering directories. ENTRY_DELETE: C:\watchtest\subdir\file1.txt ENTRY_DELETE: C:\watchtest\subdir\file2.txt Removing watch key for C:\watchtest\subdir. ENTRY_MODIFY: C:\watchtest\subdir ENTRY_DELETE: C:\watchtest\subdir In this case everything works as expected, because this command iterates from the bottom of the file tree up. Running this example multiple times, I observe that sometimes one of the file deletions (mostly file2.txt) is not reported, probably due to the fact that the watch key was canceled!? I inserted these code lines behind line 256 (creation of file2): final Path subsubdir = subdir.resolve(Paths.get("subsubdir")); subsubdir.createDirectory(); final Path file3 = subsubdir.resolve(Paths.get("file3.txt")); file3.createFile(); final Path file4 = subsubdir.resolve(Paths.get("file4.txt")); file4.createFile(); These lines create an additional subdirectory and two files in it. The same experiment then sometimes gives: Scanning C:\watchtest ... register: C:\watchtest register: C:\watchtest\subdir register: C:\watchtest\subdir\subsubdir Done with registering directories. ENTRY_DELETE: C:\watchtest\subdir\file1.txt ENTRY_MODIFY: C:\watchtest\subdir ENTRY_DELETE: C:\watchtest\subdir\subsubdir\file3.txt Removing watch key for C:\watchtest\subdir\subsubdir. ENTRY_DELETE: C:\watchtest\subdir\file2.txt ENTRY_MODIFY: C:\watchtest\subdir\SUBSUB~1 ENTRY_DELETE: C:\watchtest\subdir\subsubdir and sometimes: Scanning S:\watchtest ... register: S:\watchtest register: S:\watchtest\subdir register: S:\watchtest\subdir\subsubdir Done with registering directories. ENTRY_DELETE: S:\watchtest\subdir\file1.txt ENTRY_DELETE: S:\watchtest\subdir\file2.txt ENTRY_MODIFY: S:\watchtest\subdir\SUBSUB~1 ENTRY_DELETE: S:\watchtest\subdir\subsubdir S:\watchtest\subdir\subsubdir does not exist... ... and key is canceled. ENTRY_MODIFY: S:\watchtest\subdir ENTRY_DELETE: S:\watchtest\subdir\subsubdir\file3.txt Removing watch key for S:\watchtest\subdir\subsubdir. There are several observations to be made: - The rmdir command iterates from the bottom of the tree up. - Sometimes, the deletion of file3 is reported after its directory subsubdir is effectively deleted. - Most of the time, the deletion of file4 is not reported, probably because the watch key is canceled. - The parent of the watched directory subsubdir is not deleted (directory subdir remains after the code has finished). This is the bug you mentioned in an earlier message. - For ENTRY_MODIFY, the 8.3 form of the directory name is used instead of the long form. Of all the tests done so far, this is the first where a name is long enough for this to happen. => This prompts me to wonder whether there is a method in Path that allows to get the long form, given the 8.3 form, and conversely the 8.3 form given the long form? - The order of the events may be different due to each directory reporting through a different thread? I hope this helps, Wolfgang. From wolfgang.baltes at laposte.net Tue Jan 11 16:03:00 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Tue, 11 Jan 2011 16:03:00 -0800 Subject: Bringing all file operations together into one place In-Reply-To: <4D2CC7CB.7090301@oracle.com> References: <4D2CC7CB.7090301@oracle.com> Message-ID: <4D2CEFB4.9020406@laposte.net> On 2011-01-11 13:12, Alan Bateman wrote: > > > One of the things that I've been looking at recently is the file > system API and to see how it might evolve in the future, post jdk7. > One confusing thing is that the file operations are currently split > over several classes. At it stands, if a file operation requires new > support from the provider implementation then it is added to the Path > class, if it is independent of the provider it can be defined by the > Files class, file attribute helper methods are defined by the > Attributes class. This needs to be cleaned up so that it's clear and > obvious where to add new file operations in the future. To that end, I > have an update to the API [1] that moves all the file operations into > the Files class and removes the Attributes class. For the most part, > the methods defined by Files just delegate to the FileSystem > associated with the Path or are implemented in terms of other methods. > The alternative moves everything to Path but that is much less > satisfactory and results in a huge class of path and file operations. > Another advantage of having Path only define path methods is that it > opens the possibility of look at interfaces again, esp. with extension > methods potentially coming in 8. > > There are a couple of other small updates too. The > readXXXFileAttributes methods to read attributes in bulk have been > replaced with a single readAttributes method that takes a type token > to specify the required attributes - this is something that Joe Darcy > suggested. There are also a few new convenience methods for > simple/common cases. > > Feedback welcome! > > -Alan > > [1] http://openjdk.java.net/projects/nio/javadoc > > For what its worth, my opinion is that this is a good change. Besides your arguments with which I agree, I observe more consistency as a result of the change. Maybe this wasn't the purpose, but I like the outcome that because almost all methods of the new Path interface deal with the abstract path concept, they don't require physical access authority, nor will they throw IOException or SecurityException (there are a few exceptions such as toURI, toAbsolutePath, toRealPath, and register). Conversely, all Files methods deal with physical access, and therefore require authority, and possibly throw the exceptions mentioned. Wolfgang. From youcangetme at yahoo.com Tue Jan 11 20:54:01 2011 From: youcangetme at yahoo.com (John Bryant) Date: Tue, 11 Jan 2011 20:54:01 -0800 (PST) Subject: Java 7 FileSystem Providers as SPI questions Message-ID: <129284.65328.qm@web113801.mail.gq1.yahoo.com> Hello! I understand that FileSystem providers are extensions to the core as SPIs. I would like to write one for a project (its an elegant solution to a problem I have) but there is almost no examples or tutorials out on the Web. I saw one that was incomplete on a Zip file system provider but I need a little more help. I did manage to find the JAR containing the classes for the 'zipfs' and an online reference to the source code but its kind of muddled seeing as there isn't a giant sign reading, "Hey, start here! This is what you need!" It would be a big helped if someone knew the magic, "Start Here" and the "this is what you need to extend/implement" to get the ball rolling. Anyone willing to point me in that direction? thx, JB -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110111/2ea9e6c7/attachment.html From youcangetme at yahoo.com Tue Jan 11 21:08:05 2011 From: youcangetme at yahoo.com (John Bryant) Date: Tue, 11 Jan 2011 21:08:05 -0800 (PST) Subject: Java 7 FileSystem Providers as SPI questions In-Reply-To: <129284.65328.qm@web113801.mail.gq1.yahoo.com> References: <129284.65328.qm@web113801.mail.gq1.yahoo.com> Message-ID: <119325.40051.qm@web113811.mail.gq1.yahoo.com> Oh, and I forgot to point out that the zipfs demos talked about on this list all bring back 404s. I'm starting to get a little depressed trying to nail down an example of this wondrous new thing! Am I chasing ghosts? Thanks for your help! JB -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110111/cd6839e0/attachment.html From xueming.shen at oracle.com Tue Jan 11 21:39:50 2011 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 11 Jan 2011 21:39:50 -0800 Subject: Java 7 FileSystem Providers as SPI questions In-Reply-To: <119325.40051.qm@web113811.mail.gq1.yahoo.com> References: <129284.65328.qm@web113801.mail.gq1.yahoo.com> <119325.40051.qm@web113811.mail.gq1.yahoo.com> Message-ID: <4D2D3EA6.5090006@oracle.com> On 11/01/11 21:08, John Bryant wrote: > Oh, and I forgot to point out that the zipfs demos talked about on > this list all bring back 404s. I'm starting to get a little depressed > trying to nail down an example of this wondrous new thing! Am I > chasing ghosts? > Thanks for your help! > > > JB John, If you can download the latest jdk7 binary from http://download.java.net/openjdk/jdk7 you should be able to find the zipfs demo at demo/nio/zipfs (Demo.java, src.zip and the binary zipfs.jar) -Sherman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110111/2d6013c2/attachment-0001.html From Alan.Bateman at oracle.com Wed Jan 12 02:43:32 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Jan 2011 10:43:32 +0000 Subject: WatchService In-Reply-To: <4D2CDCB2.8050607@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> Message-ID: <4D2D85D4.6040805@oracle.com> Wolfgang Baltes wrote: > > Alan, > > I have run the same code as sent before and repeated step 3, now using > the Windows Command Prompt instead of Windows Explorer. The bottom > line result is that the deletion of the directory is now reported > correctly. Thanks for checking this. When you use rmdir then it really deletes the directory and so the corresponding watch key will be automatically canceled and queued so that you know that it is no longer watched. I think Windows Explorer doesn't actually attempt a delete but instead attempts to move the directory into the "Recycle Bin" directory. My guess is that it gets a sharing violation because we have an open handle to the directory and then falls back to a deferred/delayed move - this is just a guess but it would explain why we don't get the event. > But read on to see some of the details. To reproduce: > > - Open a Windows Command Prompt window. > - Make c:\watchtest the current directory. > - Execute "rmdir /s /q subdir". > > The console output is: > Scanning C:\watchtest ... > register: C:\watchtest > register: C:\watchtest\subdir > Done with registering directories. > ENTRY_DELETE: C:\watchtest\subdir\file1.txt > ENTRY_DELETE: C:\watchtest\subdir\file2.txt > Removing watch key for C:\watchtest\subdir. > ENTRY_MODIFY: C:\watchtest\subdir > ENTRY_DELETE: C:\watchtest\subdir > > In this case everything works as expected, because this command > iterates from the bottom of the file tree up. Running this example > multiple times, I observe that sometimes one of the file deletions > (mostly file2.txt) is not reported, probably due to the fact that the > watch key was canceled!? I don't think we can do anything about this. When you delete a watched directory on Windows then you will sometimes get events to indicate that the entries in the directory were deleted, sometimes you don't. There's lot of wriggle room in the WatchService spec to cover us for these implementation specific differences. > > : > > There are several observations to be made: > - The rmdir command iterates from the bottom of the tree up. > - Sometimes, the deletion of file3 is reported after its directory > subsubdir is effectively deleted. > - Most of the time, the deletion of file4 is not reported, probably > because the watch key is canceled. > - The parent of the watched directory subsubdir is not deleted > (directory subdir remains after the code has finished). This is the > bug you mentioned in an earlier message. Yes, it's because there is an open handle to each directory that you register with the WatchService. It doesn't prevent the directory from being deleted but we need the notification to trigger the closing of the handle and so there is a tiny window where it is possible to delete the parent directory after deleting the sub-directory. We have in the bug database but I don't think we can do anything about it. One thing I should have mentioned before is that our Windows WatchService implementation supports an implementation specific modifier that allows you to monitor a file tree with a single registration. If you want to try it out do this: register(watcher, new WatchEvent.Kind[]{ ENTRY_DELETE }, com.sun.nio.file.ExtendedWatchEventModifier.FILE_TREE); With this modifier then you only register the top directory and the context for each event is the relative path from the top directory. It's not a portable solution and only works on Windows but it would be interesting to how it compares. > - For ENTRY_MODIFY, the 8.3 form of the directory name is used instead > of the long form. Of all the tests done so far, this is the first > where a name is long enough for this to happen. => This prompts me to > wonder whether there is a method in Path that allows to get the long > form, given the 8.3 form, and conversely the 8.3 form given the long > form? That is a good point as our implementation does not convert events for 8.3 names into long names (which might be problematic with delete events). I'm curious how you generated these events. Were you in CMD or COMMAND or maybe you monitored the file using the 8.3 form explicitly? > - The order of the events may be different due to each directory > reporting through a different thread? The implementation preserves the order of the create and delete events that we get the events from Windows. -Alan. From Alan.Bateman at oracle.com Wed Jan 12 02:57:14 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Jan 2011 10:57:14 +0000 Subject: Java 7 FileSystem Providers as SPI questions In-Reply-To: <129284.65328.qm@web113801.mail.gq1.yahoo.com> References: <129284.65328.qm@web113801.mail.gq1.yahoo.com> Message-ID: <4D2D890A.3010006@oracle.com> John Bryant wrote: > Hello! > > I understand that FileSystem providers are extensions to the core as > SPIs. I would like to write one for a project (its an elegant solution > to a problem I have) but there is almost no examples or tutorials out > on the Web. I saw one that was incomplete on a Zip file system > provider but I need a little more help. I did manage to find the JAR > containing the classes for the 'zipfs' and an online reference to the > source code but its kind of muddled seeing as there isn't a giant sign > reading, "Hey, start here! This is what you need!" > > It would be a big helped if someone knew the magic, "Start Here" and > the "this is what you need to extend/implement" to get the ball rolling. > > Anyone willing to point me in that direction? > > > thx, > > > JB As Sherman said, the $JAVA_HOME/demo/nio/zipfs directory is where you'll find the zip provider. There's a README in that directory to get you started, and also a Demo class that allows you to copy files in/out of zip files and do various other operations. Note that this provider is a complete provider and this replaces the incomplete/read-only provider that we originally had to demonstrate the provider interface. One of docs engineer has recently created some material for the online tutorials which should also be useful to those starting out with the zip provider. That said, I suspect you are actually looking for material on how to develop your own provider. Sadly we are lacking in this regard so all I can suggest is to start with java.nio.file.spi.FileSystemProvider and use the zip provider as a reference (the complete source code to the provider is in demo/nio/zipfs/src.zip). Another thing to say is that developing a complete provider is a non-trivial task and much more work compared to say developing a URL protocol handler. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110112/95a64ee2/attachment.html From youcangetme at yahoo.com Wed Jan 12 04:45:52 2011 From: youcangetme at yahoo.com (John Bryant) Date: Wed, 12 Jan 2011 04:45:52 -0800 (PST) Subject: Java 7 FileSystem Providers as SPI questions In-Reply-To: <4D2D890A.3010006@oracle.com> References: <129284.65328.qm@web113801.mail.gq1.yahoo.com> <4D2D890A.3010006@oracle.com> Message-ID: <75181.80998.qm@web113818.mail.gq1.yahoo.com> Alan and Sherman, Thank you! I found the demo and examples. I have been working with Java for a number of years and just quit paying attention to what actually came with the JDK. If my IDE didn't have it, Google it ;-) Again, thanks guys for your speedy help. Nothing like crying out for help, going to sleep all depressed. Then, waking up and skidding into work (ice everywhere here in Atlanta, GA) only to find bright and wonderful messages in one's inbox! jB-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110112/03376946/attachment.html From wolfgang.baltes at laposte.net Wed Jan 12 06:17:47 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Wed, 12 Jan 2011 06:17:47 -0800 Subject: WatchService In-Reply-To: <4D2D85D4.6040805@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> <4D2D85D4.6040805@oracle.com> Message-ID: <4D2DB80B.1090309@laposte.net> On 2011-01-12 02:43, Alan Bateman wrote: > > > One thing I should have mentioned before is that our Windows > WatchService implementation supports an implementation specific > modifier that allows you to monitor a file tree with a single > registration. If you want to try it out do this: > > register(watcher, > new WatchEvent.Kind[]{ ENTRY_DELETE }, > com.sun.nio.file.ExtendedWatchEventModifier.FILE_TREE); > > With this modifier then you only register the top directory and the > context for each event is the relative path from the top directory. > It's not a portable solution and only works on Windows but it would be > interesting to how it compares. > I'll try this out and let you know the result. >> - For ENTRY_MODIFY, the 8.3 form of the directory name is used >> instead of the long form. Of all the tests done so far, this is the >> first where a name is long enough for this to happen. => This prompts >> me to wonder whether there is a method in Path that allows to get the >> long form, given the 8.3 form, and conversely the 8.3 form given the >> long form? > That is a good point as our implementation does not convert events for > 8.3 names into long names (which might be problematic with delete > events). I'm curious how you generated these events. Were you in CMD > or COMMAND or maybe you monitored the file using the 8.3 form explicitly? > I just use the standard Command Prompt window; that is, %windir%\system32\cmd.exe. No special settings. And no, I did not use the 8.3 form at all. It just popped up and I thought that this was an interesting inconsistency because the ENTRY_DELETE line that follows uses the long name. Wolfgang. From Alan.Bateman at oracle.com Wed Jan 12 07:38:06 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Jan 2011 15:38:06 +0000 Subject: WatchService In-Reply-To: <4D2DB80B.1090309@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> <4D2D85D4.6040805@oracle.com> <4D2DB80B.1090309@laposte.net> Message-ID: <4D2DCADE.8080904@oracle.com> Wolfgang Baltes wrote: > > : > I just use the standard Command Prompt window; that is, > %windir%\system32\cmd.exe. No special settings. And no, I did not use > the 8.3 form at all. It just popped up and I thought that this was an > interesting inconsistency because the ENTRY_DELETE line that follows > uses the long name. I've created bug 7011822 to track this one. -Alan. From wolfgang.baltes at laposte.net Wed Jan 12 14:15:50 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Wed, 12 Jan 2011 14:15:50 -0800 Subject: WatchService In-Reply-To: <4D2DCADE.8080904@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> <4D2D85D4.6040805@oracle.com> <4D2DB80B.1090309@laposte.net> <4D2DCADE.8080904@oracle.com> Message-ID: <4D2E2816.9040204@laposte.net> On 2011-01-12 07:38, Alan Bateman wrote: > > Wolfgang Baltes wrote: >> >> : >> I just use the standard Command Prompt window; that is, >> %windir%\system32\cmd.exe. No special settings. And no, I did not use >> the 8.3 form at all. It just popped up and I thought that this was an >> interesting inconsistency because the ENTRY_DELETE line that follows >> uses the long name. > I've created bug 7011822 to track this one. > > -Alan. > > Thanks Alan. Wolfgang. From forax at univ-mlv.fr Wed Jan 12 15:41:26 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 13 Jan 2011 00:41:26 +0100 Subject: Bringing all file operations together into one place In-Reply-To: <4D2CC7CB.7090301@oracle.com> References: <4D2CC7CB.7090301@oracle.com> Message-ID: <4D2E3C26.40901@univ-mlv.fr> On 01/11/2011 10:12 PM, Alan Bateman wrote: > > One of the things that I've been looking at recently is the file > system API and to see how it might evolve in the future, post jdk7. > One confusing thing is that the file operations are currently split > over several classes. At it stands, if a file operation requires new > support from the provider implementation then it is added to the Path > class, if it is independent of the provider it can be defined by the > Files class, file attribute helper methods are defined by the > Attributes class. This needs to be cleaned up so that it's clear and > obvious where to add new file operations in the future. To that end, I > have an update to the API [1] that moves all the file operations into > the Files class and removes the Attributes class. For the most part, > the methods defined by Files just delegate to the FileSystem > associated with the Path or are implemented in terms of other methods. > The alternative moves everything to Path but that is much less > satisfactory and results in a huge class of path and file operations. > Another advantage of having Path only define path methods is that it > opens the possibility of look at interfaces again, esp. with extension > methods potentially coming in 8. > > There are a couple of other small updates too. The > readXXXFileAttributes methods to read attributes in bulk have been > replaced with a single readAttributes method that takes a type token > to specify the required attributes - this is something that Joe Darcy > suggested. There are also a few new convenience methods for > simple/common cases. > > Feedback welcome! > > -Alan > > [1] http://openjdk.java.net/projects/nio/javadoc First, I think this version of the API easier to use. Some comments about java.nio.file.Files: readAttributes(Path path,String attributes,LinkOption ... options) should returns a Map if possible. Map will let users deal with wildcard capture. As an exercise, try to find the type XXX in this code: XXX entrySet = readAttributes(path, attributes, options).entrySet(); readLines: why not returning a lazy Collection instead of a List. if a user want a List, this code List lines = new ArrayList<>(Files.readLines(...)) Or perhaps better, wait jdk 8 and the stream API. readBytes() can be renamed to readAllBytes(). write(...): javadoc should also contains an example showing how to create a new File: Files.write(path, bytes, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE); because I'm sure that users will forget TRUNCATE_EXISTING. R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110113/11b4c03b/attachment.html From Alan.Bateman at oracle.com Thu Jan 13 03:03:39 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Jan 2011 11:03:39 +0000 Subject: Bringing all file operations together into one place In-Reply-To: <4D2E3C26.40901@univ-mlv.fr> References: <4D2CC7CB.7090301@oracle.com> <4D2E3C26.40901@univ-mlv.fr> Message-ID: <4D2EDC0B.7030207@oracle.com> R?mi Forax wrote: > : > First, I think this version of the API easier to use. Thanks for spending time on it. > > Some comments about java.nio.file.Files: > readAttributes(Path path, String attributes, LinkOption ... options) > > > should returns a Map if possible. > Map will let users deal with wildcard capture. > As an exercise, try to find the type XXX in this code: > XXX entrySet = readAttributes(path, attributes, options).entrySet(); I thought that one was fixed a long time ago but it seems not. Thanks for bringing it up. I'll include this in this round of updates. > > readLines: why not returning a lazy Collection instead of a List. > if a user want a List, this code > List lines = new ArrayList<>(Files.readLines(...)) Like readBytes (or readAllBytes as you've suggested), this method reads the entire file so I didn't make it lazy. An Iterable might be better but I initially proposed it as List so that it could be used as such if needed. > Or perhaps better, wait jdk 8 and the stream API. > > readBytes() can be renamed to readAllBytes(). Maybe, it was originally readAllBytes way back. > > write(...): javadoc should also contains an example showing how to > create a new File: > Files.write(path, bytes, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE); > > because I'm sure that users will forget TRUNCATE_EXISTING. This is the default if you don't specify any options ("That is, if no options are present then file is created if it doesn't exist or is first truncated to a size of 0 if it exists"). The idea is that you shouldn't need to specify any ideas for the common case, ie: Files.write(path, bytes). Maybe I need to make that clearer in the docs. Thanks again (as always). -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110113/92a08c2a/attachment.html From forax at univ-mlv.fr Thu Jan 13 03:23:04 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 13 Jan 2011 12:23:04 +0100 Subject: Bringing all file operations together into one place In-Reply-To: <4D2EDC0B.7030207@oracle.com> References: <4D2CC7CB.7090301@oracle.com> <4D2E3C26.40901@univ-mlv.fr> <4D2EDC0B.7030207@oracle.com> Message-ID: <4D2EE098.7010609@univ-mlv.fr> On 01/13/2011 12:03 PM, Alan Bateman wrote: > R?mi Forax wrote: >> : >> First, I think this version of the API easier to use. > Thanks for spending time on it. [...] > >> >> readLines: why not returning a lazy Collection instead of a List. >> if a user want a List, this code >> List lines = new ArrayList<>(Files.readLines(...)) > Like readBytes (or readAllBytes as you've suggested), this method > reads the entire file so I didn't make it lazy. An Iterable > might be better but I initially proposed it as List so that it could > be used as such if needed. > > >> Or perhaps better, wait jdk 8 and the stream API. >> >> readBytes() can be renamed to readAllBytes(). > Maybe, it was originally readAllBytes way back. I think I prefer readAllBytes and readAllLines. And future will tell if lazy versions will be added later. > >> >> write(...): javadoc should also contains an example showing how to >> create a new File: >> Files.write(path, bytes, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE); >> >> because I'm sure that users will forget TRUNCATE_EXISTING. > This is the default if you don't specify any options ("That is, if no > options are present then file is created if it doesn't exist or is > first truncated to a size of 0 if it exists"). The idea is that you > shouldn't need to specify any ideas for the common case, ie: > Files.write(path, bytes). Maybe I need to make that clearer in the docs. Perhaps just something saying that this is equivalent to write(WRITE|CREATE|TRUNCATE_EXISTING) > > Thanks again (as always). > > -Alan > > cheers, R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110113/5f4d6717/attachment.html From Alan.Bateman at oracle.com Thu Jan 13 11:44:34 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Jan 2011 19:44:34 +0000 Subject: Bringing all file operations together into one place In-Reply-To: <4D2EE098.7010609@univ-mlv.fr> References: <4D2CC7CB.7090301@oracle.com> <4D2E3C26.40901@univ-mlv.fr> <4D2EDC0B.7030207@oracle.com> <4D2EE098.7010609@univ-mlv.fr> Message-ID: <4D2F5622.4020403@oracle.com> R?mi Forax wrote: > : > I think I prefer readAllBytes and readAllLines. And future will tell > if lazy versions will be added later. Make sense. > > : > Perhaps just something saying that this is equivalent to > write(WRITE|CREATE|TRUNCATE_EXISTING) > OK, I'll try to improve the wording to make this clear. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110113/3bb7875b/attachment-0001.html From wolfgang.baltes at laposte.net Mon Jan 17 15:54:58 2011 From: wolfgang.baltes at laposte.net (Wolfgang Baltes) Date: Mon, 17 Jan 2011 15:54:58 -0800 Subject: WatchService In-Reply-To: <4D2D85D4.6040805@oracle.com> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> <4D2D85D4.6040805@oracle.com> Message-ID: <4D34D6D2.7080407@laposte.net> On 2011-01-12 02:43, Alan Bateman wrote: > > Wolfgang Baltes wrote: > >> There are several observations to be made: >> - The rmdir command iterates from the bottom of the tree up. >> - Sometimes, the deletion of file3 is reported after its directory >> subsubdir is effectively deleted. >> - Most of the time, the deletion of file4 is not reported, probably >> because the watch key is canceled. >> - The parent of the watched directory subsubdir is not deleted >> (directory subdir remains after the code has finished). This is the >> bug you mentioned in an earlier message. > Yes, it's because there is an open handle to each directory that you > register with the WatchService. It doesn't prevent the directory from > being deleted but we need the notification to trigger the closing of > the handle and so there is a tiny window where it is possible to > delete the parent directory after deleting the sub-directory. We have > in the bug database but I don't think we can do anything about it. > > One thing I should have mentioned before is that our Windows > WatchService implementation supports an implementation specific > modifier that allows you to monitor a file tree with a single > registration. If you want to try it out do this: > > register(watcher, > new WatchEvent.Kind[]{ ENTRY_DELETE }, > com.sun.nio.file.ExtendedWatchEventModifier.FILE_TREE); > > With this modifier then you only register the top directory and the > context for each event is the relative path from the top directory. > It's not a portable solution and only works on Windows but it would be > interesting to how it compares. Hi Alan, thanks again for this tip. I modified the WatchDir program to use the suggested Windows-specific registration. This works as expected: when deleting a file tree using Windows Explorer, then I only get an event for the directory being removed (no event for any file or subdirectory contained therein). When using rmdir from the command prompt, then I get events for each item. When adding/creating a subtree, then I get events for all items. Will this modifier remain in the final version of JDK7, or is it only experimental and may get removed? Thanks, Wolfgang. From Alan.Bateman at oracle.com Thu Jan 20 05:59:59 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Jan 2011 13:59:59 +0000 Subject: WatchService In-Reply-To: <4D34D6D2.7080407@laposte.net> References: <1728174.485578.1292455140760.JavaMail.www@wwinf8212> <4D0A26C9.5000105@oracle.com> <4D0A57B3.3010108@laposte.net> <4D0A79EE.5000002@laposte.net> <4D2B1FAA.40906@oracle.com> <4D2B46AC.9020001@laposte.net> <4D2B51BA.10000@oracle.com> <4D2B7EA8.2040708@laposte.net> <4D2C492C.4060401@oracle.com> <4D2CDCB2.8050607@laposte.net> <4D2D85D4.6040805@oracle.com> <4D34D6D2.7080407@laposte.net> Message-ID: <4D383FDF.2040303@oracle.com> Wolfgang Baltes wrote: > > Hi Alan, > > thanks again for this tip. I modified the WatchDir program to use the > suggested Windows-specific registration. This works as expected: when > deleting a file tree using Windows Explorer, then I only get an event > for the directory being removed (no event for any file or subdirectory > contained therein). When using rmdir from the command prompt, then I > get events for each item. > > When adding/creating a subtree, then I get events for all items. > > Will this modifier remain in the final version of JDK7, or is it only > experimental and may get removed? Thanks for trying out that modifier. The reason it is there is to test out the ability to extend this part of the API with provider specific capabilities. My guess is that we'll need to look at this area some time (not jdk7 but maybe later) to see if it makes sense to add a standard modifier or some means to watch directories recursively. This may be insane in some environments but it is clear that some people are trying to use it this way. The hard part is dealing with errors and issues that arise on platforms where this isn't supported natively. I just mention this as one reason why this provider specific modifier could go away. -Alan From Alan.Bateman at oracle.com Thu Jan 20 07:06:28 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Jan 2011 15:06:28 +0000 Subject: 7006126: (fs) Updates to file system API (1/2011) Message-ID: <4D384F74.9090309@oracle.com> Just to follow-up on last week's mail [1]. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/7006126/ On first glance it might seem a lot but most of the changes is refactoring/moving the methods and so it looks more than it really is. In particular, the changes to AbstractFileSystem from L65 is mostly just moved from AbstractPath. The "new" code in UnixFileSystem from L400 onwards was moved from UnixPath. The "new" code in WindowsFileSystem from L315 onwards was moved from WindowsPath. In Files then the bulk of the "additions" were in Path previously with most of the new code at L2250 onwards. Partial reviews are welcome (just let me know which files/areas you have reviewed so that I can keep track). Thanks, Alan. [1] http://mail.openjdk.java.net/pipermail/nio-dev/2011-January/001174.html From chris.hegarty at oracle.com Thu Jan 27 08:28:20 2011 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 27 Jan 2011 16:28:20 +0000 Subject: 7006126: (fs) Updates to file system API (1/2011) In-Reply-To: <4D384F74.9090309@oracle.com> References: <4D384F74.9090309@oracle.com> Message-ID: <4D419D24.30308@oracle.com> Alan, I really like these changes. Files is a one stop shop for file operations, and should really simplify applications that use it. I also really like the convenience methods for common use cases. I've taken a walk through all of the source changes and given my comments in a private mail. Mostly small/trivial issues, typos, incorrect links, etc. Thanks for fixing these. I'm finished with reviewing this and give my approval. -Chris. On 20/01/2011 15:06, Alan Bateman wrote: > > Just to follow-up on last week's mail [1]. > > The webrev with the changes is here: > http://cr.openjdk.java.net/~alanb/7006126/ > > On first glance it might seem a lot but most of the changes is > refactoring/moving the methods and so it looks more than it really is. > In particular, the changes to AbstractFileSystem from L65 is mostly just > moved from AbstractPath. The "new" code in UnixFileSystem from L400 > onwards was moved from UnixPath. The "new" code in WindowsFileSystem > from L315 onwards was moved from WindowsPath. In Files then the bulk of > the "additions" were in Path previously with most of the new code at > L2250 onwards. > > Partial reviews are welcome (just let me know which files/areas you have > reviewed so that I can keep track). > > Thanks, > > Alan. > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2011-January/001174.html From Alan.Bateman at oracle.com Fri Jan 28 02:58:52 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Jan 2011 10:58:52 +0000 Subject: 7006126: (fs) Updates to file system API (1/2011) In-Reply-To: <4D419D24.30308@oracle.com> References: <4D384F74.9090309@oracle.com> <4D419D24.30308@oracle.com> Message-ID: <4D42A16C.8010100@oracle.com> Chris Hegarty wrote: > Alan, > > I really like these changes. Files is a one stop shop for file > operations, and should really simplify applications that use it. I > also really like the convenience methods for common use cases. > > I've taken a walk through all of the source changes and given my > comments in a private mail. Mostly small/trivial issues, typos, > incorrect links, etc. Thanks for fixing these. > > I'm finished with reviewing this and give my approval. > > -Chris. Thanks Chris, I realize this one was tedious. Sherman has also gone through this code and also found a few typos too. I'm going to try to get this into jdk7/tl today and I've created 7015391 to track updating the zip provider. -Alan. From Alan.Bateman at oracle.com Fri Jan 28 11:47:06 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Jan 2011 19:47:06 +0000 Subject: 7014794: (file) lookupPrincipalByGroupName fails to find NIS group Message-ID: <4D431D3A.4090902@oracle.com> The lookupPrincipalByGroupName method is currently failing with NIS groups that have a lot of members. There is also an issue going the other way (gid -> group) where the GroupPrincipal's getName or toString method returns the gid rather than the group name. The problem is that the implementation uses a buffer of size SC_GETGR_R_SIZE_MAX which is not sufficient for large groups. The fix bumps up the buffer size until it is sufficient for the size of the group entry. The webrev with the changes is here: http://cr.openjdk.java.net/~alanb/7014794/webrev/ Thanks, Alan. From Alan.Bateman at oracle.com Mon Jan 31 02:43:52 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 10:43:52 +0000 Subject: sendfile is broken on solaris with -d64? In-Reply-To: <4CF633AD.4030109@desy.de> References: <4CEFCD3C.1000506@desy.de> <4CEFD521.4050109@oracle.com> <4CF63220.5020404@desy.de> <4CF632B6.8090305@oracle.com> <4CF633AD.4030109@desy.de> Message-ID: <4D469268.8020500@oracle.com> Tigran Mkrtchyan wrote: > On 12/01/2010 12:34 PM, Alan Bateman wrote: >> Tigran Mkrtchyan wrote: >>> Hi Alan, >>> >>> do you have a ticket number? >>> >>> Thanks, >>> Tigran. >> Here it is: >> >> http://bugs.sun.com/view_bug.do?bug_id=7002957 >> >> I'll try to get this fixed in jdk7 soon. > > and what about jdk6 ? This is the current production version. Tigran - the fix for this one should be in jdk7-b128 which should be available later this week. Do you mind testing it in your environment once that build is available to ensure that you see the difference when running in 64-bit. I don't know if/when this will be fixed in Oracle's jdk6. -Alan From Alan.Bateman at oracle.com Mon Jan 31 02:39:27 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 10:39:27 +0000 Subject: Deleting symbolic links via the java.nio.file.Path API In-Reply-To: References: <4CAB334B.4000706@oracle.com> <4CB0B999.1030909@oracle.com> Message-ID: <4D46915F.6080707@oracle.com> fm wrote: > Sorry. I was being careless when I threw the checkAccess method into > my conversation...I didn't think that one through. You are right, > checkAccess does not seem to make sense for the symbolic links > themselves (as opposed to their targets). > > However, I believe my concern with respect to checking the existence > of symbolic links with the exists method is still reasonable. > > Just to provide another example. The following code throws a > FileAlreadyExistsException when somePath corresponds to a broken > symbolic link. > > try { > if (!somePath.exists()) // or use somePath.notExists() > somePath.createFile(); > } > catch (FileAlreadyExistsException e) { > System.err.println("File exists!?!"); > } > catch (IOException e) { } > > Being able to call something like > "somePath.exists(LinkOption.NOFOLLOW_LINKS)" would be helpful. > Just to follow up on this thread (from 10/2010). With the latest updates that were recently pushed then the exists method has the LinkOption[] parameter as discussed. This means that exists(path) will follow sym links, and exists(path, NOFOLLOW_LINKS) will not. Same thing for the isDirectory and isRegularFile methods. Thanks again for bringing this up. -Alan. From martijnverburg at gmail.com Mon Jan 31 07:30:33 2011 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 31 Jan 2011 15:30:33 +0000 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> Message-ID: Hi all, I've finally come back to this Chapter and am thinking about the various rough code samples I've written (running b127 on Windows XP SP3 32bit). My next code sample simply takes the reader through some of the file system api in order to explore some of the basics. import java.io.IOException; import java.nio.file.*; public class Listing_2_1 { public static void main(String[] args) throws IOException { try (FileSystem fileSystem = FileSystems.getDefault()) { Path timeSheetFile = fileSystem.getPath("C:/projects/timesheet.txt"); Path backupDir = fileSystem.getPath("H:/projects/"); Path backupFile = fileSystem.getPath("H:/projects/timesheet.txt"); timeSheetFile.checkAccess(AccessMode.READ); backupDir.checkAccess(AccessMode.WRITE); FileStore backupFileStore = backupDir.getFileStore(); if (!backupFileStore.type().equals("NTFS")) { throw new IOException("Suspicious storage type"); } CopyOption copyOptions = StandardCopyOption.REPLACE_EXISTING; timeSheetFile.copyTo(backupFile, copyOptions); } } } The code example is a pretty patsy one, but one gotcha in particular was noticed by earlier reviewers. The FileStore object's type is simply a String as opposed to a constant, enum or other defined type. This makes it very hard to rely on for decision making (as per the if statement I have above). I assume this is a limitation of being able to consistently detect the underlying storage type? Thanks again all. Martijn (@java7developer - twitter) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110131/812fdb45/attachment.html From forax at univ-mlv.fr Mon Jan 31 07:57:17 2011 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Mon, 31 Jan 2011 16:57:17 +0100 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> Message-ID: <4D46DBDD.8010603@univ-mlv.fr> Le 31/01/2011 16:30, Martijn Verburg a ?crit : > Hi all, > > I've finally come back to this Chapter and am thinking about the > various rough code samples I've written (running b127 on Windows XP > SP3 32bit). My next code sample simply takes the reader through some > of the file system api in order to explore some of the basics. > > import java.io.IOException; > import java.nio.file.*; > > public class Listing_2_1 > { > public static void main(String[] args) throws IOException > { > try (FileSystem fileSystem = FileSystems.getDefault()) > { > Path timeSheetFile > = fileSystem.getPath("C:/projects/timesheet.txt"); > Path backupDir = fileSystem.getPath("H:/projects/"); > Path backupFile > = fileSystem.getPath("H:/projects/timesheet.txt"); > > timeSheetFile.checkAccess(AccessMode.READ); > backupDir.checkAccess(AccessMode.WRITE); > > FileStore backupFileStore = backupDir.getFileStore(); > if (!backupFileStore.type().equals("NTFS")) > { > throw new IOException("Suspicious storage type"); > } > > CopyOption copyOptions = StandardCopyOption.REPLACE_EXISTING; > timeSheetFile.copyTo(backupFile, copyOptions); > } > } > } > > The code example is a pretty patsy one, but one gotcha in particular > was noticed by earlier reviewers. > > The FileStore object's type is simply a String as opposed to a > constant, enum or other defined type. This makes it very hard to rely > on for decision making (as per the if statement I have above). I > assume this is a limitation of being able to consistently detect the > underlying storage type? Yes, the problem is that the string can be easily misspelled. I wondering if it worth an API like this: interface FileStoreType { public String getName(); } public enum StandardFileStoreType implements FileStoreType { EXT3, EXT4, FAT32, NTFS; ... } public class FileStoreTypes { public static FileStoreType getFileStoreType(final String name) { // StandardFileStoreType.valueOf(name) // otherwise new FileStoreType() { // public String getName() { // return name; // } //}; } } In that case, the code can be rewrite to: backupFileStore.type().equals(StandardFileStoreType.NTFS)) { ... } > > Thanks again all. > Martijn > (@java7developer - twitter) > R?mi From gnu_andrew at member.fsf.org Mon Jan 31 08:05:46 2011 From: gnu_andrew at member.fsf.org (Dr Andrew John Hughes) Date: Mon, 31 Jan 2011 16:05:46 +0000 Subject: Questions on the NIO.2 API In-Reply-To: <4D46DBDD.8010603@univ-mlv.fr> References: <4CB354C4.4010605@oracle.com> <4D46DBDD.8010603@univ-mlv.fr> Message-ID: On 31 January 2011 15:57, R?mi Forax wrote: > ?Le 31/01/2011 16:30, Martijn Verburg a ?crit : >> >> Hi all, >> >> I've finally come back to this Chapter and am thinking about the various >> rough code samples I've written (running b127 on Windows XP SP3 32bit). ?My >> next code sample simply takes the reader through some of the file system api >> in order to explore some of the basics. >> >> import java.io.IOException; >> import java.nio.file.*; >> >> public class Listing_2_1 >> { >> ? ?public static void main(String[] args) throws IOException >> ? ?{ >> ? ? ? ?try (FileSystem fileSystem = FileSystems.getDefault()) >> ? ? ? ?{ >> ? ? ? ? ? ?Path timeSheetFile = >> fileSystem.getPath("C:/projects/timesheet.txt"); >> ? ? ? ? ? ?Path backupDir = fileSystem.getPath("H:/projects/"); >> ? ? ? ? ? ?Path backupFile = >> fileSystem.getPath("H:/projects/timesheet.txt"); >> >> ? ? ? ? ? ?timeSheetFile.checkAccess(AccessMode.READ); >> ? ? ? ? ? ?backupDir.checkAccess(AccessMode.WRITE); >> >> ? ? ? ? ? ?FileStore backupFileStore = backupDir.getFileStore(); >> ? ? ? ? ? ?if (!backupFileStore.type().equals("NTFS")) >> ? ? ? ? ? ?{ >> ? ? ? ? ? ? ? ?throw new IOException("Suspicious storage type"); >> ? ? ? ? ? ?} >> >> ? ? ? ? ? ?CopyOption copyOptions = StandardCopyOption.REPLACE_EXISTING; >> ? ? ? ? ? ?timeSheetFile.copyTo(backupFile, copyOptions); >> ? ? ? ?} >> ? ?} >> } >> >> The code example is a pretty patsy one, but one gotcha in particular was >> noticed by earlier reviewers. >> >> The FileStore object's type is simply a String as opposed to a constant, >> enum or other defined type. ?This makes it very hard to rely on for decision >> making (as per the if statement I have above). ?I assume this is a >> limitation of being able to consistently detect the underlying storage type? > > Yes, the problem is that the string can be easily misspelled. > I wondering if it worth an API like this: > > interface FileStoreType { > ?public String getName(); > } > > public enum StandardFileStoreType implements FileStoreType { > ?EXT3, EXT4, FAT32, NTFS; > ?... > } > > public class FileStoreTypes { > ?public static FileStoreType getFileStoreType(final String name) { > ? ? // StandardFileStoreType.valueOf(name) > ? ? // otherwise new FileStoreType() { > ? ? // ? public String getName() { > ? ? // ? ? ?return name; > ? ? // ? } > ? ? //}; > ?} > } > > In that case, the code can be rewrite to: > backupFileStore.type().equals(StandardFileStoreType.NTFS)) { ... } > >> >> Thanks again all. >> Martijn >> (@java7developer - twitter) >> > > R?mi > > > What happens if the file store uses a type not defined in the enum? It may be possible to misspell a String, but it's worse if the API is outdated before it even gets released because it only lists certain filesystems. -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA? 7927 142C 2591 94EF D9D8 From martijnverburg at gmail.com Mon Jan 31 08:34:02 2011 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 31 Jan 2011 16:34:02 +0000 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> <4D46DBDD.8010603@univ-mlv.fr> Message-ID: Hi all, On Mon, Jan 31, 2011 at 4:05 PM, Dr Andrew John Hughes < gnu_andrew at member.fsf.org> wrote: > On 31 January 2011 15:57, R?mi Forax wrote: > > Le 31/01/2011 16:30, Martijn Verburg a ?crit : > >> > >> Hi all, > >> > >> I've finally come back to this Chapter and am thinking about the various > >> rough code samples I've written (running b127 on Windows XP SP3 32bit). > My > >> next code sample simply takes the reader through some of the file system > api > >> in order to explore some of the basics. > >> > >> import java.io.IOException; > >> import java.nio.file.*; > >> > >> public class Listing_2_1 > >> { > >> public static void main(String[] args) throws IOException > >> { > >> try (FileSystem fileSystem = FileSystems.getDefault()) > >> { > >> Path timeSheetFile = > >> fileSystem.getPath("C:/projects/timesheet.txt"); > >> Path backupDir = fileSystem.getPath("H:/projects/"); > >> Path backupFile = > >> fileSystem.getPath("H:/projects/timesheet.txt"); > >> > >> timeSheetFile.checkAccess(AccessMode.READ); > >> backupDir.checkAccess(AccessMode.WRITE); > >> > >> FileStore backupFileStore = backupDir.getFileStore(); > >> if (!backupFileStore.type().equals("NTFS")) > >> { > >> throw new IOException("Suspicious storage type"); > >> } > >> > >> CopyOption copyOptions = StandardCopyOption.REPLACE_EXISTING; > >> timeSheetFile.copyTo(backupFile, copyOptions); > >> } > >> } > >> } > >> > >> The code example is a pretty patsy one, but one gotcha in particular was > >> noticed by earlier reviewers. > >> > >> The FileStore object's type is simply a String as opposed to a constant, > >> enum or other defined type. This makes it very hard to rely on for > decision > >> making (as per the if statement I have above). I assume this is a > >> limitation of being able to consistently detect the underlying storage > type? > > > > Yes, the problem is that the string can be easily misspelled. > > I wondering if it worth an API like this: > > > > interface FileStoreType { > > public String getName(); > > } > > > > public enum StandardFileStoreType implements FileStoreType { > > EXT3, EXT4, FAT32, NTFS; > > ... > > } > > > > public class FileStoreTypes { > > public static FileStoreType getFileStoreType(final String name) { > > // StandardFileStoreType.valueOf(name) > > // otherwise new FileStoreType() { > > // public String getName() { > > // return name; > > // } > > //}; > > } > > } > > > > In that case, the code can be rewrite to: > > backupFileStore.type().equals(StandardFileStoreType.NTFS)) { ... } > > > >> > >> Thanks again all. > >> Martijn > >> (@java7developer - twitter) > >> > > > > R?mi > > > > > > > > What happens if the file store uses a type not defined in the enum? > It may be possible to misspell a String, but it's worse if the API is > outdated before it even gets released because it only lists certain > filesystems. > -- > Andrew :-) Free Java Software Engineer > Red Hat, Inc. (http://www.redhat.com) > > Support Free Java! > Contribute to GNU Classpath and the OpenJDK > http://www.gnu.org/software/classpath > http://openjdk.java.net > > PGP Key: 94EFD9D8 (http://subkeys.pgp.net) > Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 > Perhaps a compromise? Have a certain defined set of StandardFileStoreTypes, yet still allow more to be added on the fly? Perhaps something a little like how CopyOptions are dealt with? OK, I'm probably starting to get out of my depth here :) Cheers, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110131/7387d413/attachment-0001.html From Alan.Bateman at oracle.com Mon Jan 31 08:45:05 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 16:45:05 +0000 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> Message-ID: <4D46E711.20804@oracle.com> Martijn Verburg wrote: > : > > The FileStore object's type is simply a String as opposed to a > constant, enum or other defined type. This makes it very hard to rely > on for decision making (as per the if statement I have above). I > assume this is a limitation of being able to consistently detect the > underlying storage type? The FileStore name and type are really meant for display/informational purposes. Say for example someone right clicks on the FileStore to look at its properties and is shows the name, type, capacity, and space available. Maybe we need to make this clearer in the javadoc. Do you have examples in mind where you might do something differently if the volume is NTFS or FAT or whatever? The only cases that we do handle is where you want to know if the file store supports certain file attributes or not, and you can query file store specific attributes. -Alan. From Alan.Bateman at oracle.com Mon Jan 31 08:55:12 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 16:55:12 +0000 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> <4D46DBDD.8010603@univ-mlv.fr> Message-ID: <4D46E970.3020308@oracle.com> Martijn Verburg wrote: > : > Perhaps a compromise? Have a certain defined set of > StandardFileStoreTypes, yet still allow more to be added on the fly? > Perhaps something a little like how CopyOptions are dealt with? OK, > I'm probably starting to get out of my depth here :) I think this is what R?mi is suggesting (and we use this pattern in a few places in the API to allow for provider specific extensions). The question is whether this is really needed and what would the standard types be (as they are many formats in use and growing all the time). -Alan From martijnverburg at gmail.com Mon Jan 31 09:00:17 2011 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 31 Jan 2011 17:00:17 +0000 Subject: Questions on the NIO.2 API In-Reply-To: <4D46E711.20804@oracle.com> References: <4CB354C4.4010605@oracle.com> <4D46E711.20804@oracle.com> Message-ID: Hi Alan, The FileStore object's type is simply a String as opposed to a constant, >> enum or other defined type. This makes it very hard to rely on for decision >> making (as per the if statement I have above). I assume this is a >> limitation of being able to consistently detect the underlying storage type? >> > The FileStore name and type are really meant for display/informational > purposes. Say for example someone right clicks on the FileStore to look at > its properties and is shows the name, type, capacity, and space available. > Maybe we need to make this clearer in the javadoc. Do you have examples in > mind where you might do something differently if the volume is NTFS or FAT > or whatever? The only cases that we do handle is where you want to know if > the file store supports certain file attributes or not, and you can query > file store specific attributes. > I think it could be made clearer in the Javadocs at the level of the class description? Something like that type and store are almost free text field meta information? The docs on the two getters for name and type are pretty clear ("...implementation specific.."). I instinctively thought about using storage type as an additional check. For example I may have plugged in an incorrect USB stick (with an incompatible file storage) or some dastardly Hollywood villain has inserted a storage device that's almost identical to the one that our system is used to seeing. Pity s/he didn't know about different storage types - sound the alarm! Far fetched? Probably yes :-) Cheers, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110131/2783e26e/attachment.html From martijnverburg at gmail.com Mon Jan 31 09:01:56 2011 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 31 Jan 2011 17:01:56 +0000 Subject: Questions on the NIO.2 API In-Reply-To: <4D46E970.3020308@oracle.com> References: <4CB354C4.4010605@oracle.com> <4D46DBDD.8010603@univ-mlv.fr> <4D46E970.3020308@oracle.com> Message-ID: Hi all, On Mon, Jan 31, 2011 at 4:55 PM, Alan Bateman wrote: > Martijn Verburg wrote: > >> : >> Perhaps a compromise? Have a certain defined set of >> StandardFileStoreTypes, yet still allow more to be added on the fly? Perhaps >> something a little like how CopyOptions are dealt with? OK, I'm probably >> starting to get out of my depth here :) >> > I think this is what R?mi is suggesting (and we use this pattern in a few > places in the API to allow for provider specific extensions). The question > is whether this is really needed and what would the standard types be (as > they are many formats in use and growing all the time). > My hypothetical examples are probably a bit far-fetched (to put it mildly)? I'm happy to go back to my corner and be quiet for a bit :) Cheers, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110131/27101eac/attachment.html From Thomas.Salter at unisys.com Mon Jan 31 09:18:44 2011 From: Thomas.Salter at unisys.com (Salter, Thomas A) Date: Mon, 31 Jan 2011 11:18:44 -0600 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> <4D46E711.20804@oracle.com> Message-ID: <63D5DCACD1E9E34C89C8203C64F521C3D75AC263CF@USEA-EXCH7.na.uis.unisys.com> It really needs to be stressed that applications should rarely, if ever, be dependent on a particular type of storage. There?s no telling what different FileStores will be supported by vendors who?ve ported Java to platforms other than Windows/Linux/Solaris. As portable as Java is, there are still a multitude of ways for people to write non-portable code. Let?s not create another one here. My company has ported Java to our proprietary operating systems whose file systems are nothing like anybody else?s. It?s amazing how often we?ve run into third party software that?s made assumptions that cause their software to fail on our systems. Tom From: nio-dev-bounces at openjdk.java.net [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Martijn Verburg Sent: Monday, January 31, 2011 12:00 PM To: Alan Bateman Cc: nio-dev Subject: Re: Questions on the NIO.2 API Hi Alan, The FileStore object's type is simply a String as opposed to a constant, enum or other defined type. This makes it very hard to rely on for decision making (as per the if statement I have above). I assume this is a limitation of being able to consistently detect the underlying storage type? The FileStore name and type are really meant for display/informational purposes. Say for example someone right clicks on the FileStore to look at its properties and is shows the name, type, capacity, and space available. Maybe we need to make this clearer in the javadoc. Do you have examples in mind where you might do something differently if the volume is NTFS or FAT or whatever? The only cases that we do handle is where you want to know if the file store supports certain file attributes or not, and you can query file store specific attributes. I think it could be made clearer in the Javadocs at the level of the class description? Something like that type and store are almost free text field meta information? The docs on the two getters for name and type are pretty clear ("...implementation specific.."). I instinctively thought about using storage type as an additional check. For example I may have plugged in an incorrect USB stick (with an incompatible file storage) or some dastardly Hollywood villain has inserted a storage device that's almost identical to the one that our system is used to seeing. Pity s/he didn't know about different storage types - sound the alarm! Far fetched? Probably yes :-) Cheers, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20110131/a7e5939d/attachment-0001.html From Alan.Bateman at oracle.com Mon Jan 31 09:55:35 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 17:55:35 +0000 Subject: Questions on the NIO.2 API In-Reply-To: References: <4CB354C4.4010605@oracle.com> <4D46E711.20804@oracle.com> Message-ID: <4D46F797.9010901@oracle.com> Martijn Verburg wrote: > : > I think it could be made clearer in the Javadocs at the level of the > class description? Something like that type and store are almost free > text field meta information? The docs on the two getters for name and > type are pretty clear ("...implementation specific.."). > > I instinctively thought about using storage type as an additional > check. For example I may have plugged in an incorrect USB stick (with > an incompatible file storage) or some dastardly Hollywood villain has > inserted a storage device that's almost identical to the one that our > system is used to seeing. Pity s/he didn't know about different > storage types - sound the alarm! I suspect you might be thinking about the volume name/label or volume serial number. In our Windows provider then the name method returns the volume name. The volume serial number (that get sets when the volume is formatted) isn't known to the API but can be retrieved via the getAttribute method if really needed. -Alan. From martijnverburg at gmail.com Mon Jan 31 10:49:44 2011 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 31 Jan 2011 18:49:44 +0000 Subject: Questions on the NIO.2 API In-Reply-To: <4D46F797.9010901@oracle.com> References: <4CB354C4.4010605@oracle.com> <4D46E711.20804@oracle.com> <4D46F797.9010901@oracle.com> Message-ID: Hi Alan, On Monday, January 31, 2011, Alan Bateman wrote: > Martijn Verburg wrote: > > : > I think it could be made clearer in the Javadocs at the level of the class description? ?Something like that type and store are almost free text field meta information? ?The docs on the two getters for name and type are pretty clear ("...implementation specific.."). > > I instinctively thought about using storage type as an additional check. For example I may have plugged in an incorrect USB stick (with an incompatible file storage) or some dastardly Hollywood villain has inserted a storage device that's almost identical to the one that our system is used to seeing. ?Pity s/he didn't know about different storage types - sound the alarm! > > I suspect you might be thinking about the volume name/label or volume serial number. In our Windows provider then the name method returns the volume name. The volume serial number (that get sets when the volume is formatted) isn't known to the API but can be retrieved via the getAttribute method if really needed. That would be my use case as it were yes, thanks for the pointer! Cheers, Martijn From Alan.Bateman at oracle.com Mon Jan 31 12:03:09 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 31 Jan 2011 20:03:09 +0000 Subject: /etc/mtab vs. /proc/mounts Message-ID: <4D47157D.8060200@oracle.com> I think I need help to (re)understand the difference between /etc/mtab and /proc/mounts on Linux. On Linux, our getFileStores() is implemented to essentially enumerate /etc/mtab. That should give a similar list to what df(1) prints although the filtering may be slightly different. On the other hand the getFileStore method to get the FileStore for where a file is located uses /proc/mounts to find the mount point (folks on this list may recall the reasons why we switched to using /proc [1]). Using both leads to at least one reported anomaly where it has been observed that getFileStore returns a FileStore representing the root file system with both a name and type of rootfs but df -T (and getFileStores) report something different (/dev/sda1 and ext4 on one system I tried). I'm trying to figure if this code should be changed and what the right answer is for this case. Any suggestions? Thanks, -Alan [1] http://mail.openjdk.java.net/pipermail/nio-dev/2009-May/000545.html From tigran.mkrtchyan at desy.de Mon Jan 31 12:16:10 2011 From: tigran.mkrtchyan at desy.de (Tigran Mkrtchyan) Date: Mon, 31 Jan 2011 21:16:10 +0100 Subject: /etc/mtab vs. /proc/mounts In-Reply-To: <4D47157D.8060200@oracle.com> References: <4D47157D.8060200@oracle.com> Message-ID: <4D47188A.9080003@desy.de> I guess one of the reasons can be that you may mount with -n option. Then there will be no entry in /etc/mtab file. Nevertheless, I will expect that getFileStores() still will show such mounts and this information available from /proc/mounts Regards, Tigran. On 01/31/2011 09:03 PM, Alan Bateman wrote: > > I think I need help to (re)understand the difference between /etc/mtab > and /proc/mounts on Linux. > > On Linux, our getFileStores() is implemented to essentially enumerate > /etc/mtab. That should give a similar list to what df(1) prints > although the filtering may be slightly different. On the other hand > the getFileStore method to get the FileStore for where a file is > located uses /proc/mounts to find the mount point (folks on this list > may recall the reasons why we switched to using /proc [1]). Using > both leads to at least one reported anomaly where it has been observed > that getFileStore returns a FileStore representing the root file > system with both a name and type of rootfs but df -T (and > getFileStores) report something different (/dev/sda1 and ext4 on one > system I tried). I'm trying to figure if this code should be changed > and what the right answer is for this case. Any suggestions? > > Thanks, > > -Alan > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2009-May/000545.html From forax at univ-mlv.fr Mon Jan 31 12:13:36 2011 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Mon, 31 Jan 2011 21:13:36 +0100 Subject: /etc/mtab vs. /proc/mounts In-Reply-To: <4D47157D.8060200@oracle.com> References: <4D47157D.8060200@oracle.com> Message-ID: <4D4717F0.7050804@univ-mlv.fr> On 01/31/2011 09:03 PM, Alan Bateman wrote: > > I think I need help to (re)understand the difference between /etc/mtab > and /proc/mounts on Linux. > > On Linux, our getFileStores() is implemented to essentially enumerate > /etc/mtab. That should give a similar list to what df(1) prints > although the filtering may be slightly different. On the other hand > the getFileStore method to get the FileStore for where a file is > located uses /proc/mounts to find the mount point (folks on this list > may recall the reasons why we switched to using /proc [1]). Using > both leads to at least one reported anomaly where it has been observed > that getFileStore returns a FileStore representing the root file > system with both a name and type of rootfs but df -T (and > getFileStores) report something different (/dev/sda1 and ext4 on one > system I tried). I'm trying to figure if this code should be changed > and what the right answer is for this case. Any suggestions? > > Thanks, > > -Alan > > [1] http://mail.openjdk.java.net/pipermail/nio-dev/2009-May/000545.html man mount see section "The /etc/fstab, /etc/mtab and /proc/mounts files" R?mi From david.lloyd at redhat.com Mon Jan 31 12:20:14 2011 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 31 Jan 2011 14:20:14 -0600 Subject: /etc/mtab vs. /proc/mounts In-Reply-To: <4D47157D.8060200@oracle.com> References: <4D47157D.8060200@oracle.com> Message-ID: <4D47197E.4@redhat.com> On 01/31/2011 02:03 PM, Alan Bateman wrote: > > I think I need help to (re)understand the difference between /etc/mtab > and /proc/mounts on Linux. > > On Linux, our getFileStores() is implemented to essentially enumerate > /etc/mtab. That should give a similar list to what df(1) prints although > the filtering may be slightly different. On the other hand the > getFileStore method to get the FileStore for where a file is located > uses /proc/mounts to find the mount point (folks on this list may recall > the reasons why we switched to using /proc [1]). Using both leads to at > least one reported anomaly where it has been observed that getFileStore > returns a FileStore representing the root file system with both a name > and type of rootfs but df -T (and getFileStores) report something > different (/dev/sda1 and ext4 on one system I tried). I'm trying to > figure if this code should be changed and what the right answer is for > this case. Any suggestions? I think you'll find that /proc/mounts returns *both* the rootfs and the real filesystem mounted atop it (in other words, the mount point can be non-unique). The rootfs filesystem type is used as an early bootstrap phase in many distributions (it replaces the older initramfs IIRC). That said - /etc/mtab will give you the software's idea of what the right mounts are. /proc/mounts will give you *everything* that is currently mounted, even if multiple filesystems are mounted in the same spot. Using /proc/mounts to get the filesystem is probably *not* the right approach, at least not by itself, because of the non-uniqueness of mounts, however it is probably better than /etc/mtab which may have inaccurate information. You *might* be able to rely on /proc/mounts ordering though, I'd have to look that up to be sure. If that is the case then you can use a longest-match-then-latest-match kind of algorithm and get an accurate result. Either way I'd recommend not using /etc/mtab as the information source for either method. -- - DML From hjohn at xs4all.nl Mon Jan 31 23:50:15 2011 From: hjohn at xs4all.nl (John Hendrikx) Date: Tue, 01 Feb 2011 08:50:15 +0100 Subject: /etc/mtab vs. /proc/mounts In-Reply-To: <4D47197E.4@redhat.com> References: <4D47157D.8060200@oracle.com> <4D47197E.4@redhat.com> Message-ID: <4D47BB37.3000201@xs4all.nl> David M. Lloyd wrote: > On 01/31/2011 02:03 PM, Alan Bateman wrote: >> >> I think I need help to (re)understand the difference between /etc/mtab >> and /proc/mounts on Linux. >> >> On Linux, our getFileStores() is implemented to essentially enumerate >> /etc/mtab. That should give a similar list to what df(1) prints although >> the filtering may be slightly different. On the other hand the >> getFileStore method to get the FileStore for where a file is located >> uses /proc/mounts to find the mount point (folks on this list may recall >> the reasons why we switched to using /proc [1]). Using both leads to at >> least one reported anomaly where it has been observed that getFileStore >> returns a FileStore representing the root file system with both a name >> and type of rootfs but df -T (and getFileStores) report something >> different (/dev/sda1 and ext4 on one system I tried). I'm trying to >> figure if this code should be changed and what the right answer is for >> this case. Any suggestions? > > I think you'll find that /proc/mounts returns *both* the rootfs and > the real filesystem mounted atop it (in other words, the mount point > can be non-unique). > > The rootfs filesystem type is used as an early bootstrap phase in many > distributions (it replaces the older initramfs IIRC). > > That said - /etc/mtab will give you the software's idea of what the > right mounts are. /proc/mounts will give you *everything* that is > currently mounted, even if multiple filesystems are mounted in the > same spot. Not sure what you mean by that, but if I mount two ISO images on the same spot (say /iso-image), both of the mounts show up in /etc/mtab, /proc/mounts and in df, even though only one is accessible. See here: # cat /etc/mtab (...) /temp/debian.iso /iso-image iso9660 rw,loop=/dev/loop1 0 0 /temp/ubuntu.iso /iso-image iso9660 rw,loop=/dev/loop2 0 0 # cat /proc/mounts (...) /dev/loop1 /iso-image iso9660 ro,relatime 0 0 /dev/loop2 /iso-image iso9660 ro,relatime 0 0 # df Filesystem 1K-blocks Used Available Use% Mounted on (...) /temp/debian.iso 1527898 1527898 0 100% /iso-image /temp/ubuntu.iso 1527898 1527898 0 100% /iso-image Note that only the last one mounted is accessible. --John