Federation of file systems and the default file system provider (sorry, looong post)

[Alan Bateman]

Christian Schlichtherle wrote:
>
> Hi everyone,
>
>  
>
> I am the author of TrueZIP, a virtual file system to allow 
> multi-threaded, transparent read/write access to ZIP64, TAR and many 
> derived archive types. I am currently evaluating JDK 7 in order to 
> learn how I could integrate TrueZIP 7 (or whatever its name will be by 
> then) with JDK 7.
>
>  
>
> I've learned from a presentation that federation is not provided by 
> NIO2. I would like to understand this better: What exactly does this mean?
>
>  
>
> For me, federation would mean that I could address different file 
> systems with one path. For example, with TrueZIP 6.7 you could use the 
> path "/home/christian/archive.zip/dir/some.entry" to address the ZIP 
> file entry "dir/some.entry" in the ZIP file 
> "/home/christian/archive.zip".  The former addresses an entity in a 
> ZIP file system while the latter addresses an entity in some OS file 
> system. So in TrueZIP 6.7, federation of file systems apparently means 
> the Unix concept of "file system mount points" (TrueZIP works on 
> Windows, too, of course). In order to determine which portion of a 
> path addresses an archive file, TrueZIP uses an ArchiveDetector 
> interface when scanning the path -- this is completely configurable at 
> runtime of course.
>
>  
>
> So when it's said that NIO2 doesn't support federation, I suppose this 
> means that it does not support a mechanism of automatically detecting 
> which portion of a path belongs to which FileSystemProvider except 
> when explicitly addressing it by a schema, right? So to address the 
> same path with NIO2, one would have to rewrite it from 
> "/home/christian/archive.zip/dir/some.entry" to 
> "zip:file:///home/christian/archive.zip!/dir/some.entry", right?
>
>  
>
> How does this affect copying or other file operations then? Is it 
> possible to copy from e.g. 
> "zip:file:///home/christian/archive.zip/dir/some.entry" to 
> "file:///home/christian/some.entry"?
>
>  
>
> Given the fact that TrueZIP 6.7 does support federation (configured 
> via ArchiveDetector instances), I think it should be possible to bring 
> the same flexibility to JDK 7. Let's assume that there exists a 
> FileSystemProvider implementation of TrueZIP which is associated to 
> the schema "truezip:" and is configured with the appropriate 
> ArchiveDetector instances. Then it should be possible to rewrite the 
> previous path simply to 
> "truezip:///home/christian/archive.zip/dir/some.entry", right?
>
>  
>
> I should then be able to bring all the benefits of TrueZIP over to JDK 
> 7, including fully transparent creating, reading, updating and 
> deleting of archive entries, even cross (archive) file system type 
> (e.g. when moving a virtual file from "archive.zip/entry" to 
> "archive.tar.gz/entry").
>
>  
>
> Did I get everything right so far?
>
>  
>
> Now going one step further, one might even want to replace the default 
> FileSystemProvider implementation with the TrueZIP FileSystemProvider 
> in order to benefit from its federation by default, skip the schema 
> and simplify the path to what you could use in TrueZIP 6.7 today: 
> "/home/christian/archive.zip/dir/some.entry". So is it possible to 
> replace the default FileSystemProvider at runtime?
>
>  
>
> I guess some people may be concerned by now that this could break all 
> kinds of things? Well yes, it would be possible to do so. But if this 
> feature exists, this is a risk pertaining to ANY FileSystemProvider 
> implementation, not just TrueZIP in particular. Regarding TrueZIP, it 
> just augments the (real) file system tree with virtual archive file 
> systems similar to the way mount points augment the unified file 
> system tree in Unix-like OS.
>
>  
>
> So all I'm asking for is the freedom of choice. The same freedom of 
> choice you have today with SAX or DOM parser implementations: You 
> could replace the JSE provided implementations with any arbitrary 
> implementation, and there's no way the JSE could guarantee they'll do 
> what you want or even work at all. If that's fair use there, it should 
> be fair use here, too.
>
>  
>
> What do you think? Would it be possible?
>
>  
>
> Best regards,
>
> Christian Schlichtherle
>
Christian - the provider mechanism can be used to develop and deploy 
your own virtual file system along the lines of what you are doing. The 
default provider is replaceable and can also be wrapped by another 
provider that becomes the default - look at FileSystems.getDefault for 
the details. The only thing that isn't currently working is that 
java.io.File, FileInputStream, etc. don't redirect when the default 
provider is replaced.

One of the questions in the above was on copy operations - the copyTo 
doesn't require target location to be associated with the same provider 
as the source location. In our default provider for example, it falls 
back to a simple copy implementation when the target is associated a 
provider other than the default. There's another example in one of the 
sanity tests for the demo zip provider where it copies a file from a zip 
file to the regular file system.

Hope this helps,

-Alan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20100805/c3fc1375/attachment-0001.html