NIO performance

[John Hendrikx]

Alan Bateman wrote:
> Files.walkFileTree is essentially an internal iterator built on an 
> external iterator (DirectoryStream). So for the maxDepth == 1 case 
> then it is reasonable to expect the performance to be  the same as 
> using DirectoryStream to iterate over all entries in the directory, 
> calling Attributes.readBasicFileAttributes to read the attributes of 
> each file. The anomaly you are seeing is a Windows only anomaly. 
> Elsewhere (on Solaris and Linux at least) the performance will be as 
> you would expect. For example, I did a quick test on Solaris with a 
> directory of 9000 files and the simple iteration took 22ms, the 
> iteration + reading the attributes took 88ms, and walkFileTree with 
> maxDepth==1 took 83ms. On Windows, the anomaly (or why is 
> Files.walkFileTree so much faster) is because the attributes are 
> obtained during the directory traversal so the implementation can 
> avoid re-reading them - if it re-read the attributes for each file 
> then it would take about the same time as calling 
> Attributes.readBasicFileAttributes for each file in the directory, an 
> operation that is expensive on Windows. One thing to say is that 
> difference isn't as obvious with NTFS - for example, I repeated your 
> test with a directory of 9000 files and the simple iteration took 
> 21ms, the iteration + reading the attributes took 237ms, and 
> Files.walkFileTree took 20ms. With FAT32 or when the volume is remote 
> then the difference is very obvious - I'll guess this is what you are 
> testing on.
>
> This issue does bring up the question as to if we need a method that 
> returns a DirectoryStream where the elements are a pair consisting of 
> the entry and its attributes. It's come up once or twice. From a 
> performance point of view it helps Windows, and maybe some custom file 
> systems. The other potential justification is convenience in that the 
> basic attributes will often be required when iterating over a 
> directory. It's worth looking at.
>
> -Alan.
>
You are right, I hadn't realized this was important, but the files were 
on a Samba share (Java running on Windows, scanning a network share on 
Linux using a path like: //Server/BigDir).  So, the volume was remote as 
you suspected. 

--John