JDK-8259873: (fs) Add java.nio.file.attribute.BasicWithKeyFileAttributeView interface

[Martin Buchholz]

I'm mostly cheerleading here, and haven't developed for Windows in many
years, but ... I agree with the strategy here.  More use of file keys would
benefit performance on all platforms.  OTOH I worry that filekeys are
inconsistently implemented on different OSes and filesystems.

On Tue, Jan 19, 2021 at 2:36 PM Renaud Paquay <rpaquay at google.com> wrote:

> As per comment in JDK-8259873
> <https://bugs.openjdk.java.net/browse/JDK-8259873>, I am starting a
> discussion topic for the proposed change in
> https://github.com/openjdk/jdk/pull/2121.
>
> Some context: The change, along with
> https://github.com/openjdk/jdk/pull/2122, is about improving performance
> of Files.walkFileTree on the Windows platform.
>
> PR 2122
> =======
>
> PR 2122 is an "in place" (i.e. no behavior or public API change)
> implementation change in WindowsDirectoryStream: Instead of using
> FindFirstFile
> <https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-findfirstfilew>
> ,FindNextFile
> <https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-findnextfilew>,
> the new implementation uses NtQueryDirectoryFile
> <https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntquerydirectoryfile>,
> which is more efficient at enumerating directory entries and allows
> acquiring file keys for "free" with each entry.
>
> With a file key available, Files.walkFileTree can implement symlink loop
> detection very efficiently (comparing 2 file keys in memory), as opposed to
> calling Files.isSameFile which incurs 2 file I/O on Windows (see code here
> in FileTreeWalker here:
> https://github.com/openjdk/jdk/blob/05a764f4ffb8030d6b768f2d362c388e5aabd92d/src/java.base/share/classes/java/nio/file/FileTreeWalker.java#L241
> )
>
> PR 2121
> =======
>
> PR 2121 (the subject of this email) is about removing one remaining call
> to "isSameFile" in FileTreeWalker: The initial directory (this.dir) is
> part of the stack of directories used for loop detection in the wouldLoop
> <https://github.com/openjdk/jdk/blob/05a764f4ffb8030d6b768f2d362c388e5aabd92d/src/java.base/share/classes/java/nio/file/FileTreeWalker.java#L236> method.
> The goal of the change is to ensure there is a file key available for that
> initial directory so that there is never a call made to "isSameFile".
>
> With change 2122, *all* entries have a file key (in virtue of being
> retrieved with NtQueryDirectoryInformation), *except* for the "this.dir"
> entry (because that initial directory is provided by the caller of the API).
>
> The problem is then how do we allow FileTreeWalker to retrieve the file
> key for the initial directory.
>
> Option 1: Change Files.readAttributes(file, BasicFileAttributes.class,
> linkOptions) on Windows to always retrieve a file key.
>
> The current implementation
> <https://github.com/openjdk/jdk/blob/6948456dbf76bf314f17f24040c778a2d36a4431/src/java.base/windows/classes/sun/nio/fs/WindowsFileAttributes.java#L296>
> uses GetFileAttributes
> <https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfileattributesw> which
> does *not* provide a file key. To get a file key, the implementation would
> need to call GetFileInformationByHandle
> <https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfileinformationbyhandle>
> (see
> https://github.com/openjdk/jdk/blob/6948456dbf76bf314f17f24040c778a2d36a4431/src/java.base/windows/classes/sun/nio/fs/WindowsFileAttributes.java#L270).
> Unfortunately, GetFileInformationByHandle is at least an order of magnitude
> slower than GetFileAttributes, so it does not seem appropriate to slow down
> all callers, including those who don't need a file key.
>
> Option 2: Find a way for FileTreeWalker to give a "hint" to the file
> system provider that its implementation would benefit from having a file
> key available to file equality detection purposes, even if it means using a
> slower implementation.
>
> This is why PR 2121 introduces a new "BasicWithKeyFileAttributeView"
> interface. FileTreeWalker queries the file system provider using this new
> interface, hinting that it would like to obtain a file ket even if there is
> some performance penalty. The main issue with this option is that it
> introduces a new public type. One benefit is that it is a general purpose
> API that could benefit other consumers (other than FileTreeWalker). AFAIK,
> it is currently not possible for a consumer to ask for a file key
> explicitly (consumers gets them for free on Linux, and never gets them on
> Windows).
>
> Option 3: Create a Windows specific version of FileTreeWalker that calls
> directly into the WindowsFileProvider to obtain the file key.
>
> The pro here is that there is no new public type, with the cons that we'd
> have to create a windows specific version of FileTreeWalker, which is
> currently platform agnostic.
>
> PR 2121 currently implements option 2.
>
> Thoughts?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/nio-dev/attachments/20210119/d796baa6/attachment.htm>