Using NIO FS in java.util.zip.ZipFile

[Alan Bateman]

On 12/07/2024 16:06, Konstantin Nisht wrote:
> Hello everyone,
>
> In JetBrains, we are working on the improvement of interaction with 
> WSL. Due to a bug in Windows, certain interaction with WSL filesystem 
> is unreasonably slow: 
> https://youtrack.jetbrains.com/issue/IJPL-149160/java.util.zip.ZipFile-is-slow-on-WSL

Is there a brief summary on what the performance is?


> The reproducer uses plain Java IO API, so the problem is not in our 
> codebase. We think that it is possible to fix it by using Java NIO 
> with a custom `java.nio.file.spi.FileSystemProvider` (which we also 
> need for other purposes), but there is no way of currently doing it 
> for `java.util.zip.ZipFile`.
>
> We have an idea of a fix: one can observe that `ZipFile` uses 
> `java.io.RandomAccessFile` internally, which has a similar interface 
> to `SeekableByteChannel`. We attempted to patch JDK in order for 
> `ZipFile` to use `FileChannel` 
> (https://github.com/JetBrains/JetBrainsRuntime/pull/413), and noticed 
> the following issues:
> 1. On Windows, `RandomAccessFile` opens a file without 
> `FILE_SHARE_DELETE` mode. `FileChannel`, on the contrary, does open a 
> file in this mode, which results in subtle differences reflected in 
> the test `TestZipError.java`. See more in 
> https://github.com/JetBrains/JetBrainsRuntime/pull/413#issuecomment-2218299753 
> . I suspect that it is possible to fix by the use of 
> `ExtendedOpenOption.NOSHARE_DELETE`, but the patch would be more 
> complicated.

As it happens, Brian Burkhalter and I have been noodling on a 
re-implementation of java.io, including RandomFileAccess that sits on 
FileChannel. It sets up the FileChannel to not be interruptible as that 
java.io is not interruptible. We also have prototype changes to ZipFile 
to cleanup how OPEN_DELETE works. The motive for these prototypes is 
virtual threads. The main hurdle is startup performance, we still have 
some regressions due to some additional class loading + code executed 
that need to be worked on. The prototype we have handles 
FILE_SHARE_DELETE so there are no observable behavioral differences.


> 2. A more serious issue, `FileChannel` performs reading and writing 
> using direct buffers. It breaks an (implicit?) contract that by 
> default JVM does not allocate memory-mapped files, as reflected by the 
> test `LimitDirectMemory.java`.
There is nothing to prohibit JDK classes from using direct buffers or do 
memory mapping, there are several examples of this already.

-Alan