parallel IO anomaly with windows REFS formatted drive

[Bernd Eckenfels]

Just a FYI, I would not worry too much about this, Microsoft removed the ability to create ReFS volumes in Win 10 1710 as they seem to only want to support it with Servers and the new Workstation SKUs. Wouldn't it be enough to put a warning into the release notes?

However it is again and again interesting to see just how problematic the NIO2 implementations in some areas are...

Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
From: core-libs-dev <core-libs-dev-bounces at openjdk.java.net> on behalf of Michael Skells <mike.skells1 at gmail.com>
Sent: Sunday, October 29, 2017 11:09:23 PM
To: core-libs-dev at openjdk.java.net
Subject: parallel IO anomaly with windows REFS formatted drive

Hi
It seems to me that some NIO based file creates don't scale work well with
REFS formatted drives
REFS is an alternative (to NTFS) drive format for Windows 8 and above

When working with NTFS on my test system, as we use more threads the
performance improves so some degree, (1000 files creates take 5-600 ms with
one thread and 2-300 ms with 4 threads)
when using REFS the single threaded times are roughly the same for 1 and 2
threads, but get much worse for 3 and 4 threads (1200 - 1600 ms)

this showdown is only when using FileChanel.open and synchronous IO, for
the AsynchronousFileChannel it seems roughly the to scale as expected

CSV attached for the test results, and some test code. The test code is in
scala, and I can convert if needed, but it just really 3 calls, to open
write and close the files

My test system is an I7 4 core window 10 pro with Norton installed, but an
exclusion for the specified drives, and indexing disabled. All writes are
to an M.2 SSD with write-though cache

Some background

I am working on the back end code generation of the scala compiler which
generates many thousand  small .class files, and part of that is to tune
the IO. For windows where the IO cost is significant, so I had a few test
app to simulate and measure the relative performance of the different
alternatives

Any ideas for a bulk file creation alternative welcome. I know we could
write to a JAR but that requires other re-tooling

Regards
Mike