fastest API for writing many small files

[Mike Skells]

Hi NIO experts,

I am working on an application that generate 1000 to 100,000 small files.
The use case will already have a byte array with all of the data required

 

This application is targeted to run on Windows (7/10, 64 bit)  and Linux
(64bit) running java 8, machines all have ssds

 

One of the limiting characteristics is the time to write all of these files,
particularly when running on Windows.

 

I have been experimenting with various io and nio APIs to do this (test app
is in scala). Happy to share this comparison code if it helps

 

Are there any recommendations as to the fastest way to do this? Happy for
the answer to be OS or java version specific. I can't think that this is an
uncommon requirement, but could not see a question related to this on the
usual sources :(

 

If this is not the correct forum for this sort of question then please
recommend an appropriate one

 

Thanks

Mike

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20171001/210f786f/attachment.html>