AIO and some advice needed.

[Avinash Lakshman]

Looks like Linux has AIO support only if the file has been open with
O_DIRECT flag. I guess AIO is moot at this point. However what does it take
for us to request O_DIRECT support in this NIO release? We are building a
large scale storage system called Cassandra, here at Facebook, details of
which can be found here (http://code.google.com/p/the-cassandra-project/)
and would really love to have O_DIRECT support via the JDK for certain class
of operations.

Thanks
Avinash

On Fri, Nov 28, 2008 at 11:01 AM, Alan Bateman <Alan.Bateman at sun.com> wrote:

> Avinash Lakshman wrote:
>
>> Thanks Alan. Do you have any performance benchmark numbers that you can
>> share comparing the performance of synchronous and asynchronous approaches
>> on Linux?
>>
> I don't have numbers to share but one thing to say is that kernel AIO on
> Linux isn't supported (to my knowledge) for files that aren't opened for
> direct I/O. Direct I/O is very much for databases and a few cases that do
> their own caching. This means that when you are using an asynchronous file
> channel it is actually doing synchronous I/O with a thread pool. This works
> surprisingly well but you need to play around with sizing of the thread pool
> to get the right concurrency. In the future we can make use of the kernel
> support if it changes. I don't think I understand your environment
> sufficiently to make any reasonable recommendations. I understand there is a
> big 45GB file but it's not clear to me how it is used. If indeed you can
> read and process areas of the file in parallel then try
> AsynchronousFileChannel and send feedback. FileChannel also defines a
> positional read method and so can support concurrent access without concern
> for the global file position.
>
> -Alan.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20081128/f03c98c3/attachment.html