Proposal for adding O_DIRECT support into JDK 9

[Lu, Yingqi]

Hi Alan,

Thank you for your feedback.

The reason we added "direct" flag to the FileDescriptor class is to follow the example of "append" flag. We can surely change the "direct" check to FileChannelImpl constructor, should be easy to do.

You are correct about the "extra copy" with DirectIO. Will it be acceptable if we add a function "Util.getAlignedTemporaryDirectBuffer" and use that for the DirectIO operation? In this case, I think we should be able to avoid the additional copy?

Thanks,
Lucy

-----Original Message-----
From: Alan Bateman [mailto:Alan.Bateman at oracle.com] 
Sent: Wednesday, October 12, 2016 6:51 AM
To: Lu, Yingqi <yingqi.lu at intel.com>
Cc: nio-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>; Kharbas, Kishor <kishor.kharbas at intel.com>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9

On 04/10/2016 18:24, Lu, Yingqi wrote:

> Hi Christoph,
>
> Thank you very much for trying our patch.
>
> We are still seeking the feedback from the community. When we get closer to the final version of the patch, we will modify the copyright years. Thank you for reminding us!
>
> Anyone else has any feedback/comments?
>
Lucy,

I've looked through the changes and have a number of concerns.

The main concern is that I/O with files opened for O_DIRECT will usually require the buffer to be page aligned and the buffer sized to be a multiple of the file system block size. The patch deals with the alignment by doing a posix_memalign + copy + free for every I/O operation. I don't know if you've tried doing a write with a ByteBuffer backed with an array in the heap but if I read the patch correctly then you'll end up copying it twice, once into a thread local direct buffer and then a second time into the temporary allocated buffer. Ditto for read where it will initially read it into a temporary allocated buffer, then into a thread local direct buffer, and finally into the user's buffer backed by an array in the heap. In the past when we've looked at this issue then the conclusion was that there needed to be a way to get a ByteBuffer that is appropriate aligned and also a way to get information about the file system to size I/O operations too. It is possible to do it transparently (like you are doing) but it requires the thread local buffer cache support aligned buffers.

Aside from that then the other concern is addding a direct flag to FileDescriptor. I would think it should be simpler to capture this in FileChannelImpl constructor and use that to send the I/O through the appropriate path (this avoids have the methods in IOUtils fan out).

It may have been mentioned in other mails but there aren't any tests in the patch and I think this will require a good set of tests in order to be confident with all the cases.

-Alan