Proposal for adding O_DIRECT support into JDK 9

[Paul Sandoz]

Hi Peter,

Thanks for pointing this out, you beat me to it :-)

Note that the alignment for a unit size corresponding to page size will only be stable for direct byte buffers, since for heap byte buffers the GC can move the underlying byte array and the alignment properties may change.

Paul.

> On 3 Nov 2016, at 12:38, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi Lucy,
> 
> Did you know that in JDK 9 the following two methods have been added to java.nio.ByteBuffer:
> 
> /** @return  The indexed byte's memory address modulus the unit size
>   */
> public final int alignmentOffset(int index, int unitSize);
> 
> /** @return new byte buffer whose content is a shared and aligned subsequence of this buffer's content
>   */
> public final ByteBuffer alignedSlice(int unitSize);
> 
> 
> So you could express your proposed methods ByteBuffer.allocateDirectAligned() and ByteBuffer.isAligned() with the above two methods in the following way:
> 
> instead of:
> 
>     ByteBuffer bb = ByteBuffer.allocateDirectAligned(capacity)
> 
> you could do:
> 
>     int pageSize = ...
>     ByteBuffer bb = ByteBuffer.allocateDirect(capacity + pageSize - 1).alignedSlice(pageSize);
> 
> And instead of:
> 
>     bb.isAligned()
> 
> you could do:
> 
>     bb.alignmentOffset(pageSize) == 0
> 
> 
> 
> The only thing that needs to be added is a public API to return the pageSize (i.e. Bits.pageSize()).
> 
> 
> What do you think?
> 
> Regards, Peter
> 
> On 11/03/2016 12:34 AM, Lu, Yingqi wrote:
>> Hi All,
>> 
>> Our most recent DirectIO patch is available at http://cr.openjdk.java.net/~igraves/8164900-3/ <http://cr.openjdk.java.net/~igraves/8164900-3/>
>> 
>> In this version, we made following changes:
>> 
>> 1. Removed the flag "direct" from FileDescriptor class. Instead, moved it to the FileChannelImpl class.
>> 
>> 2. Provided a way for user to allocate a page aligned direct ByteBuffer.
>>     1) Added a constructor DirectByteBuffer(int cap, boolean direct) to allocate a direct ByteBuffer that is aligned to the page size.
>>     2) Added Util.getTemporaryAlignedDirectBuffer(int size)
>>     3) Added DirectByteBuffer.isAligned(int pos) to check if the buffer is aligned before doing native IO with it.
>> 
>> 3. Moved all the alignment check from C code to Java code (mainly FileChannelImpl and IOUtil.java).
>> 
>> 4. Made the DirectIO functionality consistent between read and write operations. With current version of the patch, user would be responsible for the alignment with file offset and IO size.
>> 
>> 5. Made the API for DirectIO more extensible to all the supporting platforms.
>>     1) Unix OS specific code are done through UnixConstants.java.template and FileChannelImpl.c.
>>     2) Coded and tested for Linux and OS X (OS X testing is done through VirtualBox with OS X VM on top of Linux OS).
>>     3) Coded for Solaris. We do not have environment to test it so that we commented the changes out.
>> 
>> 6. We added 4 test cases following the existing non-direct IO examples.
>> 
>> 7. We did jtreg test for the entire nio package and no errors were found due to our changes.
>> 
>> Please let us know your feedback and comment. Thank you very much for your time and consideration!
>> 
>> Thanks,
>> Lucy
>> 
>>> -----Original Message-----
>>> From: Alan Bateman [mailto:Alan.Bateman at oracle.com <mailto:Alan.Bateman at oracle.com>]
>>> Sent: Monday, October 17, 2016 7:59 AM
>>> To: Lu, Yingqi <yingqi.lu at intel.com> <mailto:yingqi.lu at intel.com>
>>> Cc: nio-dev at openjdk.java.net <mailto:nio-dev at openjdk.java.net>; Kaczmarek, Eric <eric.kaczmarek at intel.com> <mailto:eric.kaczmarek at intel.com>;
>>> Kharbas, Kishor <kishor.kharbas at intel.com> <mailto:kishor.kharbas at intel.com>
>>> Subject: Re: Proposal for adding O_DIRECT support into JDK 9
>>> 
>>> On 12/10/2016 17:41, Lu, Yingqi wrote:
>>> 
>>>> :
>>>> 
>>>> You are correct about the "extra copy" with DirectIO. Will it be acceptable if we
>>> add a function "Util.getAlignedTemporaryDirectBuffer" and use that for the
>>> DirectIO operation? In this case, I think we should be able to avoid the additional
>>> copy?
>>> Yes, that should work but it still lacks a way for the user to get an aligned buffer
>>> and so you will always be copying in and out of an aligned buffer. The other thing
>>> is the sizing of the I/O operation where I think you will also need a way to expose
>>> the multiple (or block size) to the user.
>>> 
>>> -Alan.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20161103/d566c376/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20161103/d566c376/signature-0001.asc>