Proposal for adding O_DIRECT support into JDK 9

[Lu, Yingqi]

Another approach might be use ioctl to get the block size and use that as the alignment value. I think ioctl is available on UNIX based OSes (please chime in if I miss any OS here) and has a similar call for Windows named DeviceIoControl. The request code of ioctl is different from OS to OS, but we could address that in the native code.

What do you all think on this?

Thanks,
Lucy

From: Thomas Stüfe [mailto:thomas.stuefe at gmail.com]
Sent: Sunday, November 06, 2016 3:22 AM
To: David Holmes <david.holmes at oracle.com>
Cc: Paul Sandoz <paul.sandoz at oracle.com>; Lu, Yingqi <yingqi.lu at intel.com>; nio-dev at openjdk.java.net; Kaczmarek, Eric <eric.kaczmarek at intel.com>; core-libs-dev at openjdk.java.net; Kharbas, Kishor <kishor.kharbas at intel.com>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9



On Sat, Nov 5, 2016 at 6:42 PM, David Holmes <david.holmes at oracle.com<mailto:david.holmes at oracle.com>> wrote:
Hi Paul,

On 5/11/2016 2:58 AM, Paul Sandoz wrote:
Hi Lucy,

I am struggling to find an entirely satisfactory location to place a method such as getPageSize, but java.lang.Runtime seems less worse than other places :-)

This does not seem an appropriate method to expose in this form at the java.lang.Runtime level. If we want a public, platform agnostic API for exposing virtual memory page sizes (plural) then we should design such an API IMHO.

I absolutely agree. Different memory regions can have different page sizes following different rules. The fact that in hotspot coding os::vm_page_size() is used in many places as "the pagesize" gave us lots of headaches in the AIX port, where we can have up to four different page sizes for heap, stack, text and different instances of shared memory. I consider the fact that unsafe.pageSize() exists sloppy API design. Even on Linux we can have two page sizes within one process, or?

If we really need a generic "pageSize" API this should be designed to cover a reasonable large array of OS breeds. IMHO a good generic API should allow me to query, given a memory range, its page-size-composition, because within a region of memory one may have different page sizes.

Thomas

David

Paul.
On 3 Nov 2016, at 12:49, Lu, Yingqi <yingqi.lu at intel.com<mailto:yingqi.lu at intel.com>> wrote:

Hi Paul and Peter,

Thank you very much for pointing this out. We did not know about it J

We will modify the patch accordingly and will send it here for review.

Thanks,
Lucy

From: Paul Sandoz [mailto:paul.sandoz at oracle.com<mailto:paul.sandoz at oracle.com>]
Sent: Thursday, November 03, 2016 12:46 PM
To: Peter Levart <peter.levart at gmail.com<mailto:peter.levart at gmail.com>>
Cc: Lu, Yingqi <yingqi.lu at intel.com<mailto:yingqi.lu at intel.com>>; Alan Bateman <Alan.Bateman at oracle.com<mailto:Alan.Bateman at oracle.com>>; nio-dev at openjdk.java.net<mailto:nio-dev at openjdk.java.net>; Kaczmarek, Eric <eric.kaczmarek at intel.com<mailto:eric.kaczmarek at intel.com>>; core-libs-dev at openjdk.java.net<mailto:core-libs-dev at openjdk.java.net>; Kharbas, Kishor <kishor.kharbas at intel.com<mailto:kishor.kharbas at intel.com>>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9

Hi Peter,

Thanks for pointing this out, you beat me to it :-)

Note that the alignment for a unit size corresponding to page size will only be stable for direct byte buffers, since for heap byte buffers the GC can move the underlying byte array and the alignment properties may change.

Paul.

On 3 Nov 2016, at 12:38, Peter Levart <peter.levart at gmail.com<mailto:peter.levart at gmail.com>> wrote:

Hi Lucy,

Did you know that in JDK 9 the following two methods have been added to java.nio.ByteBuffer:

/** @return  The indexed byte's memory address modulus the unit size
  */
public final int alignmentOffset(int index, int unitSize);

/** @return new byte buffer whose content is a shared and aligned subsequence of this buffer's content
  */
public final ByteBuffer alignedSlice(int unitSize);


So you could express your proposed methods ByteBuffer.allocateDirectAligned() and ByteBuffer.isAligned() with the above two methods in the following way:

instead of:

    ByteBuffer bb = ByteBuffer.allocateDirectAligned(capacity)

you could do:

    int pageSize = ...
    ByteBuffer bb = ByteBuffer.allocateDirect(capacity + pageSize - 1).alignedSlice(pageSize);

And instead of:

    bb.isAligned()

you could do:

    bb.alignmentOffset(pageSize) == 0



The only thing that needs to be added is a public API to return the pageSize (i.e. Bits.pageSize()).


What do you think?

Regards, Peter

On 11/03/2016 12:34 AM, Lu, Yingqi wrote:
Hi All,

Our most recent DirectIO patch is available at http://cr.openjdk.java.net/~igraves/8164900-3/

In this version, we made following changes:

1. Removed the flag "direct" from FileDescriptor class. Instead, moved it to the FileChannelImpl class.

2. Provided a way for user to allocate a page aligned direct ByteBuffer.
    1) Added a constructor DirectByteBuffer(int cap, boolean direct) to allocate a direct ByteBuffer that is aligned to the page size.
    2) Added Util.getTemporaryAlignedDirectBuffer(int size)
    3) Added DirectByteBuffer.isAligned(int pos) to check if the buffer is aligned before doing native IO with it.

3. Moved all the alignment check from C code to Java code (mainly FileChannelImpl and IOUtil.java).

4. Made the DirectIO functionality consistent between read and write operations. With current version of the patch, user would be responsible for the alignment with file offset and IO size.

5. Made the API for DirectIO more extensible to all the supporting platforms.
    1) Unix OS specific code are done through UnixConstants.java.template and FileChannelImpl.c.
    2) Coded and tested for Linux and OS X (OS X testing is done through VirtualBox with OS X VM on top of Linux OS).
    3) Coded for Solaris. We do not have environment to test it so that we commented the changes out.

6. We added 4 test cases following the existing non-direct IO examples.

7. We did jtreg test for the entire nio package and no errors were found due to our changes.

Please let us know your feedback and comment. Thank you very much for your time and consideration!

Thanks,
Lucy

-----Original Message-----
From: Alan Bateman [mailto:Alan.Bateman at oracle.com<mailto:Alan.Bateman at oracle.com>]
Sent: Monday, October 17, 2016 7:59 AM
To: Lu, Yingqi <yingqi.lu at intel.com<mailto:yingqi.lu at intel.com>>
Cc: nio-dev at openjdk.java.net<mailto:nio-dev at openjdk.java.net>; Kaczmarek, Eric <eric.kaczmarek at intel.com<mailto:eric.kaczmarek at intel.com>>;
Kharbas, Kishor <kishor.kharbas at intel.com<mailto:kishor.kharbas at intel.com>>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9

On 12/10/2016 17:41, Lu, Yingqi wrote:

:

You are correct about the "extra copy" with DirectIO. Will it be acceptable if we
add a function "Util.getAlignedTemporaryDirectBuffer" and use that for the
DirectIO operation? In this case, I think we should be able to avoid the additional
copy?
Yes, that should work but it still lacks a way for the user to get an aligned buffer
and so you will always be copying in and out of an aligned buffer. The other thing
is the sizing of the I/O operation where I think you will also need a way to expose
the multiple (or block size) to the user.

-Alan.