Proposal for adding O_DIRECT support into JDK 9

[Lu, Yingqi]

Hi Hamlin,

Thank you for testing the feature. Here is my thinking on the results:

1.     In general, DirectIO is an advanced performance feature that needs to be used with caution. It provides most benefits on modern fast storage, such as PCIeSSDs with random reads. In addition, performance benefits also vary with different IO Size and the ratio between dataset and memory capacity.

2.     Besides the throughput increase and latency reduction DirectIO might provide, the feature also makes sure the IO performance is consistent and predictable. This is because with DirectIO, IO operations bypass kernel file system cache and read ahead buffer. The page cache thrashing is avoid.

3.     In your test, what is the storage type you have on the system? HDD or faster SSDs? This might be the first and main reason that is why you do not see benefits with DirectIO.

4.     In addition, I see the maximum dataset size is 2GB (it might contain a single file or a group of files). I am not sure how much memory you have on the system, but I guess a good chunk of the data will end up reside in the memory page cache very soon after the run starts. This might be another reason why you do not see benefits.
Please let me know if you have any additional questions.
Thanks,
Lucy

From: huaming.li at oracle.com [mailto:huaming.li at oracle.com]
Sent: Thursday, September 07, 2017 1:55 AM
To: Lu, Yingqi <yingqi.lu at intel.com>; Brian Burkhalter <brian.burkhalter at oracle.com>
Cc: nio-dev at openjdk.java.net
Subject: Re: Proposal for adding O_DIRECT support into JDK 9




Besides of functionality of this feature, I did some benchmark based on webrev.07, result is at the bottom.

jmh benchmark code is at http://cr.openjdk.java.net/~mli/8164900/benchmark.test/

I'm running test on locally built jdk, which is
$ hg incoming
comparing with http://hg.openjdk.java.net/jdk10/jdk10
searching for changes
changeset:   2796:13a1041e2950
tag:         tip
date:        Wed Sep 06 16:05:49 2017 +0200

my mac is
$ uname -a
Darwin huamli-mac 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64

Seems it's not improving performance, maybe there is some degression.
I'm not sure if I'm measuring the expected scenario or not, please let me know if I miss something, or misuse jmh or new DirectIO feature.

Thank you
-Hamlin
________________________________

=========================== direct IO ===========================

Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureRead":
  767108.639 ops/s
Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureWrite":
  815677.653 ops/s

Result "directio.DirectIOPageSizeFileBenchmarkTest.measureRead":
  851990.726 ops/s
Result "directio.DirectIOPageSizeFileBenchmarkTest.measureWrite":
  848017.535 ops/s

Result "directio.DirectIOMedianFileBenchmarkTest.measureRead":
  729527.785 ops/s
Result "directio.DirectIOMedianFileBenchmarkTest.measureWrite":
  812342.645 ops/s

Result "directio.DirectIOHugeFileBenchmarkTest.measureRead":
  1331178.163 ops/s
Result "directio.DirectIOHugeFileBenchmarkTest.measureWrite":
  1345364.562 ops/s

=========================== normal IO ===========================
Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureRead":
  1110321.245 ops/s
Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureWrite":
  1038699.077 ops/s

Result "directio.DirectIOPageSizeFileBenchmarkTest.measureRead":
  1049606.473 ops/s
Result "directio.DirectIOPageSizeFileBenchmarkTest.measureWrite":
  1212248.812 ops/s

Result "directio.DirectIOMedianFileBenchmarkTest.measureRead":
  1263085.875 ops/s
Result "directio.DirectIOMedianFileBenchmarkTest.measureWrite":
  1225198.455 ops/s

Result "directio.DirectIOHugeFileBenchmarkTest.measureRead":
  2630492.405 ops/s
Result "directio.DirectIOHugeFileBenchmarkTest.measureWrite":
  2647528.429 ops/s

On 07/09/2017 3:52 PM, huaming.li at oracle.com<mailto:huaming.li at oracle.com> wrote:

Hi Lucy,

Just for your information.

I applied the patch, face following compilation error, I worked on Mac, Darwin Kernel Version 16.7.0.

/workspace/jdk/jdk10/jdk/src/java.base/unix/classes/sun/nio/fs/UnixChannelFactory.java:247: error: cannot find symbol
            oflags |= O_DIRECT;
                      ^
  symbol:   variable O_DIRECT
  location: class UnixChannelFactory

Thank you

-Hamlin

On 07/09/2017 2:38 AM, Lu, Yingqi wrote:
Hi Brian,

The webrev.08 is available at http://cr.openjdk.java.net/~kkharbas/8164900/webrev.08/<http://cr.openjdk.java.net/%7Ekkharbas/8164900/webrev.08/>

It supposes to address the following the issues. Please let me know if there is anything missing.

Thanks,
Lucy

From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com]
Sent: Tuesday, September 05, 2017 3:35 PM
To: Lu, Yingqi <yingqi.lu at intel.com><mailto:yingqi.lu at intel.com>
Cc: huaming.li at oracle.com<mailto:huaming.li at oracle.com>; nio-dev at openjdk.java.net<mailto:nio-dev at openjdk.java.net>
Subject: Re: Proposal for adding O_DIRECT support into JDK 9

Hi Lucy,

Based on what I recall seeing in the e-mail thread since the previous version this sounds to be about right.

Thanks,

Brian

On Sep 5, 2017, at 3:27 PM, Lu, Yingqi <yingqi.lu at intel.com<mailto:yingqi.lu at intel.com>> wrote:



Based on the recent feedback I receive, I am going to make following changes into the next version of the patch. Please let me know if there is anything missing or not correctly understood.


1.     Apply Volker's updated patch for AIX DirectIO support.

2.     Add 2 static functions inside IOUtil.java for position check and remaining buffer size check. Change IOUtil.java and FileChannelImpl.java accordingly.

3.     Put Solaris Filesystem check into DirectIOTest.java and use it for all the existing tests.

4.     Add 1 additional test for both read and write to test buffer location alignment.

5.     Add 1 test case into ReadDirect test represent the case file size is smaller than block size.

6.     Remove 4096 from all the tests and replace it with a meaningful variable.

7.     Change the function name inside the tests to be more meaningful.
Please let me know if there is anything missing.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20170907/1aa56dac/attachment-0001.html>