Proposal for adding O_DIRECT support into JDK 9
huaming.li at oracle.com
huaming.li at oracle.com
Fri Sep 8 10:22:16 UTC 2017
Hi Lucy,
Thank you for follow-up. I'm not familiar with direct io, it may report
false positive issue, or testing unexpected scenario, so the test result
is just for your information
my local dist type:
Device / Media Name: APPLE SSD SM0512G
Thank you
-Hamlin
On 08/09/2017 12:43 AM, Lu, Yingqi wrote:
>
> Hi Hamlin,
>
> Thank you for testing the feature. Here is my thinking on the results:
>
> 1.In general, DirectIO is an advanced performance feature that needs
> to be used with caution. It provides most benefits on modern fast
> storage, such as PCIeSSDs with random reads. In addition, performance
> benefits also vary with different IO Size and the ratio between
> dataset and memory capacity.
>
> 2.Besides the throughput increase and latency reduction DirectIO might
> provide, the feature also makes sure the IO performance is consistent
> and predictable. This is because with DirectIO, IO operations bypass
> kernel file system cache and read ahead buffer. The page cache
> thrashing is avoid.
>
> 3.In your test, what is the storage type you have on the system? HDD
> or faster SSDs? This might be the first and main reason that is why
> you do not see benefits with DirectIO.
>
> 4.In addition, I see the maximum dataset size is 2GB (it might contain
> a single file or a group of files). I am not sure how much memory you
> have on the system, but I guess a good chunk of the data will end up
> reside in the memory page cache very soon after the run starts. This
> might be another reason why you do not see benefits.
>
> Please let me know if you have any additional questions.
>
> Thanks,
>
> Lucy
>
> *From:*huaming.li at oracle.com [mailto:huaming.li at oracle.com]
> *Sent:* Thursday, September 07, 2017 1:55 AM
> *To:* Lu, Yingqi <yingqi.lu at intel.com>; Brian Burkhalter
> <brian.burkhalter at oracle.com>
> *Cc:* nio-dev at openjdk.java.net
> *Subject:* Re: Proposal for adding O_DIRECT support into JDK 9
>
> Besides of functionality of this feature, I did some benchmark based
> on webrev.07, result is at the bottom.
>
> jmh benchmark code is at
> http://cr.openjdk.java.net/~mli/8164900/benchmark.test/
> <http://cr.openjdk.java.net/%7Emli/8164900/benchmark.test/>
>
> I'm running test on locally built jdk, which is
>
> $ hg incoming
> comparing with http://hg.openjdk.java.net/jdk10/jdk10
> searching for changes
> changeset: 2796:13a1041e2950
> tag: tip
> date: Wed Sep 06 16:05:49 2017 +0200
>
> my mac is
> $ uname -a
> Darwin huamli-mac 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15
> 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
>
> Seems it's not improving performance, maybe there is some degression.
> I'm not sure if I'm measuring the expected scenario or not, please let
> me know if I miss something, or misuse jmh or new DirectIO feature.
>
> Thank you
> -Hamlin
>
> ------------------------------------------------------------------------
>
>
> =========================== direct IO ===========================
>
> Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureRead":
> 767108.639 ops/s
> Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureWrite":
> 815677.653 ops/s
>
> Result "directio.DirectIOPageSizeFileBenchmarkTest.measureRead":
> 851990.726 ops/s
> Result "directio.DirectIOPageSizeFileBenchmarkTest.measureWrite":
> 848017.535 ops/s
>
> Result "directio.DirectIOMedianFileBenchmarkTest.measureRead":
> 729527.785 ops/s
> Result "directio.DirectIOMedianFileBenchmarkTest.measureWrite":
> 812342.645 ops/s
>
> Result "directio.DirectIOHugeFileBenchmarkTest.measureRead":
> 1331178.163 ops/s
> Result "directio.DirectIOHugeFileBenchmarkTest.measureWrite":
> 1345364.562 ops/s
>
> =========================== normal IO ===========================
> Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureRead":
> 1110321.245 ops/s
> Result "directio.DirectIOBlockSizeFileBenchmarkTest.measureWrite":
> 1038699.077 ops/s
>
> Result "directio.DirectIOPageSizeFileBenchmarkTest.measureRead":
> 1049606.473 ops/s
> Result "directio.DirectIOPageSizeFileBenchmarkTest.measureWrite":
> 1212248.812 ops/s
>
> Result "directio.DirectIOMedianFileBenchmarkTest.measureRead":
> 1263085.875 ops/s
> Result "directio.DirectIOMedianFileBenchmarkTest.measureWrite":
> 1225198.455 ops/s
>
> Result "directio.DirectIOHugeFileBenchmarkTest.measureRead":
> 2630492.405 ops/s
> Result "directio.DirectIOHugeFileBenchmarkTest.measureWrite":
> 2647528.429 ops/s
>
> On 07/09/2017 3:52 PM, huaming.li at oracle.com
> <mailto:huaming.li at oracle.com> wrote:
>
> Hi Lucy,
>
> Just for your information.
>
> I applied the patch, face following compilation error, I worked on
> Mac, Darwin Kernel Version 16.7.0.
>
> /workspace/jdk/jdk10/jdk/src/java.base/unix/classes/sun/nio/fs/UnixChannelFactory.java:247:
> error: cannot find symbol
> oflags |= O_DIRECT;
> ^
> symbol: variable O_DIRECT
> location: class UnixChannelFactory
>
> Thank you
>
> -Hamlin
>
> On 07/09/2017 2:38 AM, Lu, Yingqi wrote:
>
> Hi Brian,
>
> The webrev.08 is available at
> http://cr.openjdk.java.net/~kkharbas/8164900/webrev.08/
> <http://cr.openjdk.java.net/%7Ekkharbas/8164900/webrev.08/>
>
> It supposes to address the following the issues. Please let me
> know if there is anything missing.
>
> Thanks,
>
> Lucy
>
> *From:*Brian Burkhalter [mailto:brian.burkhalter at oracle.com]
> *Sent:* Tuesday, September 05, 2017 3:35 PM
> *To:* Lu, Yingqi <yingqi.lu at intel.com>
> <mailto:yingqi.lu at intel.com>
> *Cc:* huaming.li at oracle.com <mailto:huaming.li at oracle.com>;
> nio-dev at openjdk.java.net <mailto:nio-dev at openjdk.java.net>
> *Subject:* Re: Proposal for adding O_DIRECT support into JDK 9
>
> Hi Lucy,
>
> Based on what I recall seeing in the e-mail thread since the
> previous version this sounds to be about right.
>
> Thanks,
>
> Brian
>
> On Sep 5, 2017, at 3:27 PM, Lu, Yingqi <yingqi.lu at intel.com
> <mailto:yingqi.lu at intel.com>> wrote:
>
>
>
>
> Based on the recent feedback I receive, I am going to make
> following changes into the next version of the patch.
> Please let me know if there is anything missing or not
> correctly understood.
>
> 1.Apply Volker’s updated patch for AIX DirectIO support.
>
> 2.Add 2 static functions inside IOUtil.java for position
> check and remaining buffer size check. Change IOUtil.java
> and FileChannelImpl.java accordingly.
>
> 3.Put Solaris Filesystem check into DirectIOTest.java and
> use it for all the existing tests.
>
> 4.Add 1 additional test for both read and write to test
> buffer location alignment.
>
> 5.Add 1 test case into ReadDirect test represent the case
> file size is smaller than block size.
>
> 6.Remove 4096 from all the tests and replace it with a
> meaningful variable.
>
> 7.Change the function name inside the tests to be more
> meaningful.
>
> Please let me know if there is anything missing.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20170908/35df9322/attachment-0001.html>
More information about the nio-dev
mailing list