From Alan.Bateman at oracle.com Sat Sep 1 07:57:30 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 1 Sep 2018 08:57:30 +0100 Subject: 8210279: (bf) Remove unused package private method java.nio.Buffer.truncate() In-Reply-To: <5CEA5EF4-AD6F-4582-8CAE-93EB24280808@oracle.com> References: <802c7168-f4b4-500f-0875-8bfc6f7a021f@oracle.com> <5CEA5EF4-AD6F-4582-8CAE-93EB24280808@oracle.com> Message-ID: <0480e1cf-9b4e-6c37-fd47-527961545627@oracle.com> On 31/08/2018 23:03, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8210279 > > Remove the indicated method which was inadvertently overlooked in the > fix of JDK-8193822 . > Looks good. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjpt777 at gmail.com Sun Sep 2 17:28:14 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Sun, 2 Sep 2018 18:28:14 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> Message-ID: I've put together some benchmarks and can quantify the impact of the current implementation. https://github.com/mjpt777/java-benchmarks If we consider the latency impact via loopback, which is similar to a fast kernel bypass network, there is a 25% increase in round trip latency with two sources compared to one source. This is apply dual sources in only one direction so half the actual RTT impact. With Java 11 the delta is slightly increased between the single and dual source cases. https://github.com/mjpt777/java-benchmarks/blob/master/results/latency.txt This suggests a 50% total cost increase for each one-way transmission when the source does not match and a new address has to be created and set in the cache field. When profiled the timing should reflect this hypothesis. It does in the flame graphs. https://github.com/mjpt777/java-benchmarks/blob/master/src/jmh/java/uk/co/real_logic/benchmarks/nio/MultiThreadedConnectedDatagramChannelBenchmark.java https://github.com/mjpt777/java-benchmarks/blob/master/results/cpu-single-source.svg https://github.com/mjpt777/java-benchmarks/blob/master/src/jmh/java/uk/co/real_logic/benchmarks/nio/MultiThreadedSeparateConnectedDatagramChannelBenchmark.java https://github.com/mjpt777/java-benchmarks/blob/master/results/cpu-dual-source.svg With a single source per channel, we see the majority of the time is spent in the underlying receive from network. When two sources exist then ~50% of the time is spent in the JNI code, and note the up call to allocate the address object which seems high given such a relatively small object. The NET_SockaddrToInetAddress is the other major component of the cost within the JNI call. Having compared native C implementations I see a further saving of > 0.5?s over the single source case which suggests the JNI and DatagramChannel code has significant opportunities to be more efficient. Maybe JNI up calls are not one of the better design choices. Most services have more than one source and whatever approach is taken it is best if allocation is not utilised unless we have a change of configuration. GC pauses can result in data loss, retransmits, and network partitions. Nodes appearing unavailable is far more commonly due to GC than actual network partitions in managed languages. Regards, Martin... On Tue, 28 Aug 2018 at 08:31, Alan Bateman wrote: >> As regards implementation changes then I think start by eliminating the > JNI calls from the receive implementation and move it all to java. Each > DCI can have an area of memory (probably 18 bytes) for the native code > to write the source address and port. The wrapper can create the > InetAddress/InetSocketAddress. Once you get that far then you can > evaluating if there is a need for any caching of InetAddress objects, > maybe the existing one-entry cache can be removed, maybe it would be > beneficial to have more caching. The benchmarks, along with other > measurements, should be help evaluate. From Alan.Bateman at oracle.com Sun Sep 2 18:28:03 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 2 Sep 2018 19:28:03 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> Message-ID: On 02/09/2018 18:28, Martin Thompson wrote: > : > > Having compared native C implementations I see a further saving of > > 0.5?s over the single source case which suggests the JNI and > DatagramChannel code has significant opportunities to be more > efficient. Maybe JNI up calls are not one of the better design > choices. > It shouldn't be too hard to eliminate the upcalls and construct the socket address at at the java level, it's just not something that has been brought up (at least not here or in JBS). Is this something that you want to work on? As regards the benchmarks, can you package these up as and send an attachment? This is important because we can't accept contributions that don't come via OpenJDK infrastructure. -Alan From mjpt777 at gmail.com Sun Sep 2 19:06:16 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Sun, 2 Sep 2018 20:06:16 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> Message-ID: On Sun, 2 Sep 2018 at 19:28, Alan Bateman wrote: > It shouldn't be too hard to eliminate the upcalls and construct the > socket address at at the java level, it's just not something that has > been brought up (at least not here or in JBS). Is this something that > you want to work on? I'm surprised you have not heard of performance issues with NIO before. Most people I know that work in this space have issues with Java. Maybe there could be better channels to solicit feedback and help Java. Survivor bias would suggest we do not get to hear of the failed cases that have to go with other languages and don't come back to Java for IO code. As I said I'd be happy to help. What would the process involve and what would be the expectation of my efforts being a worthwhile investment? > As regards the benchmarks, can you package these up as and send an > attachment? This is important because we can't accept contributions that > don't come via OpenJDK infrastructure. Attached a stripped down version of the repo. -------------- next part -------------- A non-text attachment was scrubbed... Name: nio-benchmarks.zip Type: application/x-zip-compressed Size: 67344 bytes Desc: not available URL: From Alan.Bateman at oracle.com Sun Sep 2 19:48:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 2 Sep 2018 20:48:34 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> Message-ID: <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> On 02/09/2018 20:06, Martin Thompson wrote: > : > I'm surprised you have not heard of performance issues with NIO > before. Most people I know that work in this space have issues with > Java. Maybe there could be better channels to solicit feedback and > help Java. Survivor bias would suggest we do not get to hear of the > failed cases that have to go with other languages and don't come back > to Java for IO code. It's important to report bugs performance issues. If folks don't have time to submit a bug then sending mail here is okay too. There are issues and changes discussed here all the time but mostly in other areas of NIO such as charsets, buffers, files, selectors or socket channels. DatagramChannel does not come up very often. > : > > As I said I'd be happy to help. What would the process involve and > what would be the expectation of my efforts being a worthwhile > investment? If you are looking to contribute then you'll need to sign the SCA (details on the "Contributing" page [1]). For the specific issue then I think prototype changes to pass in an area of memory to the native method so it can populate it with the family/address/port. That will allow you to create the socket address in java and avoid the expensive upcalls from the JNI code. You'll need to be careful with the endianness but it should otherwise be straight forward. > >> As regards the benchmarks, can you package these up as and send an >> attachment? This is important because we can't accept contributions that >> don't come via OpenJDK infrastructure. > Attached a stripped down version of the repo. Thanks, I will look at them. [1] http://openjdk.java.net/contribute/ From forax at univ-mlv.fr Sun Sep 2 20:42:07 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 2 Sep 2018 22:42:07 +0200 (CEST) Subject: DatagramChannel performance issue In-Reply-To: <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> References: <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> Message-ID: <276183210.1097.1535920927775.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alan Bateman" > ?: "Martin Thompson" > Cc: "nio-dev" > Envoy?: Dimanche 2 Septembre 2018 21:48:34 > Objet: Re: DatagramChannel performance issue > On 02/09/2018 20:06, Martin Thompson wrote: >> : >> I'm surprised you have not heard of performance issues with NIO >> before. Most people I know that work in this space have issues with >> Java. Maybe there could be better channels to solicit feedback and >> help Java. Survivor bias would suggest we do not get to hear of the >> failed cases that have to go with other languages and don't come back >> to Java for IO code. > It's important to report bugs performance issues. If folks don't have > time to submit a bug then sending mail here is okay too. There are > issues and changes discussed here all the time but mostly in other areas > of NIO such as charsets, buffers, files, selectors or socket channels. > DatagramChannel does not come up very often. > >> : >> >> As I said I'd be happy to help. What would the process involve and >> what would be the expectation of my efforts being a worthwhile >> investment? > If you are looking to contribute then you'll need to sign the SCA > (details on the "Contributing" page [1]). or better the OCA given that signing the SCA now requires a time machine :) > > For the specific issue then I think prototype changes to pass in an area > of memory to the native method so it can populate it with the > family/address/port. That will allow you to create the socket address in > java and avoid the expensive upcalls from the JNI code. You'll need to > be careful with the endianness but it should otherwise be straight forward. > >> >>> As regards the benchmarks, can you package these up as and send an >>> attachment? This is important because we can't accept contributions that >>> don't come via OpenJDK infrastructure. >> Attached a stripped down version of the repo. > Thanks, I will look at them. > > [1] http://openjdk.java.net/contribute/ regards, R?mi From mjpt777 at gmail.com Mon Sep 3 06:53:24 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Mon, 3 Sep 2018 07:53:24 +0100 Subject: DatagramChannel performance issue In-Reply-To: <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> Message-ID: On Sun, 2 Sep 2018 at 20:48, Alan Bateman wrote: > If you are looking to contribute then you'll need to sign the SCA > (details on the "Contributing" page [1]). I'll have a look at how onerous the terms are then make a decision. > For the specific issue then I think prototype changes to pass in an area > of memory to the native method so it can populate it with the > family/address/port. That will allow you to create the socket address in > java and avoid the expensive upcalls from the JNI code. You'll need to > be careful with the endianness but it should otherwise be straight forward. Can you provide a link to an example of how to "pass in an area of memory"? Is this different than the ByteBuffer example for the header I provided? > > Attached a stripped down version of the repo. > Thanks, I will look at them. Let me know if you'd like further refinements. While putting this together it is clear there is a performance difference on the send vs write calls which I'll separately quantify. Martin... From Alan.Bateman at oracle.com Mon Sep 3 07:39:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 3 Sep 2018 08:39:14 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> Message-ID: <03ce1be7-e727-b725-d7ac-e52338adecc7@oracle.com> On 03/09/2018 07:53, Martin Thompson wrote: > : > Can you provide a link to an example of how to "pass in an area of > memory"? Is this different than the ByteBuffer example for the header > I provided? I don't think we have other examples in the core libraries to point. DatagramSocketImpl just needs a direct buffer for the socket address which the new native receive method will populate with the source address. > : > Let me know if you'd like further refinements. While putting this > together it is clear there is a performance difference on the send vs > write calls which I'll separately quantify. > Which operating system is this? You may want to run it with -Djava.net.preferIPv4Stack=true to see if it makes a difference (as IPv6 targets, even if it's an IPv4-mapped IPv6 address, as IPv6 targets involves additional code). -Alan From mjpt777 at gmail.com Mon Sep 3 15:26:47 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Mon, 3 Sep 2018 16:26:47 +0100 Subject: DatagramChannel performance issue In-Reply-To: <03ce1be7-e727-b725-d7ac-e52338adecc7@oracle.com> References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> <03ce1be7-e727-b725-d7ac-e52338adecc7@oracle.com> Message-ID: > On Mon, 3 Sep 2018 at 08:39, Alan Bateman wrote: > I don't think we have other examples in the core libraries to point. > DatagramSocketImpl just needs a direct buffer for the socket address > which the new native receive method will populate with the source address. Is that a direct ByteBuffer? > Which operating system is this? You may want to run it with > -Djava.net.preferIPv4Stack=true to see if it makes a difference (as IPv6 > targets, even if it's an IPv4-mapped IPv6 address, as IPv6 targets > involves additional code). The posted results are for Linux (Ubuntu 18.04 LTS). I've also tested on Windows 10 and the ratios are similar. I tested with -Djava.net.preferIPv4Stack=true. The single source case is similar. The dual source case does see a reduction in latency of a few hundred nanoseconds. Thanks, Martin... From Alan.Bateman at oracle.com Mon Sep 3 17:08:35 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 3 Sep 2018 18:08:35 +0100 Subject: DatagramChannel performance issue In-Reply-To: References: <0e01d165-dda4-ed1d-66ad-bfa5c0d29c72@oracle.com> <215e4db8-b0ee-4913-9e71-88687b3cf507@oracle.com> <67e6308a-b1ab-3ffb-5c50-d21a948e894e@oracle.com> <03ce1be7-e727-b725-d7ac-e52338adecc7@oracle.com> Message-ID: <3d251a74-c50f-bd3b-64f1-6560aa5867fe@oracle.com> On 03/09/2018 16:26, Martin Thompson wrote: > Is that a direct ByteBuffer? A direct buffer or a region of memory allocated with Unsafe will do too. The former might be easier as it will be deallocated via the cleaner mechanism. > : > The posted results are for Linux (Ubuntu 18.04 LTS). I've also tested > on Windows 10 and the ratios are similar. > > I tested with -Djava.net.preferIPv4Stack=true. The single source case > is similar. The dual source case does see a reduction in latency of a > few hundred nanoseconds. There are a couple of additional field accesses in the JNI code for IPv6 addresses so this probably explains that. -Alan From Alan.Bateman at oracle.com Tue Sep 4 06:38:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Sep 2018 07:38:50 +0100 Subject: 8210341: (fs) Typos in PosixFileAttributeView javadoc Message-ID: <6040e14c-68ee-afb6-2ce4-13bf8d32111a@oracle.com> I need a Reviewer to fix two typos in the PosixFileAttributeView's javadoc. The proposed change is attached. -Alan diff --git a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java --- a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java +++ b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -50,7 +50,7 @@ ? * ? *

The {@link PosixFileAttributes#permissions() permissions} attribute is a ? * set of access permissions. This file attribute view provides access to the nine - * permission defined by the {@link PosixFilePermission} class. + * permission bits defined by the {@link PosixFilePermission} class. ? * These nine permission bits determine the read, write, and ? * execute access for the file owner, group, and others (others ? * meaning identities other than the owner and members of the group). Some @@ -126,7 +126,7 @@ ? * ? * ? *

When the access permissions are set at file creation time then the actual - * value of the permissions may differ that the value of the attribute object. + * value of the permissions may differ from the value of the attribute object. ? * The reasons for this are implementation specific. On UNIX systems, for ? * example, a process has a umask that impacts the permission bits ? * of newly created files. Where an implementation supports the setting of From daniel.fuchs at oracle.com Tue Sep 4 10:06:13 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 4 Sep 2018 11:06:13 +0100 Subject: 8210341: (fs) Typos in PosixFileAttributeView javadoc In-Reply-To: <6040e14c-68ee-afb6-2ce4-13bf8d32111a@oracle.com> References: <6040e14c-68ee-afb6-2ce4-13bf8d32111a@oracle.com> Message-ID: <44be6274-2a34-0a98-fecc-10967a5ebc0e@oracle.com> Looks good to me Alan. best regards, -- daniel On 04/09/2018 07:38, Alan Bateman wrote: > I need a Reviewer to fix two typos in the PosixFileAttributeView's > javadoc. The proposed change is attached. > > -Alan > > diff --git > a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java > b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java > > --- > a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java > > +++ > b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java > > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -50,7 +50,7 @@ > ? * > ? *

The {@link PosixFileAttributes#permissions() permissions} > attribute is a > ? * set of access permissions. This file attribute view provides access > to the nine > - * permission defined by the {@link PosixFilePermission} class. > + * permission bits defined by the {@link PosixFilePermission} class. > ? * These nine permission bits determine the read, > write, and > ? * execute access for the file owner, group, and others (others > ? * meaning identities other than the owner and members of the group). > Some > @@ -126,7 +126,7 @@ > ? * > ? * > ? *

When the access permissions are set at file creation time then > the actual > - * value of the permissions may differ that the value of the attribute > object. > + * value of the permissions may differ from the value of the attribute > object. > ? * The reasons for this are implementation specific. On UNIX systems, for > ? * example, a process has a umask that impacts the permission > bits > ? * of newly created files. Where an implementation supports the > setting of > From Alan.Bateman at oracle.com Tue Sep 4 18:12:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 4 Sep 2018 19:12:58 +0100 Subject: 8209152: (so) ServerSocketChannel::supportedOptions includes IP_TOS Message-ID: JDK-8209152 [1] is the issue about ServerSocketChannel::supportedOptions including IP_TOS in the set of supported options. Norman brought it up here a few weeks ago [2]. Digging into the history, it seems that changes in the networking area for JDK-8029607 [3] to fix the IPv6 traffic class handling in JDK 9 (and 8ux40 and 7uX) changed ServerSocketChannel. It's not a significant issue as ServerSocketChannel doesn't document this option and setting it on an listener socket isn't really interesting anyway (esp. when StandardSocketOption does not define the behavior of this socket option on stream oriented sockets). The changes to remove IP_TOS from the set are here: ?? http://cr.openjdk.java.net/~alanb/8209152/webrev/index.html I've included a test to query all socket options on the assumption that they can be queried on an unbound channel. This will at least catch cases where a channel claims to support a socket option that doesn't implement. The test restricts the set of platforms that it runs on because there are extended options, specific the SLA FLOW support on Solaris, that require special setup and aren't easy to exercise in tests that run on all platforms. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8209152 [2] http://mail.openjdk.java.net/pipermail/nio-dev/2018-August/005365.html [3] https://bugs.openjdk.java.net/browse/JDK-8029607 From chris.hegarty at oracle.com Wed Sep 5 10:05:40 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 5 Sep 2018 11:05:40 +0100 Subject: 8209152: (so) ServerSocketChannel::supportedOptions includes IP_TOS In-Reply-To: References: Message-ID: <05605a32-c229-91da-001b-b2eebca546a2@oracle.com> On 04/09/18 19:12, Alan Bateman wrote: > ... > The changes to remove IP_TOS from the set are here: > ?? http://cr.openjdk.java.net/~alanb/8209152/webrev/index.html This looks ok to me. -Chris. From claes.redestad at oracle.com Tue Sep 4 09:45:20 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 04 Sep 2018 11:45:20 +0200 Subject: 8210341: (fs) Typos in PosixFileAttributeView javadoc In-Reply-To: <6040e14c-68ee-afb6-2ce4-13bf8d32111a@oracle.com> References: <6040e14c-68ee-afb6-2ce4-13bf8d32111a@oracle.com> Message-ID: <6B11579B-811A-4FD0-AD3A-978237A37DAE@oracle.com> Looks good /Claes Alan Bateman skrev: (4 september 2018 08:38:50 CEST) >I need a Reviewer to fix two typos in the PosixFileAttributeView's >javadoc. The proposed change is attached. > >-Alan > >diff --git >a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java > >b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java >--- >a/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java >+++ >b/src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java >@@ -1,5 +1,5 @@ > ?/* >- * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights >reserved. >+ * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights >reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * >? * This code is free software; you can redistribute it and/or modify >it >@@ -50,7 +50,7 @@ > ? * > ? *

The {@link PosixFileAttributes#permissions() permissions} >attribute is a >? * set of access permissions. This file attribute view provides access > >to the nine >- * permission defined by the {@link PosixFilePermission} class. >+ * permission bits defined by the {@link PosixFilePermission} class. > ? * These nine permission bits determine the read, >write, and >? * execute access for the file owner, group, and others >(others >? * meaning identities other than the owner and members of the group). >Some >@@ -126,7 +126,7 @@ > ? * > ? * >? *

When the access permissions are set at file creation time then >the actual >- * value of the permissions may differ that the value of the attribute > >object. >+ * value of the permissions may differ from the value of the attribute > >object. >? * The reasons for this are implementation specific. On UNIX systems, >for >? * example, a process has a umask that impacts the permission > >bits >? * of newly created files. Where an implementation supports the >setting of -- Skickat fr?n min Android-enhet med K-9 Mail. Urs?kta min f?ordighet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Wed Sep 5 17:34:03 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 5 Sep 2018 17:34:03 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! ! f-8e62-2774-11c6-cc1221964b64@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F6FA5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F78A0@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1501607@ORSMSX113.amr.corp.intel.com> Hi Brian, Sorry for the late reply. I was just back from a conference trip :) Your changes look great! Thank you for the help! I am still looking at dlopen/dlsym approach for librdmacm. Will submit a patch this week. Do you want me to merge your changes into the next version of the patch? Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Friday, August 31, 2018 4:41 PM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, On Aug 29, 2018, at 3:13 PM, Lu, Yingqi > wrote: Thank you for the help!! Really nice to work with you again here :) Thanks - likewise! I will look into your comments and try to fix the issues. Together with changes following Alan's feedback, I will submit version 10 for review. I have a few more minor suggestions which I've uploaded as webrev versus version .09: http://cr.openjdk.java.net/~bpb/8195160/webrev.09-rev/ Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 5 17:55:23 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Sep 2018 10:55:23 -0700 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD1501607@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! ! ! f-8e62-2774-11c6-cc1221964b64@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F6FA5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F78A0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1501607@ORSMSX113.amr.corp.intel.com> Message-ID: Hi Lucy, On Sep 5, 2018, at 10:34 AM, Lu, Yingqi wrote: > Sorry for the late reply. I was just back from a conference trip J No worries: Monday was a holiday and I was off yesterday. > Your changes look great! Thank you for the help! You?re welcome. > I am still looking at dlopen/dlsym approach for librdmacm. Will submit a patch this week. Do you want me to merge your changes into the next version of the patch? No, don?t bother with that. I am planning to spend time today and tomorrow at least looking over version .09 so it would actually be simpler to leave it alone for now from my viewpoint. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Wed Sep 5 18:00:28 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 5 Sep 2018 18:00:28 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! ! ! f-8e62-2774-11c6-cc1221964b64@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F6FA5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14F78A0@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1501607@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1501748@ORSMSX113.amr.corp.intel.com> Hi Brian, Great!! I will wait for your feedback on version 09 before modifying anything then. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Wednesday, September 5, 2018 10:55 AM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, On Sep 5, 2018, at 10:34 AM, Lu, Yingqi > wrote: Sorry for the late reply. I was just back from a conference trip :) No worries: Monday was a holiday and I was off yesterday. Your changes look great! Thank you for the help! You're welcome. I am still looking at dlopen/dlsym approach for librdmacm. Will submit a patch this week. Do you want me to merge your changes into the next version of the patch? No, don't bother with that. I am planning to spend time today and tomorrow at least looking over version .09 so it would actually be simpler to leave it alone for now from my viewpoint. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjpt777 at gmail.com Thu Sep 6 19:42:30 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Thu, 6 Sep 2018 20:42:30 +0100 Subject: Absolute index bulk put/get byte operations on ByteBuffer Message-ID: There has been a long outstanding issue of ByteBuffer not having methods to get or put a ByteBuffer or byte[] at an absolute index. https://bugs.openjdk.java.net/browse/JDK-5029431 This seriously hampers the usefulness of ByteBuffer in a concurrent environment due to contention on the position and limit fields. Slicing or duplicating are not valid options are this are not threadsafe and they allocate. Is there any reason why this bug has not been progressed? Regards, Martin... -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Sep 7 10:43:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 7 Sep 2018 11:43:24 +0100 Subject: Absolute index bulk put/get byte operations on ByteBuffer In-Reply-To: References: Message-ID: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> On 06/09/2018 20:42, Martin Thompson wrote: > There has been a long outstanding issue of ByteBuffer not having > methods to get or put a ByteBuffer or byte[] at an absolute index. > > https://bugs.openjdk.java.net/browse/JDK-5029431 > > This seriously hampers the usefulness of ByteBuffer in a concurrent > environment due to contention on the position and limit fields. > Slicing or duplicating are not valid options are this are not > threadsafe and they allocate. > > Is there any reason why this bug has not been progressed? Many of the calls for absolute operations on ByteBuffers over the years have really been cases where someone wanted to operate on a region of memory and ended up using a direct buffer because they didn't have anything better. The buffer position/limit get in the way in many of these cases of course. As regards JDK-5029431 then I think these methods should have been added a long time ago but adding them now does beg the question as to how ByteBuffers will interact with memory regions or similar concept that Project Panama may introduce. There is also the question of addressing with large files (mapping regions > 2GB). Stuart might want to comment on this as I think he has concerns about adding further methods here until some of these other interactions are worked out. -Alan From mjpt777 at gmail.com Fri Sep 7 12:04:37 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Fri, 7 Sep 2018 13:04:37 +0100 Subject: Absolute index bulk put/get byte operations on ByteBuffer In-Reply-To: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> References: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> Message-ID: On Fri, 7 Sep 2018 at 11:43, Alan Bateman wrote: > On 06/09/2018 20:42, Martin Thompson wrote: > > There has been a long outstanding issue of ByteBuffer not having > > methods to get or put a ByteBuffer or byte[] at an absolute index. > > > > https://bugs.openjdk.java.net/browse/JDK-5029431 > > > > This seriously hampers the usefulness of ByteBuffer in a concurrent > > environment due to contention on the position and limit fields. > > Slicing or duplicating are not valid options are this are not > > threadsafe and they allocate. > > > > Is there any reason why this bug has not been progressed? > Many of the calls for absolute operations on ByteBuffers over the years > have really been cases where someone wanted to operate on a region of > memory and ended up using a direct buffer because they didn't have > anything better. The buffer position/limit get in the way in many of > these cases of course. > > As regards JDK-5029431 then I think these methods should have been added > a long time ago but adding them now does beg the question as to how > ByteBuffers will interact with memory regions or similar concept that > Project Panama may introduce. There is also the question of addressing > with large files (mapping regions > 2GB). Stuart might want to comment > on this as I think he has concerns about adding further methods here > until some of these other interactions are worked out. > I've tried to migrate a number of projects away from using Unsafe but issues like this keep hampering progress. When is Panama likely to deliver? Until then will Unsafe be available? It would be really nice to have a better alternative but until then it feels wrong to keep putting off fixing an immediate need for some solution that might come in the future. -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Sep 7 12:22:09 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 7 Sep 2018 14:22:09 +0200 (CEST) Subject: Absolute index bulk put/get byte operations on ByteBuffer In-Reply-To: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> References: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> Message-ID: <1647813034.1668886.1536322929534.JavaMail.zimbra@u-pem.fr> I wonder if it's not better to have a new copy mode on a VarHandle created from byteArrayViewVarHandle or byteBufferViewVarHandle with the same parameters as arraycopy ? R?mi ----- Mail original ----- > De: "Alan Bateman" > ?: "Martin Thompson" , "nio-dev" > Envoy?: Vendredi 7 Septembre 2018 12:43:24 > Objet: Re: Absolute index bulk put/get byte operations on ByteBuffer > On 06/09/2018 20:42, Martin Thompson wrote: >> There has been a long outstanding issue of ByteBuffer not having >> methods to get or put a ByteBuffer or byte[] at an absolute index. >> >> https://bugs.openjdk.java.net/browse/JDK-5029431 >> >> This seriously hampers the usefulness of ByteBuffer in a concurrent >> environment due to contention on the position and limit fields. >> Slicing or duplicating are not valid options are this are not >> threadsafe and they allocate. >> >> Is there any reason why this bug has not been progressed? > Many of the calls for absolute operations on ByteBuffers over the years > have really been cases where someone wanted to operate on a region of > memory and ended up using a direct buffer because they didn't have > anything better. The buffer position/limit get in the way in many of > these cases of course. > > As regards JDK-5029431 then I think these methods should have been added > a long time ago but adding them now does beg the question as to how > ByteBuffers will interact with memory regions or similar concept that > Project Panama may introduce. There is also the question of addressing > with large files (mapping regions > 2GB). Stuart might want to comment > on this as I think he has concerns about adding further methods here > until some of these other interactions are worked out. > > -Alan From stuart.marks at oracle.com Tue Sep 11 04:13:21 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 10 Sep 2018 21:13:21 -0700 Subject: Absolute index bulk put/get byte operations on ByteBuffer In-Reply-To: References: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> Message-ID: On 9/7/18 5:04 AM, Martin Thompson wrote: > I've tried to migrate a number of projects away from using Unsafe but issues > like this keep hampering progress. When is Panama likely to deliver? Until then > will Unsafe be available? > > It would be really nice to have a better alternative but until then it feels > wrong to keep putting off fixing an immediate need for some solution that might > come in the future. Hi Martin, Yes, I think it's reasonable to consider fixing some immediate needs instead of waiting for Panama. I'm confident there are some improvements we can make in the near term, and we can manage coordination with Panama. I don't think you need to worry about Unsafe going away before there is a suitable replacement. On that note, what kinds of issues have you run into trying to migrate away from Unsafe? There's the absolute indexing, and there's also the 2GB limit as Alan mentioned. I'm certain there are others. What I'd like to avoid is a situation where we add, say, absolute indexing to buffers, and then have the response be "thanks, that helps a little, but it really doesn't solve the problem without X, Y, and Z." I'd like to know more about X, Y, and Z. Thanks, s'marks From mjpt777 at gmail.com Wed Sep 12 12:07:42 2018 From: mjpt777 at gmail.com (Martin Thompson) Date: Wed, 12 Sep 2018 13:07:42 +0100 Subject: Absolute index bulk put/get byte operations on ByteBuffer In-Reply-To: References: <8a32f770-d89a-faa6-3e06-1f87b9584caf@oracle.com> Message-ID: On Tue, 11 Sep 2018 at 05:13, Stuart Marks wrote: > Yes, I think it's reasonable to consider fixing some immediate needs > instead of > waiting for Panama. I'm confident there are some improvements we can make > in the > near term, and we can manage coordination with Panama. > > I don't think you need to worry about Unsafe going away before there is a > suitable replacement. > > On that note, what kinds of issues have you run into trying to migrate > away from > Unsafe? There's the absolute indexing, and there's also the 2GB limit as > Alan > mentioned. I'm certain there are others. > > What I'd like to avoid is a situation where we add, say, absolute indexing > to > buffers, and then have the response be "thanks, that helps a little, but > it > really doesn't solve the problem without X, Y, and Z." I'd like to know > more > about X, Y, and Z. > Thanks Stuart, This is a good question. Yes in addition to what you mentioned there my be other things. I've had a quick scan and major things can be covered by Var Handles. However we still have limited production experience with Var Handles due to the lack of adoption of Java 9 and 10 as yet. The common thing I can find is Unsafe.setMemory which is mostly used for zero'ing out a buffer. Doing this via the existing API is a bit nasty requiring a loop. People often assume Unsafe is use to update buffers to avoid bounds checks and avoid the indirection. This is a minor reason. The main reason is to avoid allocation and the ability to use buffers in a concurrent fashion. Any future design should keep this in mind. Regards, Martin... -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 12 21:49:39 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 12 Sep 2018 14:49:39 -0700 Subject: adding rsockets support into JDK In-Reply-To: <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! f-8e62-2774-11c6-cc1221964b64@oracle.com> Message-ID: <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> Hi Lucy, On Aug 29, 2018, at 5:27 AM, Alan Bateman wrote: > I've skimmed through the implementation changes in the latest webrev and it looks reasonable. The implementation is currently in the rdma.ch tree and I assume you'll move that to a jdk.internal package at some point. Also it might be simpler to a simple "throws UOE" implementation in the shared directory - by this I mean the RmdaPollSelectorProvider and RdmaSocketImpl can be in the linux tree so that we aren't compiling all that code on other platforms. I?ve looked over the implementation. I did not delve into the low level details, but inspired by Alan?s comments I think some refactoring is needed. I?ve put some ideas together in the form of a webrev [1] which is generated with respect to your version 9 of the patch with my previous webrev [2] applied. This change would move all of the existing Rdma* classes out of the share tree into the linux tree. The implementation classes are also decoupled from the jdk.net.Sockets API via reflection. I verified that with this change the only Java files which are built on macOS are RdmaProviderFactory and RdmaSocketOptions. I also verified that the code builds and all the tests pass on Linux (Ubuntu 18.04 with the librdmacm-dev package installed). Note that none of this addresses the dlopen/dlsym issue which Alan raised. I added two packages here, jdk.net.internal and jdk.net.internal.rdma. I don?t know that these are the best package names to use. Assuming however that they are OK, then maybe you would want to move all the Java code currently in the rdma.ch package into jdk.net.internal.rdma in the linux-only tree. I am also wondering whether, assuming my changes in [1] or something similar were in place, some of the other reflection currently in use might be able to be removed and some of the classes consolidated. It seems that if checks are done up front in something like RdmaProviderFactory then they should not need to be redone elsewhere given that the reflection in this factory isolates the RDMA-specific code from the API points. (Note that in the RdmaProviderFactory static initializer there still needs to be added a reflective check of whether RDMA is available on the system.) Lastly, it looks like most of the tests were copied almost verbatim from existing tests and only slightly modified. I have not investigated it but I wonder if there might be some way to keep these things more consolidated. Please note that Alan has not seen [1] so it might be worthwhile to wait and see whether he has any comments before doing more work based on this. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-02/ [2] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Thu Sep 13 05:19:13 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Thu, 13 Sep 2018 05:19:13 +0000 Subject: adding rsockets support into JDK In-Reply-To: <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! f-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD150DDCD@ORSMSX113.amr.corp.intel.com> Hi Brian, Thank you very much for your time reviewing the code and implementing all the changes. While we are waiting for the comments from Alan and others, I will be working on cleaning up/consolidating the tests. This should be a pretty isolate task from the rest of the implementations. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Wednesday, September 12, 2018 2:50 PM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, On Aug 29, 2018, at 5:27 AM, Alan Bateman > wrote: I've skimmed through the implementation changes in the latest webrev and it looks reasonable. The implementation is currently in the rdma.ch tree and I assume you'll move that to a jdk.internal package at some point. Also it might be simpler to a simple "throws UOE" implementation in the shared directory - by this I mean the RmdaPollSelectorProvider and RdmaSocketImpl can be in the linux tree so that we aren't compiling all that code on other platforms. I've looked over the implementation. I did not delve into the low level details, but inspired by Alan's comments I think some refactoring is needed. I've put some ideas together in the form of a webrev [1] which is generated with respect to your version 9 of the patch with my previous webrev [2] applied. This change would move all of the existing Rdma* classes out of the share tree into the linux tree. The implementation classes are also decoupled from the jdk.net.Sockets API via reflection. I verified that with this change the only Java files which are built on macOS are RdmaProviderFactory and RdmaSocketOptions. I also verified that the code builds and all the tests pass on Linux (Ubuntu 18.04 with the librdmacm-dev package installed). Note that none of this addresses the dlopen/dlsym issue which Alan raised. I added two packages here, jdk.net.internal and jdk.net.internal.rdma. I don't know that these are the best package names to use. Assuming however that they are OK, then maybe you would want to move all the Java code currently in the rdma.ch package into jdk.net.internal.rdma in the linux-only tree. I am also wondering whether, assuming my changes in [1] or something similar were in place, some of the other reflection currently in use might be able to be removed and some of the classes consolidated. It seems that if checks are done up front in something like RdmaProviderFactory then they should not need to be redone elsewhere given that the reflection in this factory isolates the RDMA-specific code from the API points. (Note that in the RdmaProviderFactory static initializer there still needs to be added a reflective check of whether RDMA is available on the system.) Lastly, it looks like most of the tests were copied almost verbatim from existing tests and only slightly modified. I have not investigated it but I wonder if there might be some way to keep these things more consolidated. Please note that Alan has not seen [1] so it might be worthwhile to wait and see whether he has any comments before doing more work based on this. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-02/ [2] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Sep 14 09:41:19 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Sep 2018 10:41:19 +0100 Subject: 8208780: (se) test SelectWithConsumer.testReadableAndWriteable(): failure Message-ID: <0da5c592-1152-e160-b46b-de5769285c0a@oracle.com> This is an intermittent failure with the testReadableAndWriteable test in Selector/SelectWithConsumer.java. The test needs to check that the action is invoked to notify the consumer that the channel is both readable and writeable. In the test, the channel is ready for writing for a brief period before it also becomes ready for reading. The test needs to ensure the channel is ready for all expected operations before it tests the select(Consumer) methods. The proposed patch to the test is here: ?? http://cr.openjdk.java.net/~alanb/8208780/webrev/index.html -Alan From brian.burkhalter at oracle.com Fri Sep 14 15:24:59 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 14 Sep 2018 08:24:59 -0700 Subject: 8208780: (se) test SelectWithConsumer.testReadableAndWriteable(): failure In-Reply-To: <0da5c592-1152-e160-b46b-de5769285c0a@oracle.com> References: <0da5c592-1152-e160-b46b-de5769285c0a@oracle.com> Message-ID: This looks fine. Reviewed. Brian On Sep 14, 2018, at 2:41 AM, Alan Bateman wrote: > This is an intermittent failure with the testReadableAndWriteable test in Selector/SelectWithConsumer.java. [?] The proposed patch to the test is here: > http://cr.openjdk.java.net/~alanb/8208780/webrev/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri Sep 14 15:34:23 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 14 Sep 2018 08:34:23 -0700 Subject: 8210741: Typo in Java API documentation of java.nio.file.Paths Message-ID: <6A321659-59F9-4A93-8958-04C1EC1EB1A2@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8210741 --- a/src/java.base/share/classes/java/nio/file/Paths.java +++ b/src/java.base/share/classes/java/nio/file/Paths.java @@ -73,7 +73,8 @@ * Converts the given URI to a {@link Path} object. * * @implSpec - * This method simply invokes {@link Path#of(URI) * Path.of(URI)} with the given parameter. + * This method simply invokes {@link Path#of(URI) Path.of(URI)} with the + * given parameter. * * @param uri * the URI to convert Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger.riggs at oracle.com Fri Sep 14 15:37:57 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 14 Sep 2018 11:37:57 -0400 Subject: 8210741: Typo in Java API documentation of java.nio.file.Paths In-Reply-To: <6A321659-59F9-4A93-8958-04C1EC1EB1A2@oracle.com> References: <6A321659-59F9-4A93-8958-04C1EC1EB1A2@oracle.com> Message-ID: +1 On 9/14/18 11:34 AM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8210741 > > --- a/src/java.base/share/classes/java/nio/file/Paths.java > +++ b/src/java.base/share/classes/java/nio/file/Paths.java > @@ -73,7 +73,8 @@ > ? ? * Converts the given URI to a {@link Path} object. > ? ? * > ? ? * @implSpec > - ? ? * This method simply invokes {@link Path#of(URI) * Path.of(URI)} > with the given parameter. > + ? ? * This method simply invokes {@link Path#of(URI) Path.of(URI)} > with the > + ? ? * given parameter. > ? ? * > ? ? * @param ? uri > ? ? *? ? ? ? ? the URI to convert > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Sep 14 15:39:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 14 Sep 2018 16:39:41 +0100 Subject: 8210741: Typo in Java API documentation of java.nio.file.Paths In-Reply-To: <6A321659-59F9-4A93-8958-04C1EC1EB1A2@oracle.com> References: <6A321659-59F9-4A93-8958-04C1EC1EB1A2@oracle.com> Message-ID: On 14/09/2018 16:34, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8210741 > > --- a/src/java.base/share/classes/java/nio/file/Paths.java > +++ b/src/java.base/share/classes/java/nio/file/Paths.java > @@ -73,7 +73,8 @@ > ? ? * Converts the given URI to a {@link Path} object. > ? ? * > ? ? * @implSpec > - ? ? * This method simply invokes {@link Path#of(URI) * Path.of(URI)} > with the given parameter. > + ? ? * This method simply invokes {@link Path#of(URI) Path.of(URI)} > with the > + ? ? * given parameter. > ? ? * > Looks fine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Fri Sep 14 20:29:37 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 14 Sep 2018 20:29:37 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD150DDCD@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD13CE4C7@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1466292@ORSMSX113.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952a! f-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD150DDCD@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD15103FF@ORSMSX113.amr.corp.intel.com> Hi Alan, Do you have any feedback on the approach Brian is currently drafting? Details are in the following email. I would like to gather your feedback before continuing working on the implementation. Thanks, Lucy From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, Yingqi Sent: Wednesday, September 12, 2018 10:19 PM To: Brian Burkhalter Cc: Aundhe, Shirish ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Kaczmarek, Eric Subject: RE: adding rsockets support into JDK Hi Brian, Thank you very much for your time reviewing the code and implementing all the changes. While we are waiting for the comments from Alan and others, I will be working on cleaning up/consolidating the tests. This should be a pretty isolate task from the rest of the implementations. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Wednesday, September 12, 2018 2:50 PM To: Lu, Yingqi > Cc: Alan Bateman >; nio-dev at openjdk.java.net; Viswanathan, Sandhya >; Aundhe, Shirish >; Kaczmarek, Eric > Subject: Re: adding rsockets support into JDK Hi Lucy, On Aug 29, 2018, at 5:27 AM, Alan Bateman > wrote: I've skimmed through the implementation changes in the latest webrev and it looks reasonable. The implementation is currently in the rdma.ch tree and I assume you'll move that to a jdk.internal package at some point. Also it might be simpler to a simple "throws UOE" implementation in the shared directory - by this I mean the RmdaPollSelectorProvider and RdmaSocketImpl can be in the linux tree so that we aren't compiling all that code on other platforms. I've looked over the implementation. I did not delve into the low level details, but inspired by Alan's comments I think some refactoring is needed. I've put some ideas together in the form of a webrev [1] which is generated with respect to your version 9 of the patch with my previous webrev [2] applied. This change would move all of the existing Rdma* classes out of the share tree into the linux tree. The implementation classes are also decoupled from the jdk.net.Sockets API via reflection. I verified that with this change the only Java files which are built on macOS are RdmaProviderFactory and RdmaSocketOptions. I also verified that the code builds and all the tests pass on Linux (Ubuntu 18.04 with the librdmacm-dev package installed). Note that none of this addresses the dlopen/dlsym issue which Alan raised. I added two packages here, jdk.net.internal and jdk.net.internal.rdma. I don't know that these are the best package names to use. Assuming however that they are OK, then maybe you would want to move all the Java code currently in the rdma.ch package into jdk.net.internal.rdma in the linux-only tree. I am also wondering whether, assuming my changes in [1] or something similar were in place, some of the other reflection currently in use might be able to be removed and some of the classes consolidated. It seems that if checks are done up front in something like RdmaProviderFactory then they should not need to be redone elsewhere given that the reflection in this factory isolates the RDMA-specific code from the API points. (Note that in the RdmaProviderFactory static initializer there still needs to be added a reflective check of whether RDMA is available on the system.) Lastly, it looks like most of the tests were copied almost verbatim from existing tests and only slightly modified. I have not investigated it but I wonder if there might be some way to keep these things more consolidated. Please note that Alan has not seen [1] so it might be worthwhile to wait and see whether he has any comments before doing more work based on this. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-02/ [2] http://cr.openjdk.java.net/~bpb/8195160/webrev.09-01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sat Sep 15 14:38:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 15 Sep 2018 15:38:42 +0100 Subject: adding rsockets support into JDK In-Reply-To: <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> Message-ID: <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> On 12/09/2018 22:49, Brian Burkhalter wrote: > : > > This change would move all of the existing Rdma* classes out of the > share tree into the linux tree. The implementation classes are also > decoupled from the jdk.net.Sockets API via reflection. I verified that > with this change the only Java files which are built on macOS are > RdmaProviderFactory and RdmaSocketOptions. I also verified that the > code builds and all the tests pass on Linux (Ubuntu 18.04 with > the?librdmacm-dev package installed). Note that none of this addresses > the dlopen/dlsym issue which Alan raised. > > I added two packages here, jdk.net.internal and jdk.net.internal.rdma. > I don?t know that these are the best package names to use. Assuming > however that they are OK, then maybe you would want to move all the > Java code currently in the rdma.ch package into > jdk.net.internal.rdma in the linux-only tree. > > I am also wondering whether, assuming my changes in [1] or something > similar were in place, some of the other reflection currently in use > might be able to be removed and some of the classes consolidated. It > seems that if checks are done up front in something like > RdmaProviderFactory then they should not need to be redone elsewhere > given that the reflection in this factory isolates the RDMA-specific > code from the API points. (Note that in the RdmaProviderFactory static > initializer there still needs to be added a reflective check of > whether RDMA is available on the system.) > Lucy's mail reminded me to comment on this. No objection to introducing jdk.net.internal.rdma.{SocketFactory,RdmaSocketFactoryRdmaProviderFactory} but it still means some ugly reflection in rdmaSocketProvider() to decide whether to return a provider or throw UOE. If you want something simpler just create a RdmaSocketImpl in shared code where the constructor throws UOE. The build will compile in this version on non-Linux platforms. We have a couple of other examples where platform specific classes shadow same named classes in share/classes. jdk.internal.net.rdma seem okay to me for the package name (the webrev had jdk.net.internal.rmda which I assume wasn't intended). -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Sun Sep 16 07:06:00 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 16 Sep 2018 00:06:00 -0700 Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip file is located in a custom file system Message-ID: <5B9E00D8.8020505@oracle.com> Hi Please help review the change for JDK-8034802. issue: https://bugs.openjdk.java.net/browse/JDK-8034802 webrev: http://cr.openjdk.java.net/~sherman/8034802/webrev One of the reasons that the implementation works this way is that I didn't have a complete SeekableByteChannel support (mainly the position(...)) from newByteChannel() back then (so you really can't get a ZipFileSystem from a jar/zip file inside a zfs :-)) So this changeset also includes a ByteArrayChannel which implements SBC and has better position()/position(int) support, and now we can have a zipfs from a zip file inside a zipfs. Thanks, Sherman From christoph.langer at sap.com Mon Sep 17 14:30:17 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 17 Sep 2018 14:30:17 +0000 Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip file is located in a custom file system In-Reply-To: <5B9E00D8.8020505@oracle.com> References: <5B9E00D8.8020505@oracle.com> Message-ID: <4ed47defaea74d7a95f58e44e9c58fa1@sap.com> Hi Sherman, as I'm currently looking into zip stuff as well, it's a good exercise to review your changeset. Overall it looks good. I found a few nits mostly in the area of spelling and whitespace ?? src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java: 88 ZipFileSystem(ZipFileSystemProvider provider, I think, the constructor should initialize these items as last statements: 92 this.provider = provider; 93 this.zfpath = zfpath; As per section 7-3 of the security guide which should probably apply here: https://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 132 // returns ture if there is a name=true/"true" setting in env -> spelling: // returns true if there is a name=true/"true" setting in env 282 // and sync dose not close the ch -> spelling: // and sync did not close the ch 587 // (1) writing the contents of a new entry, if the entry doesn't exits, or -> spelling: // (1) writing the contents of a new entry, if the entry doesn't exist, or 1471 } 1472 else { // untouced CEN or COPY -> put brace and else on the same line to match style above and spelling of "untouched": } else { // untouched CEN or COPY remove these 2 lines that have been commented out: 1881 // Entry(byte[] name, boolean isdir) { 1889 // this.method = METHOD_DEFLTED; 2402 fm.format(" name : %s%n", new String(name)); 2403 fm.format(" creationTime : %tc%n", creationTime().toMillis()); -> I would align name with the start of creationTime (and the other lines below) test/jdk/jdk/nio/zipfs/ZipFSTester.java: 134 try ( OutputStream os = Files.newOutputStream(src)) { -> the blank between ( and OutputStream could be removed 436 private static void checkRead(Path path, byte[] expected) throws IOException -> should take brace from line 437 on the same line, line length is not too long yet 438 //streams -> insert a blank between // and streams 447 try (SeekableByteChannel sbc = Files.newByteChannel(path); insert one more space before try to have correct indentation src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java 43 * The currently position of this channel. -> The current position of this channel. 97 throw new IllegalArgumentException("negative position " + pos); -> shouldn't it rather be "illegal position " as it must not necessarily be a negative position? Assuming that the updated test case runs correctly, you can consider the changes reviewed from my end. I'll post a change for adding executable bit support soon, based on your changes. Best regards Christoph > -----Original Message----- > From: nio-dev On Behalf Of Xueming > Shen > Sent: Sonntag, 16. September 2018 09:06 > To: core-libs-dev at openjdk.java.net; nio-dev > Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip > file is located in a custom file system > > Hi > > Please help review the change for JDK-8034802. > > issue: https://bugs.openjdk.java.net/browse/JDK-8034802 > webrev: http://cr.openjdk.java.net/~sherman/8034802/webrev > > One of the reasons that the implementation works this way is that I > didn't have a > complete SeekableByteChannel support (mainly the position(...)) from > newByteChannel() > back then (so you really can't get a ZipFileSystem from a jar/zip file > inside a zfs :-)) > > So this changeset also includes a ByteArrayChannel which implements SBC > and has > better position()/position(int) support, and now we can have a zipfs > from a zip file > inside a zipfs. > > Thanks, > Sherman > From xueming.shen at oracle.com Mon Sep 17 17:51:05 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 17 Sep 2018 10:51:05 -0700 Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip file is located in a custom file system In-Reply-To: <4ed47defaea74d7a95f58e44e9c58fa1@sap.com> References: <5B9E00D8.8020505@oracle.com> <4ed47defaea74d7a95f58e44e9c58fa1@sap.com> Message-ID: <5B9FE989.20509@oracle.com> Thanks! webrev has been updated accordingly. http://cr.openjdk.java.net/~sherman/8034802/webrev/ -Sherman On 09/17/2018 07:30 AM, Langer, Christoph wrote: > Hi Sherman, > > as I'm currently looking into zip stuff as well, it's a good exercise to review your changeset. Overall it looks good. I found a few nits mostly in the area of spelling and whitespace ?? > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java: > > 88 ZipFileSystem(ZipFileSystemProvider provider, > I think, the constructor should initialize these items as last statements: > 92 this.provider = provider; > 93 this.zfpath = zfpath; > As per section 7-3 of the security guide which should probably apply here: > https://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 > > 132 // returns ture if there is a name=true/"true" setting in env > -> spelling: // returns true if there is a name=true/"true" setting in env > > 282 // and sync dose not close the ch > -> spelling: // and sync did not close the ch > > 587 // (1) writing the contents of a new entry, if the entry doesn't exits, or > -> spelling: // (1) writing the contents of a new entry, if the entry doesn't exist, or > > 1471 } > 1472 else { // untouced CEN or COPY > -> put brace and else on the same line to match style above and spelling of "untouched": } else { // untouched CEN or COPY > > remove these 2 lines that have been commented out: > 1881 // Entry(byte[] name, boolean isdir) { > 1889 // this.method = METHOD_DEFLTED; > > 2402 fm.format(" name : %s%n", new String(name)); > 2403 fm.format(" creationTime : %tc%n", creationTime().toMillis()); > -> I would align name with the start of creationTime (and the other lines below) > > > > test/jdk/jdk/nio/zipfs/ZipFSTester.java: > 134 try ( OutputStream os = Files.newOutputStream(src)) { > -> the blank between ( and OutputStream could be removed > > 436 private static void checkRead(Path path, byte[] expected) throws IOException > -> should take brace from line 437 on the same line, line length is not too long yet > > 438 //streams > -> insert a blank between // and streams > > 447 try (SeekableByteChannel sbc = Files.newByteChannel(path); > insert one more space before try to have correct indentation > > > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java > 43 * The currently position of this channel. > -> The current position of this channel. > > 97 throw new IllegalArgumentException("negative position " + pos); > -> shouldn't it rather be "illegal position " as it must not necessarily be a negative position? > > Assuming that the updated test case runs correctly, you can consider the changes reviewed from my end. > > I'll post a change for adding executable bit support soon, based on your changes. > > Best regards > Christoph > >> -----Original Message----- >> From: nio-dev On Behalf Of Xueming >> Shen >> Sent: Sonntag, 16. September 2018 09:06 >> To: core-libs-dev at openjdk.java.net; nio-dev >> Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip >> file is located in a custom file system >> >> Hi >> >> Please help review the change for JDK-8034802. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8034802 >> webrev: http://cr.openjdk.java.net/~sherman/8034802/webrev >> >> One of the reasons that the implementation works this way is that I >> didn't have a >> complete SeekableByteChannel support (mainly the position(...)) from >> newByteChannel() >> back then (so you really can't get a ZipFileSystem from a jar/zip file >> inside a zfs :-)) >> >> So this changeset also includes a ByteArrayChannel which implements SBC >> and has >> better position()/position(int) support, and now we can have a zipfs >> from a zip file >> inside a zipfs. >> >> Thanks, >> Sherman >> From brian.burkhalter at oracle.com Mon Sep 17 18:01:08 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 17 Sep 2018 11:01:08 -0700 Subject: adding rsockets support into JDK In-Reply-To: <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> Message-ID: On Sep 15, 2018, at 7:38 AM, Alan Bateman wrote: > No objection to introducing jdk.net.internal.rdma.{SocketFactory,RdmaSocketFactoryRdmaProviderFactory} but it still means some ugly reflection in rdmaSocketProvider() to decide whether to return a provider or throw UOE. If you want something simpler just create a RdmaSocketImpl in shared code where the constructor throws UOE. The build will compile in this version on non-Linux platforms. We have a couple of other examples where platform specific classes shadow same named classes in share/classes. I?ll look into updating my most recent prototype. > jdk.internal.net.rdma seem okay to me for the package name (the webrev had jdk.net.internal.rmda which I assume wasn't intended). Will change this also. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fenix_young at hotmail.com Tue Sep 18 05:22:34 2018 From: fenix_young at hotmail.com (Young xiaofeng) Date: Tue, 18 Sep 2018 05:22:34 +0000 Subject: =?utf-8?B?562U5aSNOiBSRlIgSkRLLTgwMzQ4MDI6ICh6aXBmcykgbmV3RmlsZVN5c3Rl?= =?utf-8?B?bSB0aHJvd3MgVU9FIHdoZW4gdGhlIHppcCBmaWxlIGlzIGxvY2F0ZWQgaW4g?= =?utf-8?Q?a_custom_file_system?= In-Reply-To: <5B9FE989.20509@oracle.com> References: <5B9E00D8.8020505@oracle.com> <4ed47defaea74d7a95f58e44e9c58fa1@sap.com>,<5B9FE989.20509@oracle.com> Message-ID: Hi Sherman, just one comment, the copyright dates in ZipFileSystem.java, ZipFileSystemProvider.java and ZipFSTester.java are not updated. -Felix ________________________________ ???: core-libs-dev ?? Xueming Shen ????: 2018?9?17? 17:51:05 ???: Langer, Christoph ??: nio-dev; core-libs-dev at openjdk.java.net ??: Re: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip file is located in a custom file system Thanks! webrev has been updated accordingly. http://cr.openjdk.java.net/~sherman/8034802/webrev/ -Sherman On 09/17/2018 07:30 AM, Langer, Christoph wrote: > Hi Sherman, > > as I'm currently looking into zip stuff as well, it's a good exercise to review your changeset. Overall it looks good. I found a few nits mostly in the area of spelling and whitespace ?? > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java: > > 88 ZipFileSystem(ZipFileSystemProvider provider, > I think, the constructor should initialize these items as last statements: > 92 this.provider = provider; > 93 this.zfpath = zfpath; > As per section 7-3 of the security guide which should probably apply here: > https://www.oracle.com/technetwork/java/seccodeguide-139067.html#7 > > 132 // returns ture if there is a name=true/"true" setting in env > -> spelling: // returns true if there is a name=true/"true" setting in env > > 282 // and sync dose not close the ch > -> spelling: // and sync did not close the ch > > 587 // (1) writing the contents of a new entry, if the entry doesn't exits, or > -> spelling: // (1) writing the contents of a new entry, if the entry doesn't exist, or > > 1471 } > 1472 else { // untouced CEN or COPY > -> put brace and else on the same line to match style above and spelling of "untouched": } else { // untouched CEN or COPY > > remove these 2 lines that have been commented out: > 1881 // Entry(byte[] name, boolean isdir) { > 1889 // this.method = METHOD_DEFLTED; > > 2402 fm.format(" name : %s%n", new String(name)); > 2403 fm.format(" creationTime : %tc%n", creationTime().toMillis()); > -> I would align name with the start of creationTime (and the other lines below) > > > > test/jdk/jdk/nio/zipfs/ZipFSTester.java: > 134 try ( OutputStream os = Files.newOutputStream(src)) { > -> the blank between ( and OutputStream could be removed > > 436 private static void checkRead(Path path, byte[] expected) throws IOException > -> should take brace from line 437 on the same line, line length is not too long yet > > 438 //streams > -> insert a blank between // and streams > > 447 try (SeekableByteChannel sbc = Files.newByteChannel(path); > insert one more space before try to have correct indentation > > > > src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java > 43 * The currently position of this channel. > -> The current position of this channel. > > 97 throw new IllegalArgumentException("negative position " + pos); > -> shouldn't it rather be "illegal position " as it must not necessarily be a negative position? > > Assuming that the updated test case runs correctly, you can consider the changes reviewed from my end. > > I'll post a change for adding executable bit support soon, based on your changes. > > Best regards > Christoph > >> -----Original Message----- >> From: nio-dev On Behalf Of Xueming >> Shen >> Sent: Sonntag, 16. September 2018 09:06 >> To: core-libs-dev at openjdk.java.net; nio-dev >> Subject: RFR JDK-8034802: (zipfs) newFileSystem throws UOE when the zip >> file is located in a custom file system >> >> Hi >> >> Please help review the change for JDK-8034802. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8034802 >> webrev: http://cr.openjdk.java.net/~sherman/8034802/webrev >> >> One of the reasons that the implementation works this way is that I >> didn't have a >> complete SeekableByteChannel support (mainly the position(...)) from >> newByteChannel() >> back then (so you really can't get a ZipFileSystem from a jar/zip file >> inside a zfs :-)) >> >> So this changeset also includes a ByteArrayChannel which implements SBC >> and has >> better position()/position(int) support, and now we can have a zipfs >> from a zip file >> inside a zipfs. >> >> Thanks, >> Sherman >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Sep 18 17:27:32 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 18 Sep 2018 10:27:32 -0700 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> Message-ID: Hello, I updated the prototype changes I previously posted. The webrev versus version 9 of the patch is here: http://cr.openjdk.java.net/~bpb/8195160/webrev.09-03/ This should be able to be applied immediately on top of version 9. It builds on Linux and macOS and all the new tests pass on Linux. The main changes are: 1. rdma.ch package refactored to jdk.internal.net.rdma. 2. jdk.net.internal.rdma refactored to jdk.internal.net.rdma. 3. Reflection replaced with shadow classes. It might not hurt to add a test which verifies that the new jdk.net.Sockets methods all throw UOE on non-Linux platforms. Thanks, Brian On Sep 17, 2018, at 11:01 AM, Brian Burkhalter wrote: > On Sep 15, 2018, at 7:38 AM, Alan Bateman wrote: > >> No objection to introducing jdk.net.internal.rdma.{SocketFactory,RdmaSocketFactoryRdmaProviderFactory} but it still means some ugly reflection in rdmaSocketProvider() to decide whether to return a provider or throw UOE. If you want something simpler just create a RdmaSocketImpl in shared code where the constructor throws UOE. The build will compile in this version on non-Linux platforms. We have a couple of other examples where platform specific classes shadow same named classes in share/classes. > > I?ll look into updating my most recent prototype. > >> jdk.internal.net.rdma seem okay to me for the package name (the webrev had jdk.net.internal.rmda which I assume wasn't intended). > > Will change this also. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Tue Sep 18 17:38:47 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 18 Sep 2018 17:38:47 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <6a1355f5-136e-e033-5d35-5debac0cb088@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD146AE6C@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1514FC4@ORSMSX113.amr.corp.intel.com> Hi Brian, Thank you very much for the help here! I will apply your patch and try it here. I am currently working on consolidating tests. I will add one specifically for non-Linux platforms to make sure they throw UOE on RDMA methods. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Tuesday, September 18, 2018 10:28 AM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hello, I updated the prototype changes I previously posted. The webrev versus version 9 of the patch is here: http://cr.openjdk.java.net/~bpb/8195160/webrev.09-03/ This should be able to be applied immediately on top of version 9. It builds on Linux and macOS and all the new tests pass on Linux. The main changes are: 1. rdma.ch package refactored to jdk.internal.net.rdma. 2. jdk.net.internal.rdma refactored to jdk.internal.net.rdma. 3. Reflection replaced with shadow classes. It might not hurt to add a test which verifies that the new jdk.net.Sockets methods all throw UOE on non-Linux platforms. Thanks, Brian On Sep 17, 2018, at 11:01 AM, Brian Burkhalter > wrote: On Sep 15, 2018, at 7:38 AM, Alan Bateman > wrote: No objection to introducing jdk.net.internal.rdma.{SocketFactory,RdmaSocketFactoryRdmaProviderFactory} but it still means some ugly reflection in rdmaSocketProvider() to decide whether to return a provider or throw UOE. If you want something simpler just create a RdmaSocketImpl in shared code where the constructor throws UOE. The build will compile in this version on non-Linux platforms. We have a couple of other examples where platform specific classes shadow same named classes in share/classes. I'll look into updating my most recent prototype. jdk.internal.net.rdma seem okay to me for the package name (the webrev had jdk.net.internal.rmda which I assume wasn't intended). Will change this also. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Sep 18 20:14:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 18 Sep 2018 13:14:21 -0700 Subject: 8210817: Minor typo in java.nio.file.attribute package summary Message-ID: <92C59CFD-4680-4E06-8A27-58018F127119@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8210817 --- a/src/java.base/share/classes/java/nio/file/attribute/package-info.java +++ b/src/java.base/share/classes/java/nio/file/attribute/package-info.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -93,7 +93,7 @@ *

The {@link java.nio.file.attribute.DosFileAttributeView} * class extends {@code BasicFileAttributeView} by defining methods to * access the legacy "DOS" file attributes supported on file systems such as File - * Allocation Tabl (FAT), commonly used in consumer devices. + * Allocation Table (FAT), commonly used in consumer devices. * *

The {@link java.nio.file.attribute.AclFileAttributeView} * class defines methods to read and write the Access Control List (ACL) Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Tue Sep 18 20:16:10 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 18 Sep 2018 21:16:10 +0100 Subject: 8210817: Minor typo in java.nio.file.attribute package summary In-Reply-To: <92C59CFD-4680-4E06-8A27-58018F127119@oracle.com> References: <92C59CFD-4680-4E06-8A27-58018F127119@oracle.com> Message-ID: Looks good Brian, cheers, -- daniel On 18/09/18 21:14, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8210817 > > --- a/src/java.base/share/classes/java/nio/file/attribute/package-info.java > +++ b/src/java.base/share/classes/java/nio/file/attribute/package-info.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -93,7 +93,7 @@ > ? *

The {@link java.nio.file.attribute.DosFileAttributeView} > ? * class extends {@code BasicFileAttributeView} by defining methods to > ? * access the legacy "DOS" file attributes supported on file systems > such as File > - * Allocation Tabl (FAT), commonly used in consumer devices. > + * Allocation Table (FAT), commonly used in consumer devices. > ? * > ? *

The {@link java.nio.file.attribute.AclFileAttributeView} > ? * class defines methods to read and write the Access Control List (ACL) > > Thanks, > > Brian From Alan.Bateman at oracle.com Tue Sep 18 20:16:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 18 Sep 2018 21:16:22 +0100 Subject: 8210817: Minor typo in java.nio.file.attribute package summary In-Reply-To: <92C59CFD-4680-4E06-8A27-58018F127119@oracle.com> References: <92C59CFD-4680-4E06-8A27-58018F127119@oracle.com> Message-ID: <1563dc0e-a50f-d13d-83df-6cbfe76f2c64@oracle.com> Looks good. On 18/09/2018 21:14, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8210817 > > --- > a/src/java.base/share/classes/java/nio/file/attribute/package-info.java > +++ > b/src/java.base/share/classes/java/nio/file/attribute/package-info.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2007, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -93,7 +93,7 @@ > *

The {@link java.nio.file.attribute.DosFileAttributeView} > * class extends {@code BasicFileAttributeView} by defining methods to > * access the legacy "DOS" file attributes supported on file systems > such as File > - * Allocation Tabl (FAT), commonly used in consumer devices. > + * Allocation Table (FAT), commonly used in consumer devices. > * > *

The {@link java.nio.file.attribute.AclFileAttributeView} > * class defines methods to read and write the Access Control List (ACL) > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Wed Sep 19 00:14:30 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 18 Sep 2018 17:14:30 -0700 Subject: RFR JDK-8210899: (zipfs) ZipFileSystem.EntryOutputStreamCRC32 mistakenly set the crc32 value into size field Message-ID: <5BA194E6.5000600@oracle.com> Hi, Please help codereview the change for JDK-8210899. issue: https://bugs.openjdk.java.net/browse/JDK-8210899 webrev: http://cr.openjdk.java.net/~sherman/8210899/webrev Thanks, Sherman From brian.burkhalter at oracle.com Wed Sep 19 00:50:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 18 Sep 2018 17:50:21 -0700 Subject: RFR JDK-8210899: (zipfs) ZipFileSystem.EntryOutputStreamCRC32 mistakenly set the crc32 value into size field In-Reply-To: <5BA194E6.5000600@oracle.com> References: <5BA194E6.5000600@oracle.com> Message-ID: <851D0181-5EF1-4875-B2A2-5DE4375CF76E@oracle.com> Hi Sherman, This looks fine to me. Does the current code fail both of the added checks in the test? Thanks, Brian On Sep 18, 2018, at 5:14 PM, Xueming Shen wrote: > Please help codereview the change for JDK-8210899. > > issue: https://bugs.openjdk.java.net/browse/JDK-8210899 > webrev: http://cr.openjdk.java.net/~sherman/8210899/webrev -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Wed Sep 19 01:45:36 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 18 Sep 2018 18:45:36 -0700 Subject: RFR JDK-8210899: (zipfs) ZipFileSystem.EntryOutputStreamCRC32 mistakenly set the crc32 value into size field In-Reply-To: <851D0181-5EF1-4875-B2A2-5DE4375CF76E@oracle.com> References: <5BA194E6.5000600@oracle.com> <851D0181-5EF1-4875-B2A2-5DE4375CF76E@oracle.com> Message-ID: <5BA1AA40.4090803@oracle.com> On 9/18/18, 5:50 PM, Brian Burkhalter wrote: > Hi Sherman, > > This looks fine to me. > > Does the current code fail both of the added checks in the test? Hi Brian, Thanks looking into it. Yes, the current code (with the change for JDK-8034802) fails both of the added checks. Thanks, Shermzn > > Thanks, > > Brian > > On Sep 18, 2018, at 5:14 PM, Xueming Shen > wrote: > >> Please help codereview the change for JDK-8210899. >> >> issue:https://bugs.openjdk.java.net/browse/JDK-8210899 >> webrev:http://cr.openjdk.java.net/~sherman/8210899/webrev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 19 14:34:34 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Sep 2018 07:34:34 -0700 Subject: RFR JDK-8210899: (zipfs) ZipFileSystem.EntryOutputStreamCRC32 mistakenly set the crc32 value into size field In-Reply-To: <5BA1AA40.4090803@oracle.com> References: <5BA194E6.5000600@oracle.com> <851D0181-5EF1-4875-B2A2-5DE4375CF76E@oracle.com> <5BA1AA40.4090803@oracle.com> Message-ID: <7EDAC38A-B7A6-434B-879E-7EC303EB745F@oracle.com> Hi Sherman On Sep 18, 2018, at 6:45 PM, Xueming Shen wrote: > Thanks looking into it. You?re welcome. > Yes, the current code (with the change for JDK-8034802) fails both of the added > checks. Thanks for the update. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Sep 19 15:54:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Sep 2018 16:54:20 +0100 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> Message-ID: <0f35e51e-8297-2fe1-fd6f-90d5c464e6bc@oracle.com> On 18/09/2018 18:27, Brian Burkhalter wrote: > Hello, > > I updated the prototype changes I previously posted. The webrev versus > version 9 of the patch is here: > > http://cr.openjdk.java.net/~bpb/8195160/webrev.09-03/ > > > This should be able to be applied immediately on top of version 9. It > builds on Linux and macOS and all the new tests pass on Linux. > > The main changes are: > > 1. rdma.ch package refactored to jdk.internal.net.rdma. > 2. jdk.net.internal.rdma refactored to jdk.internal.net.rdma. > 3. Reflection replaced with shadow classes. > Thanks for taking up the suggestion to have two XXXProvider implementations, I think it looks much cleaner now. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 19 16:00:00 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Sep 2018 09:00:00 -0700 Subject: adding rsockets support into JDK In-Reply-To: <0f35e51e-8297-2fe1-fd6f-90d5c464e6bc@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! -90d5c464e6bc@oracle.com> Message-ID: On Sep 19, 2018, at 8:54 AM, Alan Bateman wrote: >> 3. Reflection replaced with shadow classes. >> > Thanks for taking up the suggestion to have two XXXProvider implementations, I think it looks much cleaner now. You?re welcome. I had not been aware of that possibility: definitely better not to have reflection. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Sep 19 20:14:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Sep 2018 21:14:58 +0100 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <5BA299FE.9090406@oracle.com> References: <5BA299FE.9090406@oracle.com> Message-ID: <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> On 19/09/2018 19:48, Joe Wang wrote: > Hi, > > After much discussion and 10 iterations of reviews, this proposal has > evolved from what was the original isSameContent method to a mismatch > method. API-wise, a compare method was also considered as it looked > like just a short step forward from mismatch, however, it was > eventually dropped since there is no convincing use case comparing > files lexicographically by contents. Impl-wise, extensive performance > benchmarking has been done to compare a buffered reading vs memory > mapping, the result was that a simple buffered reading performed > better among small files, and those with the mismatched byte closer to > the beginning of files. Since the proposed method's targeted files are > small ones, the impl currently does a buffered reading only. > > Please review. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202285 > > specdiff: > http://cr.openjdk.java.net/~joehw/jdk12/8202285/specdiff/java/nio/file/Files.html > > webrev: http://cr.openjdk.java.net/~joehw/jdk12/8202285/webrev/ Starting out using buffered I/O is probably okay but I assume we will want to change this in the future to having it use memory mapped I/O beyond a certain threshold. Can you explain the use of toRealPath and comparing the names? That shouldn't be needed. Also the catching of ProviderMismatchExcepiton seems a bit strange too. Can you replace mismatchByAttrs with isSameFile? You could call this from Files.mismatch and then use the supporting implementation for the case that the files are not the same. -Alan From brian.burkhalter at oracle.com Wed Sep 19 20:44:51 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Sep 2018 13:44:51 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> Message-ID: <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> On Sep 19, 2018, at 1:14 PM, Alan Bateman wrote: > Starting out using buffered I/O is probably okay but I assume we will want to change this in the future to having it use memory mapped I/O beyond a certain threshold. For the buffered case, could there be any performance advantage to using direct buffers with FileChannel and comparing running checksums of the two sources? Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 19 20:52:51 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Sep 2018 13:52:51 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> Message-ID: <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> On Sep 19, 2018, at 1:44 PM, Brian Burkhalter wrote: > On Sep 19, 2018, at 1:14 PM, Alan Bateman wrote: > >> Starting out using buffered I/O is probably okay but I assume we will want to change this in the future to having it use memory mapped I/O beyond a certain threshold. > > For the buffered case, could there be any performance advantage to using direct buffers with FileChannel and comparing running checksums of the two sources? Don?t think this would work however as two files with differing content could conceivable have the same checksum however likely that might be. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Wed Sep 19 20:53:56 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 19 Sep 2018 13:53:56 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> Message-ID: <566CEB7D-28FB-432A-9C91-7406DAD7768A@oracle.com> On Sep 19, 2018, at 1:52 PM, Brian Burkhalter wrote: >> For the buffered case, could there be any performance advantage to using direct buffers with FileChannel and comparing running checksums of the two sources? > > Don?t think this would work however as two files with differing content could conceivable have the same checksum however likely that might be. s/conceivable /conceivably/. s/likely/unlikely/. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From huizhe.wang at oracle.com Thu Sep 20 00:59:40 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 19 Sep 2018 17:59:40 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> Message-ID: <5BA2F0FC.90102@oracle.com> On 9/19/18, 1:52 PM, Brian Burkhalter wrote: > On Sep 19, 2018, at 1:44 PM, Brian Burkhalter wrote: > >> On Sep 19, 2018, at 1:14 PM, Alan Bateman wrote: >> >>> Starting out using buffered I/O is probably okay but I assume we will want to change this in the future to having it use memory mapped I/O beyond a certain threshold. >> For the buffered case, could there be any performance advantage to using direct buffers with FileChannel and comparing running checksums of the two sources? > Don?t think this would work however as two files with differing content could conceivable have the same checksum however likely that might be. I'll do some performance testing for both cases, FileChannel/direct buffer and possible checksum comparison, although, as you said, the later might not work since we need to be 100%. The probability of false positives is likely extremely low, but is not zero after all. I'll do the performance check any ways just out of curiosity. But in many cases, checksum comparison may not be faster than direct comparison, esp. in this case the API compares just two files. Checksum values would be nice if there are multiple files to compare. Thanks, Joe > > Brian From Alan.Bateman at oracle.com Thu Sep 20 07:07:40 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Sep 2018 08:07:40 +0100 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <5BA2EB02.7030204@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <5BA2EB02.7030204@oracle.com> Message-ID: <71b38ab1-8728-e72c-c757-e0e1128bc9ac@oracle.com> On 20/09/2018 01:34, Joe Wang wrote: > : > > The purpose of toRealPath is so that we can do a quick Path comparison > by following symlinks if any, I'll add a test to compare a file and > its symbolic link. You don't need it. If two non-equals paths locate the same file then mismatch will do the right thing. By dropping it then you avoid a potentially expensive realpath (x2). > ProviderMismatchExcepiton was added in one of the iterations when we > defined UOE, shall be removed in the next update (I'll have an update > after some performance work to compare with direct buffer is done). Good. > > mismatchByAttrs cannot be replaced with isSameFile, it does one edge > case check more than isSameFile when the size of one of the files is > zero. mismatchByAttrs opens the files once vs twice if isSameFile and > size(path) are called. Hmm, I think we make this a lot simpler. -Alan From Alan.Bateman at oracle.com Thu Sep 20 07:12:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Sep 2018 08:12:34 +0100 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> Message-ID: On 19/09/2018 22:02, Roger Riggs wrote: > This came up in off-line discussions, it seems unlikely that two files > will differ only in the last of 100Mb > and it will require a separate code path that will very infrequently > be exercised.? So I'd still to a single > technique even if it is slightly slower for very large files to keep > the size of the code in check. > If it shows up later as a performance problem it can be added. > I think this will eventually have a different implementation for the default file system where it can used memory mapping of file files. If the first 1MB of large files are identical then that it might switch over to mapping by chunk to compare the rest of the file. Starting out with a basic implementation is okay of course. -Alan. From brian.burkhalter at oracle.com Thu Sep 20 15:10:50 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 20 Sep 2018 08:10:50 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <5BA2F0FC.90102@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> <5BA2F0FC.90102@oracle.com> Message-ID: <628BAAA4-C583-473A-8396-310865E4FAC1@oracle.com> On Sep 19, 2018, at 5:59 PM, Joe Wang wrote: > I'll do some performance testing for both cases, FileChannel/direct buffer and possible checksum comparison, although, as you said, the later might not work since we need to be 100%. The probability of false positives is likely extremely low, but is not zero after all. I'll do the performance check any ways just out of curiosity. But in many cases, checksum comparison may not be faster than direct comparison, esp. in this case the API compares just two files. Checksum values would be nice if there are multiple files to compare. I doubt the checksum test is worthwhile. It might be useful however to try FileChannel and direct byte buffers combined with ByteBuffer.mismatch() [1]. Thanks, Brian [1] https://download.java.net/java/early_access/jdk12/docs/api/java.base/java/nio/ByteBuffer.html#mismatch(java.nio.ByteBuffer) -------------- next part -------------- An HTML attachment was scrubbed... URL: From huizhe.wang at oracle.com Thu Sep 20 17:28:47 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 20 Sep 2018 10:28:47 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <71b38ab1-8728-e72c-c757-e0e1128bc9ac@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <5BA2EB02.7030204@oracle.com> <71b38ab1-8728-e72c-c757-e0e1128bc9ac@oracle.com> Message-ID: <5BA3D8CF.7020907@oracle.com> On 9/20/18, 12:07 AM, Alan Bateman wrote: > On 20/09/2018 01:34, Joe Wang wrote: >> : >> >> The purpose of toRealPath is so that we can do a quick Path >> comparison by following symlinks if any, I'll add a test to compare a >> file and its symbolic link. > You don't need it. If two non-equals paths locate the same file then > mismatch will do the right thing. By dropping it then you avoid a > potentially expensive realpath (x2). > > >> ProviderMismatchExcepiton was added in one of the iterations when we >> defined UOE, shall be removed in the next update (I'll have an update >> after some performance work to compare with direct buffer is done). > Good. > >> >> mismatchByAttrs cannot be replaced with isSameFile, it does one edge >> case check more than isSameFile when the size of one of the files is >> zero. mismatchByAttrs opens the files once vs twice if isSameFile and >> size(path) are called. > Hmm, I think we make this a lot simpler. Ok, I'll call isSameFile instead, that would get rid of toRealPath too, to not let these edge case handling penalize the most, normal use cases. The other way around (a bit more cost to the edge cases) shall be fine. Thanks, Joe > > -Alan From huizhe.wang at oracle.com Thu Sep 20 18:04:05 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 20 Sep 2018 11:04:05 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> Message-ID: <5BA3E115.5090703@oracle.com> On 9/20/18, 12:12 AM, Alan Bateman wrote: > On 19/09/2018 22:02, Roger Riggs wrote: >> This came up in off-line discussions, it seems unlikely that two >> files will differ only in the last of 100Mb >> and it will require a separate code path that will very infrequently >> be exercised. So I'd still to a single >> technique even if it is slightly slower for very large files to keep >> the size of the code in check. >> If it shows up later as a performance problem it can be added. >> > I think this will eventually have a different implementation for the > default file system where it can used memory mapping of file files. If > the first 1MB of large files are identical then that it might switch > over to mapping by chunk to compare the rest of the file. Starting out > with a basic implementation is okay of course. Sounds good. Will keep the basic impl, probably use FileChannel/ByteBuffer as Brian suggested, and look forward to the more complicated version if we hear any complaints. For the majority of text files, the basic impl would be enough. For example, the largest file in java.nio is about 180K. Thanks, Joe > > -Alan. > From huizhe.wang at oracle.com Thu Sep 20 18:08:07 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 20 Sep 2018 11:08:07 -0700 Subject: RFR(JDK12/NIO) 8202285: (fs) Add a method to Files for comparing file contents In-Reply-To: <628BAAA4-C583-473A-8396-310865E4FAC1@oracle.com> References: <5BA299FE.9090406@oracle.com> <3e2990a1-33d1-600f-011f-497a85d93652@oracle.com> <74CD5888-73FD-4169-BB6F-4536CE61F979@oracle.com> <1FF20FD7-DDC8-4246-B75E-2C2B5C4B4900@oracle.com> <5BA2F0FC.90102@oracle.com> <628BAAA4-C583-473A-8396-310865E4FAC1@oracle.com> Message-ID: <5BA3E207.5060600@oracle.com> On 9/20/18, 8:10 AM, Brian Burkhalter wrote: > > On Sep 19, 2018, at 5:59 PM, Joe Wang > wrote: > >> I'll do some performance testing for both cases, FileChannel/direct >> buffer and possible checksum comparison, although, as you said, the >> later might not work since we need to be 100%. The probability of >> false positives is likely extremely low, but is not zero after all. >> I'll do the performance check any ways just out of curiosity. But in >> many cases, checksum comparison may not be faster than direct >> comparison, esp. in this case the API compares just two files. >> Checksum values would be nice if there are multiple files to compare. > > I doubt the checksum test is worthwhile. It might be useful however to > try FileChannel and direct byte buffers combined with > ByteBuffer.mismatch() [1]. Ok, save the time not to run checksum testing :-) Will do the later, actually, I'm almost sure it will be FileChannel/ByteBuffer (better than the buffered reading). Thanks, Joe > > Thanks, > > Brian > > [1] > https://download.java.net/java/early_access/jdk12/docs/api/java.base/java/nio/ByteBuffer.html#mismatch(java.nio.ByteBuffer) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Mon Sep 24 03:07:18 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Mon, 24 Sep 2018 03:07:18 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! -90d5c464e6bc@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> Hi Alan/Brian, Here is the version 10 of the patch http://cr.openjdk.java.net/~ylu/8195160.10. This patch includes all the changes Brian added based on version 09. In addition, I added following things: 1. Fixed JNI function names from various C files. This is due to the package name changing from rdma.ch to jdk.internal.net.rdma 2. Found a small issue with supportedOptions() from both java.net.Socket and java.net.ServerSocket. When a RDMA socket created after a regular TCP socket or verse visa, the private variable "options" can possibly return the wrong set of supported socket options. I create another variable named rdmaOptions and use that to track socket options for all the RDMA sockets. 3. Removed some of the test cases from RDMA test set that are targeted only for old TCP socket issues. Currently, all the test cases passed on my CentOS 7 machine (with librdmacm and librdmacm-devel installed) 4. Created one more test for non-Linux platform to test if UOE is thrown on RDMA sockets and channels. 5. For test cases, I removed "include " from C file and -lrdmacm inside JTreg make file. Instead, I use dlopen and dlsym to load the library dynamically. Next item to work on: 1. For JDK internal implementation, I am not very sure how to properly use dlopen/dlsym. If we do dlopen/dlsym every time we create a RDMA socket or socket channel, will that cause lots of overhead? Where is the best place to load the library? Please let me know if you have any suggestions. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Wednesday, September 19, 2018 9:00 AM To: Alan Bateman Cc: Lu, Yingqi ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK On Sep 19, 2018, at 8:54 AM, Alan Bateman > wrote: 3. Reflection replaced with shadow classes. Thanks for taking up the suggestion to have two XXXProvider implementations, I think it looks much cleaner now. You're welcome. I had not been aware of that possibility: definitely better not to have reflection. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Sep 24 06:49:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 24 Sep 2018 07:49:29 +0100 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> Message-ID: <0de1e776-446b-8f60-9778-c1294790e4d3@oracle.com> On 24/09/2018 04:07, Lu, Yingqi wrote: > > Next item to work on: > > 1.For JDK internal implementation, I am not very sure how to properly > use dlopen/dlsym. If we do dlopen/dlsym every time we create a RDMA > socket or socket channel, will that cause lots of overhead? Where is > the best place to load the library? Please let me know if you have any > suggestions. > You can use a static initializer that invokes a native method to do the dlopen and dlsym on each of the functions that are needed. One example that may be useful is the SCTP support as sctp.h and libsctp may not be on the system when building or running. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Sep 25 00:51:34 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 24 Sep 2018 17:51:34 -0700 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> Message-ID: Hi Lucy, On Sep 23, 2018, at 8:07 PM, Lu, Yingqi wrote: > Here is the version 10 of the patch http://cr.openjdk.java.net/~ylu/8195160.10. To simplify the review I created a diff webrev versus version 9: http://cr.openjdk.java.net/~bpb/8195160/webrev.10vs09/ > This patch includes all the changes Brian added based on version 09. In addition, I added following things: > > 1. Fixed JNI function names from various C files. This is due to the package name changing from rdma.ch to jdk.internal.net.rdma Sorry I overlooked this. In other C code I encountered an error while building: openjdk/src/jdk.net/linux/native/libextnet/LinuxRdmaSocketDispatcherImpl.c:146:1: error: ?handle? defined but not used [-Werror=unused-function] handle(JNIEnv *env, jlong rv, char *msg) ^~~~~~ If the function is simply removed then it builds. > 2. Found a small issue with supportedOptions() from both java.net.Socket and java.net.ServerSocket. When a RDMA socket created after a regular TCP socket or verse visa, the private variable ?options? can possibly return the wrong set of supported socket options. I create another variable named rdmaOptions and use that to track socket options for all the RDMA sockets. > 3. Removed some of the test cases from RDMA test set that are targeted only for old TCP socket issues. Currently, all the test cases passed on my CentOS 7 machine (with librdmacm and librdmacm-devel installed) > 4. Created one more test for non-Linux platform to test if UOE is thrown on RDMA sockets and channels. > 5. For test cases, I removed ?include ? from C file and ?lrdmacm inside JTreg make file. Instead, I use dlopen and dlsym to load the library dynamically. All tests pass in my local Linux build. Aside from the dlopen/dlsym issue that you are working this is looking pretty good. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Tue Sep 25 03:57:40 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 25 Sep 2018 03:57:40 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> Hi Brian, Thank you for the quick review. As Alan suggested, I am following the example of SCTP support to add dlopen/dlsym into JDK internal implementations. Hopefully, I will be able to submit another version of the patch tomorrow or Wednesday. Do you prefer that I submit a diff vs. webrev.09 or a complete patch to current jdk trunk as webrev.11? Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Monday, September 24, 2018 5:52 PM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, On Sep 23, 2018, at 8:07 PM, Lu, Yingqi > wrote: Here is the version 10 of the patch http://cr.openjdk.java.net/~ylu/8195160.10. To simplify the review I created a diff webrev versus version 9: http://cr.openjdk.java.net/~bpb/8195160/webrev.10vs09/ This patch includes all the changes Brian added based on version 09. In addition, I added following things: 1. Fixed JNI function names from various C files. This is due to the package name changing from rdma.ch to jdk.internal.net.rdma Sorry I overlooked this. In other C code I encountered an error while building: openjdk/src/jdk.net/linux/native/libextnet/LinuxRdmaSocketDispatcherImpl.c:146:1: error: 'handle' defined but not used [-Werror=unused-function] handle(JNIEnv *env, jlong rv, char *msg) ^~~~~~ If the function is simply removed then it builds. 2. Found a small issue with supportedOptions() from both java.net.Socket and java.net.ServerSocket. When a RDMA socket created after a regular TCP socket or verse visa, the private variable "options" can possibly return the wrong set of supported socket options. I create another variable named rdmaOptions and use that to track socket options for all the RDMA sockets. 3. Removed some of the test cases from RDMA test set that are targeted only for old TCP socket issues. Currently, all the test cases passed on my CentOS 7 machine (with librdmacm and librdmacm-devel installed) 4. Created one more test for non-Linux platform to test if UOE is thrown on RDMA sockets and channels. 5. For test cases, I removed "include " from C file and -lrdmacm inside JTreg make file. Instead, I use dlopen and dlsym to load the library dynamically. All tests pass in my local Linux build. Aside from the dlopen/dlsym issue that you are working this is looking pretty good. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Tue Sep 25 14:57:48 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 25 Sep 2018 14:57:48 +0000 Subject: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and permissions of the files Message-ID: Hi all, I had asked for opinions regarding adding posix permission support to JDK?s zip handling libraries and tools [1]. Since I didn?t get clear ?no, you can?t do this? answers, I?m now concretely proposing some enhancements in the area of java.util.zip, jdk.zipfs and jdk.jartool. I have reopened the long standing bug report (6194856) which had been set to ?Won?t fix? quite recently for this purpose. Here are the technical details: The ZIP format specifications by infozip and pkware ([2] and [3]) do not explicitly specify the handling of posix file permissions. But it seems to be common sense that if file attribute compatibility is set to ?Unix? in the upper byte of CEN field ?version made by?, the file permissions bits are stored in CEN field ?external file attributes?, byte 3 and 4. My changes try to honor this in the least obtrusive way. ?? The following changes are proposed: java.util.zip.ZipEntry: it will have an additional field ?posixPerms?. A value of -1 means ?no permission information?, positive values will contain the flag values. There will be 2 new public methods to read/set the permission information: public Optional> getPosixPermissions() public void setPosixPermissions(Set permissions) The usage of type ?Optional? reflects that posix permissions are not necessarily present in a zip file java.util.zip.ZipFile: it will have the capability to read the CEN fields and set posixPerms if available java.util.zip.ZipOutputStream: it will store entries with associated posix permissions as unix type in the CEN, together with the bit mask for the permissions jdk.jartool: I propose to add and option "--preserve-posix" or short "-o" to honor the posix bits that may be stored inside zip/jar files. By default the option is not set and hence posix permissions are ignored. If the flag is set and the file system that the jar tool is running on supports posix, posix file permissions that exist in the file system will be stored in newly created/update archives or restored to the file system if such information is present in the archive. jdk.zipfs: I added the capability for posix file permissions in the implementation. I decided to support PosixFileAttributes by subclassing ZipFileAttributes from this superclass as well as subclassing ZipFileAttributeView from PosixFileAttributeView. However, as PosixFileAttributes also include groups and owners, I would throw UnsupportedOperationExceptions in case of invoking methods to handle these attributes. But this approach seems to be most consistent with e.g. Files.setPosixFilePermissions and Files.getPosixFilePermissions. java.nio.file.attribute.PosixFilePermissions: As this class presents a collection of static helpers, I added definitions for the posix file bit masks together with methods to convert between Sets of PosixFilePermission to bit masks containing the according switches and vice versa. These definitions could theoretically also be moved inside the java.util.zip or jdk.zipfs implementations where they wouldn?t be exposed as public APIs. However, in that case the code would probably need to be duplicated. I?ve also created two jtreg testcases for both, java.util.zip and jdk.nio.zipfs. The changes also contain a few further code cleanups. Here are the links: Bug: https://bugs.openjdk.java.net/browse/JDK-6194856 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/6194856.0/ I?ll write a CSR once there?s some substantial feedback to my endeavor. Thanks in advance for reviewing/commenting. Best regards Christoph [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-September/055375.html [2] http://www.info-zip.org/doc/appnote-19970311-iz.zip [3] https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Sep 25 15:10:53 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 25 Sep 2018 16:10:53 +0100 Subject: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and permissions of the files In-Reply-To: References: Message-ID: On 25/09/2018 15:57, Langer, Christoph wrote: > > Hi all, > > I had asked for opinions regarding adding posix permission support to > JDK?s zip handling libraries and tools [1]. Since I didn?t get clear > ?no, you can?t do this? answers, I?m now concretely proposing some > enhancements in the area of java.util.zip, jdk.zipfs and jdk.jartool. > > I have reopened the long standing bug report (6194856) which had been > set to ?Won?t fix? quite recently for this purpose. > > Here are the technical details: > > The ZIP format specifications by infozip and pkware ([2] and [3]) do > not explicitly specify the handling of posix file permissions. But it > seems to be common sense that if file attribute compatibility is set > to ?Unix? in the upper byte of CEN field ?version made by?, the file > permissions bits are stored in CEN field ?external file attributes?, > byte 3 and 4. My changes try to honor this in the least obtrusive way. ?? > > The following changes are proposed: > > java.util.zip.ZipEntry: > > it will have an additional field ?posixPerms?. A value of -1 means ?no > permission information?, positive values will contain the flag values. > > There will be 2 new public methods to read/set the permission information: > > ??????????????? public Optional> > getPosixPermissions() > > ??????????????? public void > setPosixPermissions(Set permissions) > > The usage of type ?Optional? reflects that posix permissions are not > necessarily present in a zip file > > java.util.zip.ZipFile: > > ??????????????? it will have the capability to read the CEN fields and > set posixPerms if available > > java.util.zip.ZipOutputStream: > > ??????????????? it will store entries with associated posix > permissions as unix type in the CEN, together with the bit mask for > the permissions > > jdk.jartool: > > I propose to add and option "--preserve-posix" or short "-o" to honor > the posix bits that may be stored inside zip/jar files. By default the > option is not set and hence posix permissions are ignored. If the flag > is set and the file system that the jar tool is running on supports > posix, posix file permissions that exist in the file system will be > stored in newly created/update archives or restored to the file system > if such information is present in the archive. > > jdk.zipfs: > > ??????????????? I added the capability for posix file permissions in > the implementation. I decided to support PosixFileAttributes by > subclassing ZipFileAttributes from this superclass as well as > subclassing ZipFileAttributeView from PosixFileAttributeView. However, > as PosixFileAttributes also include groups and owners, I would throw > UnsupportedOperationExceptions in case of invoking methods to handle > these attributes. But this approach seems to be most consistent with > e.g. Files.setPosixFilePermissions and Files.getPosixFilePermissions. > > java.nio.file.attribute.PosixFilePermissions: > > ??????????????? As this class presents a collection of static helpers, > I added definitions for the posix file bit masks together with methods > to convert between Sets of PosixFilePermission to bit masks containing > the according switches and vice versa. These definitions could > theoretically also be moved inside the java.util.zip or jdk.zipfs > implementations where they wouldn?t be exposed as public APIs. > However, in that case the code would probably need to be duplicated. > > I?ve also created two jtreg testcases for both, java.util.zip and > jdk.nio.zipfs. > > The changes also contain a few further code cleanups. > > Here are the links: > > Bug: https://bugs.openjdk.java.net/browse/JDK-6194856 > > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/6194856.0/ > > > I?ll write a CSR once there?s some substantial feedback to my endeavor. > > I think this will require significant discussion as it was previously closed as WNF (as you found). One initial comment is that you should be able to drop the API additions to PosxFilePermissions - I think all you need there translation from Set to internal values, it doesn't require bloating the API. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue Sep 25 16:24:30 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 25 Sep 2018 09:24:30 -0700 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> Message-ID: Hi Lucy, Go ahead and submit a complete webrev version 11. I generated the one of 10 versus 9 partly for my convenience. Thanks, Brian On Sep 24, 2018, at 8:57 PM, Lu, Yingqi wrote: > Do you prefer that I submit a diff vs. webrev.09 or a complete patch to current jdk trunk as webrev.11? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Tue Sep 25 19:58:52 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 25 Sep 2018 19:58:52 +0000 Subject: adding rsockets support into JDK In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD152B5C5@ORSMSX113.amr.corp.intel.com> Hi Brian, Please find webrev.11 is available at http://cr.openjdk.java.net/~ylu/8195160.11/ In this version, 1. I removed extra function "handle" from src/jdk.net/linux/native/libextnet/LinuxRdmaSocketDispatcherImpl.c:146 2. In addition, I removed dependency of . Instead, I used dlopen/dlsym to link the library. Specifically, I created two additional files Rsocket.h and Rsocket.c from src/jdk.net/linux/native/libextnet and use them to define and link all the RDMA native functions. Please let me know if there is anything missing here. With the above changes, all test cases pass on my Linux CentOS 7 box. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Tuesday, September 25, 2018 9:25 AM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, Go ahead and submit a complete webrev version 11. I generated the one of 10 versus 9 partly for my convenience. Thanks, Brian On Sep 24, 2018, at 8:57 PM, Lu, Yingqi > wrote: Do you prefer that I submit a diff vs. webrev.09 or a complete patch to current jdk trunk as webrev.11? -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Sep 26 07:48:54 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 26 Sep 2018 07:48:54 +0000 Subject: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and permissions of the files In-Reply-To: References: Message-ID: Hi Alan, thanks for your initial comments. > One initial comment is that you should be able to drop the API additions to > PosxFilePermissions - I think all you need there translation from > Set to internal values, it doesn't require bloating the > API. I agree, it doesn't feel right to bloat PosixFilePermissions with details needed for zip operations. I now moved this coding to the ZipUtils classes. The only drawback is the duplication of the code but there's already quite some duplication of coding between java.util.zip and jdk.nio.zipfs. Here is an updated webrev: http://cr.openjdk.java.net/~clanger/webrevs/6194856.1/ Best regards Christoph From matthias.baesken at sap.com Wed Sep 26 09:24:50 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 26 Sep 2018 09:24:50 +0000 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef Message-ID: Hello, please review this small build fix . After the recent changes to the gcc compile flags with 8211029 , most of our Linux builds stopped working . Error : === Output from failing command(s) repeated here === * For target support_native_java.base_libnet_DatagramPacket.o: In file included from /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/net_util.h:31:0, from /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/DatagramPacket.c:27: /OpenJDK/8210319/jdk/src/java.base/unix/native/libnet/net_util_md.h:50:7: error: "__solaris__" is not defined [-Werror=undef] #elif __solaris__ ^ After looking into it, it seems that the jdk/src/java.base/unix/native/libnet/net_util_md.h compile error is only triggered on older Linux OS (e.g. SLES11). Probably that's why it was not seen at Oracle after puhsing after 8211029 . Some greps showed me a number of similar problematic #elif-checks for OS, I adjusted them too . Bug / webrev : https://bugs.openjdk.java.net/browse/JDK-8211146 http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Wed Sep 26 09:31:42 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 26 Sep 2018 11:31:42 +0200 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: Message-ID: <5cad1088-21c5-e591-151d-b35ac96457fe@redhat.com> On 09/26/2018 11:24 AM, Baesken, Matthias wrote: > http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ Oh, nice catch. This looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From christoph.langer at sap.com Wed Sep 26 09:48:14 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 26 Sep 2018 09:48:14 +0000 Subject: [CAUTION] RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: Message-ID: <3667f01e602a47a59739dfb6b73dcd5c@sap.com> Hi Matthias, looks good (and trivial). Ccing serviceability-dev because of change in libjdwp. Best regards Christoph From: nio-dev On Behalf Of Baesken, Matthias Sent: Mittwoch, 26. September 2018 11:25 To: 'build-dev at openjdk.java.net' ; net-dev ; nio-dev at openjdk.java.net Cc: Lindenmaier, Goetz ; Schmidt, Lutz Subject: [CAUTION] RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef Hello, please review this small build fix . After the recent changes to the gcc compile flags with 8211029 , most of our Linux builds stopped working . Error : === Output from failing command(s) repeated here === * For target support_native_java.base_libnet_DatagramPacket.o: In file included from /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/net_util.h:31:0, from /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/DatagramPacket.c:27: /OpenJDK/8210319/jdk/src/java.base/unix/native/libnet/net_util_md.h:50:7: error: "__solaris__" is not defined [-Werror=undef] #elif __solaris__ ^ After looking into it, it seems that the jdk/src/java.base/unix/native/libnet/net_util_md.h compile error is only triggered on older Linux OS (e.g. SLES11). Probably that's why it was not seen at Oracle after puhsing after 8211029 . Some greps showed me a number of similar problematic #elif-checks for OS, I adjusted them too . Bug / webrev : https://bugs.openjdk.java.net/browse/JDK-8211146 http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed Sep 26 09:51:12 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Sep 2018 10:51:12 +0100 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: Message-ID: <07071eaf-e505-4763-642d-cc227b4cdd12@oracle.com> On 26/09/2018 10:24, Baesken, Matthias wrote: > > Hello, please review this small build fix . > > After the recent changes? to ?the gcc compile flags?? with? 8211029?? > ?, most of our? Linux builds stopped working . > > Error : > > === Output from failing command(s) repeated here === > > * For target support_native_java.base_libnet_DatagramPacket.o: > > In file included from > /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/net_util.h:31:0, > > ????????????????from > /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/DatagramPacket.c:27: > > /OpenJDK/8210319/jdk/src/java.base/unix/native/libnet/net_util_md.h:50:7: > error: "__solaris__" is not defined [-Werror=undef] > > #elif __solaris__ > > ?????? ^ > > After looking into it, it seems? that? the > jdk/src/java.base/unix/native/libnet/net_util_md.h compile error is > only triggered on older Linux OS? (e.g. SLES11). > > Probably that?s why it was not seen at Oracle ?after? puhsing? after > 8211029?? . > > Some greps? showed me a number of similar problematic? #elif-checks > for OS, I adjusted them too . > > Bug / webrev : > > https://bugs.openjdk.java.net/browse/JDK-8211146 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ > > > This looks okay to me although I could imagine this issue coming back again with ongoing edits that add "ifdef __solaris" or similar. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Sep 26 10:26:21 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 26 Sep 2018 11:26:21 +0100 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: Message-ID: <5e1bccb0-9e7a-0e0f-3a31-7d09fa251574@oracle.com> On 26/09/18 10:24, Baesken, Matthias wrote: >...wse/JDK-8211146 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ Looks good Matthias. -Chris. From thomas.stuefe at gmail.com Wed Sep 26 10:35:28 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Sep 2018 12:35:28 +0200 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: <07071eaf-e505-4763-642d-cc227b4cdd12@oracle.com> References: <07071eaf-e505-4763-642d-cc227b4cdd12@oracle.com> Message-ID: On Wed, Sep 26, 2018 at 11:51 AM, Alan Bateman wrote: > > > On 26/09/2018 10:24, Baesken, Matthias wrote: >> >> >> Hello, please review this small build fix . >> >> After the recent changes to the gcc compile flags with 8211029 , >> most of our Linux builds stopped working . >> >> Error : >> >> === Output from failing command(s) repeated here === >> >> * For target support_native_java.base_libnet_DatagramPacket.o: >> >> In file included from >> /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/net_util.h:31:0, >> >> from >> /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/DatagramPacket.c:27: >> >> /OpenJDK/8210319/jdk/src/java.base/unix/native/libnet/net_util_md.h:50:7: >> error: "__solaris__" is not defined [-Werror=undef] >> >> #elif __solaris__ >> >> ^ >> >> After looking into it, it seems that the >> jdk/src/java.base/unix/native/libnet/net_util_md.h compile error is only >> triggered on older Linux OS (e.g. SLES11). >> >> Probably that?s why it was not seen at Oracle after puhsing after >> 8211029 . >> >> Some greps showed me a number of similar problematic #elif-checks for >> OS, I adjusted them too . >> >> Bug / webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8211146 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ >> >> >> > This looks okay to me although I could imagine this issue coming back again > with ongoing edits that add "ifdef __solaris" or similar. > If I understand this problem correctly (and I might not) the problem is with "#if ", not with "#ifdef ". ..thomas > -Alan From david.holmes at oracle.com Wed Sep 26 11:41:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Sep 2018 07:41:23 -0400 Subject: [CAUTION] RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: <3667f01e602a47a59739dfb6b73dcd5c@sap.com> References: <3667f01e602a47a59739dfb6b73dcd5c@sap.com> Message-ID: <9af9d8c2-2580-0f25-2881-f4d76b2596a9@oracle.com> That all seems fine to me. Thanks for fixing. David On 26/09/2018 5:48 AM, Langer, Christoph wrote: > Hi Matthias, > > looks good (and trivial). Ccing serviceability-dev because of change in > libjdwp. > > Best regards > > Christoph > > *From:*nio-dev *On Behalf Of > *Baesken, Matthias > *Sent:* Mittwoch, 26. September 2018 11:25 > *To:* 'build-dev at openjdk.java.net' ; net-dev > ; nio-dev at openjdk.java.net > *Cc:* Lindenmaier, Goetz ; Schmidt, Lutz > > *Subject:* [CAUTION] RFR : 8211146 : fix problematic elif-tests after > recent gcc warning changes Werror=undef > > Hello, please review this small build fix . > > After the recent? changes? to ?the gcc compile flags?? with? 8211029 > ?,? most of our? Linux builds stopped working . > > Error : > > === Output from failing command(s) repeated here === > > * For target support_native_java.base_libnet_DatagramPacket.o: > > In file included from > /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/net_util.h:31:0, > > ????????????????from > /OpenJDK/8210319/jdk/src/java.base/share/native/libnet/DatagramPacket.c:27: > > /OpenJDK/8210319/jdk/src/java.base/unix/native/libnet/net_util_md.h:50:7: error: > "__solaris__" is not defined [-Werror=undef] > > #elif __solaris__ > > ?????? ^ > > After looking into it,? it seems? that? the > jdk/src/java.base/unix/native/libnet/net_util_md.h??? compile error is > only triggered on older Linux OS? (e.g. SLES11). > > Probably that?s why it was not seen at Oracle ?after? puhsing? after > 8211029?? . > > Some greps? showed me a number of similar problematic? #elif-checks for > OS, I adjusted them too . > > Bug / webrev : > > https://bugs.openjdk.java.net/browse/JDK-8211146 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211146.0/ > > > Thanks, Matthias > From matthias.baesken at sap.com Wed Sep 26 12:19:56 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 26 Sep 2018 12:19:56 +0000 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: <07071eaf-e505-4763-642d-cc227b4cdd12@oracle.com> Message-ID: Hi Thomas, I think your understanding is correct : "#ifdef " or "#if defined()" Are ok while "#if " or "#elif " Lead to the compile error on gcc with current warnings. Best regards, Matthias > >> > > This looks okay to me although I could imagine this issue coming back again > > with ongoing edits that add "ifdef __solaris" or similar. > > > > If I understand this problem correctly (and I might not) the problem > is with "#if ", not with "#ifdef switch>". > > ..thomas > > > -Alan From thomas.stuefe at gmail.com Wed Sep 26 12:22:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Sep 2018 14:22:06 +0200 Subject: RFR : 8211146 : fix problematic elif-tests after recent gcc warning changes Werror=undef In-Reply-To: References: <07071eaf-e505-4763-642d-cc227b4cdd12@oracle.com> Message-ID: Thanks for clarifying! Thomas On Wed, Sep 26, 2018, 14:20 Baesken, Matthias wrote: > Hi Thomas, I think your understanding is correct : > > "#ifdef " > > or > > "#if defined()" > > Are ok while > > "#if " > > or > > "#elif " > > Lead to the compile error on gcc with current warnings. > > Best regards, Matthias > > > > >> > > > This looks okay to me although I could imagine this issue coming back > again > > > with ongoing edits that add "ifdef __solaris" or similar. > > > > > > > If I understand this problem correctly (and I might not) the problem > > is with "#if ", not with "#ifdef > switch>". > > > > ..thomas > > > > > -Alan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Fri Sep 28 15:00:59 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 28 Sep 2018 15:00:59 +0000 Subject: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and permissions of the files In-Reply-To: References: Message-ID: Ping... Are there any reviews/comments on my proposal for adding unix/posix permissions to zip APIs? Thanks Christoph > -----Original Message----- > From: Langer, Christoph > Sent: Mittwoch, 26. September 2018 09:49 > To: 'Alan Bateman' ; core-libs- > dev at openjdk.java.net; nio-dev ; Xueming > Shen > Cc: Volker Simonis ; Lindenmaier, Goetz > > Subject: RE: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and > permissions of the files > > Hi Alan, > > thanks for your initial comments. > > > One initial comment is that you should be able to drop the API additions to > > PosxFilePermissions - I think all you need there translation from > > Set to internal values, it doesn't require bloating the > > API. > > I agree, it doesn't feel right to bloat PosixFilePermissions with details needed > for zip operations. > > I now moved this coding to the ZipUtils classes. The only drawback is the > duplication of the code but there's already quite some duplication of coding > between java.util.zip and jdk.nio.zipfs. > > Here is an updated webrev: > http://cr.openjdk.java.net/~clanger/webrevs/6194856.1/ > > Best regards > Christoph From brian.burkhalter at oracle.com Fri Sep 28 23:22:08 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 28 Sep 2018 16:22:08 -0700 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD152B5C5@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! ! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152B5C5@ORSMSX113.amr.corp.intel.com> Message-ID: <13D9EA95-E63B-4764-BC33-27098D1BE48C@oracle.com> Hi Lucy, On Sep 25, 2018, at 12:58 PM, Lu, Yingqi wrote: > Please find webrev.11 is available athttp://cr.openjdk.java.net/~ylu/8195160.11/ > > [?] > > With the above changes, all test cases pass on my Linux CentOS 7 box. I get the failures [1, 2] when this version is run on my local build (Ubuntu 18.04 VM). Thanks, Brian [1] jdk/net/Sockets/rsocket/ServerSocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.serverSocket(RdmaNet.java:161) at jdk.net/jdk.internal.net.rdma.RdmaServerSocketChannelImpl.(RdmaServerSocketChannelImpl.java:84) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openServerSocketChannel(LinuxRdmaPollSelectorProvider.java:51) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openServerSocketChannel(RdmaPollSelectorProvider.java:67) at jdk.net/jdk.net.Sockets.openRdmaServerSocketChannel(Sockets.java:493) at SocketOptionTests.main(SocketOptionTests.java:55) [2] jdk/net/Sockets/rsocket/SocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:157) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:150) at jdk.net/jdk.internal.net.rdma.RdmaSocketChannelImpl.(RdmaSocketChannelImpl.java:96) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openSocketChannel(LinuxRdmaPollSelectorProvider.java:46) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openSocketChannel(RdmaPollSelectorProvider.java:63) at jdk.net/jdk.net.Sockets.openRdmaSocketChannel(Sockets.java:471) at SocketOptionTests.main(SocketOptionTests.java:54) -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Fri Sep 28 23:26:24 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 28 Sep 2018 23:26:24 +0000 Subject: adding rsockets support into JDK In-Reply-To: <13D9EA95-E63B-4764-BC33-27098D1BE48C@oracle.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! ! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152B5C5@ORSMSX113.amr.corp.intel.com> <13D9EA95-E63B-4764-BC33-27098D1BE48C@oracle.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD15306CB@ORSMSX113.amr.corp.intel.com> Hi Brian, Thank you for testing it out. It seems like the tests have issues handling the environment that does not have RDMA devices. I will look into this and get back to the mailing list by Monday. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Friday, September 28, 2018 4:22 PM To: Lu, Yingqi Cc: Alan Bateman ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Aundhe, Shirish ; Kaczmarek, Eric Subject: Re: adding rsockets support into JDK Hi Lucy, On Sep 25, 2018, at 12:58 PM, Lu, Yingqi > wrote: Please find webrev.11 is available athttp://cr.openjdk.java.net/~ylu/8195160.11/ [...] With the above changes, all test cases pass on my Linux CentOS 7 box. I get the failures [1, 2] when this version is run on my local build (Ubuntu 18.04 VM). Thanks, Brian [1] jdk/net/Sockets/rsocket/ServerSocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.serverSocket(RdmaNet.java:161) at jdk.net/jdk.internal.net.rdma.RdmaServerSocketChannelImpl.(RdmaServerSocketChannelImpl.java:84) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openServerSocketChannel(LinuxRdmaPollSelectorProvider.java:51) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openServerSocketChannel(RdmaPollSelectorProvider.java:67) at jdk.net/jdk.net.Sockets.openRdmaServerSocketChannel(Sockets.java:493) at SocketOptionTests.main(SocketOptionTests.java:55) [2] jdk/net/Sockets/rsocket/SocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:157) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:150) at jdk.net/jdk.internal.net.rdma.RdmaSocketChannelImpl.(RdmaSocketChannelImpl.java:96) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openSocketChannel(LinuxRdmaPollSelectorProvider.java:46) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openSocketChannel(RdmaPollSelectorProvider.java:63) at jdk.net/jdk.net.Sockets.openRdmaSocketChannel(Sockets.java:471) at SocketOptionTests.main(SocketOptionTests.java:54) -------------- next part -------------- An HTML attachment was scrubbed... URL: From xueming.shen at oracle.com Sat Sep 29 05:25:42 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 28 Sep 2018 22:25:42 -0700 Subject: RFR (Enhancement): 6194856: Zip Files lose ALL ownership and permissions of the files In-Reply-To: References: Message-ID: <5BAF0CD6.90803@oracle.com> Hi Langer, Thanks for working on this issue. I will take a look into the implementation details next week. Here are two comments regarding the "direction/approach". (1) There is a "masked" security concern regarding adding the "ownership/permission" into the jar file. "Security concern: The current signed jar spec only protects the name and content of the jar entries. If any extra info is added, its value will not be used in the signing/verifying process. This means anyone can change these info in a signed jar and the user has no way to check if the file is genuine." This is one of the reasons that it has been debated whether or not it is worth the effort to add these bits, given the permission and ownership are really platform/host/config dependent (and the jars are being downloaded/copied around across the systems with various different setup) and you really need a "root" permission to really take advantage of these bits. (2) Regarding the implementation whats the motivation of use the high byte of the "external file attributes" vs to use the info-zip as suggested in the report? I've not looked into zip/unzip implementation, which one is zip/unzip using? "A group known as INFO-zip has devised a number of different extensions for ZIP for Unix. Their first and second extension attempts added support for UID and GID but not permissions. The third Unix extension, also known as the "ASi" or "un" extension, provides for file permissions and symlinks also. These extensions have become de-facto standards, and have not changed now since 1996." -Sherman On 9/25/18, 7:57 AM, Langer, Christoph wrote: > > Hi all, > > I had asked for opinions regarding adding posix permission support to > JDK?s zip handling libraries and tools [1]. Since I didn?t get clear > ?no, you can?t do this? answers, I?m now concretely proposing some > enhancements in the area of java.util.zip, jdk.zipfs and jdk.jartool. > > I have reopened the long standing bug report (6194856) which had been > set to ?Won?t fix? quite recently for this purpose. > > Here are the technical details: > > The ZIP format specifications by infozip and pkware ([2] and [3]) do > not explicitly specify the handling of posix file permissions. But it > seems to be common sense that if file attribute compatibility is set > to ?Unix? in the upper byte of CEN field ?version made by?, the file > permissions bits are stored in CEN field ?external file attributes?, > byte 3 and 4. My changes try to honor this in the least obtrusive way. ?? > > The following changes are proposed: > > java.util.zip.ZipEntry: > > it will have an additional field ?posixPerms?. A value of -1 means ?no > permission information?, positive values will contain the flag values. > > There will be 2 new public methods to read/set the permission information: > > public Optional> > getPosixPermissions() > > public void > setPosixPermissions(Set permissions) > > The usage of type ?Optional? reflects that posix permissions are not > necessarily present in a zip file > > java.util.zip.ZipFile: > > it will have the capability to read the CEN fields and > set posixPerms if available > > java.util.zip.ZipOutputStream: > > it will store entries with associated posix > permissions as unix type in the CEN, together with the bit mask for > the permissions > > jdk.jartool: > > I propose to add and option "--preserve-posix" or short "-o" to honor > the posix bits that may be stored inside zip/jar files. By default the > option is not set and hence posix permissions are ignored. If the flag > is set and the file system that the jar tool is running on supports > posix, posix file permissions that exist in the file system will be > stored in newly created/update archives or restored to the file system > if such information is present in the archive. > > jdk.zipfs: > > I added the capability for posix file permissions in > the implementation. I decided to support PosixFileAttributes by > subclassing ZipFileAttributes from this superclass as well as > subclassing ZipFileAttributeView from PosixFileAttributeView. However, > as PosixFileAttributes also include groups and owners, I would throw > UnsupportedOperationExceptions in case of invoking methods to handle > these attributes. But this approach seems to be most consistent with > e.g. Files.setPosixFilePermissions and Files.getPosixFilePermissions. > > java.nio.file.attribute.PosixFilePermissions: > > As this class presents a collection of static helpers, > I added definitions for the posix file bit masks together with methods > to convert between Sets of PosixFilePermission to bit masks containing > the according switches and vice versa. These definitions could > theoretically also be moved inside the java.util.zip or jdk.zipfs > implementations where they wouldn?t be exposed as public APIs. > However, in that case the code would probably need to be duplicated. > > I?ve also created two jtreg testcases for both, java.util.zip and > jdk.nio.zipfs. > > The changes also contain a few further code cleanups. > > Here are the links: > > Bug: https://bugs.openjdk.java.net/browse/JDK-6194856 > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/6194856.0/ > > > I?ll write a CSR once there?s some substantial feedback to my endeavor. > > Thanks in advance for reviewing/commenting. > > Best regards > > Christoph > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-September/055375.html > > [2] http://www.info-zip.org/doc/appnote-19970311-iz.zip > > [3] https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Sun Sep 30 03:59:09 2018 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Sun, 30 Sep 2018 03:59:09 +0000 Subject: adding rsockets support into JDK In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AD15306CB@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AD12E2D13@fmsmsx158.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1475CB5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD1477F70@ORSMSX113.amr.corp.intel.com> <68b04ce0-6a16-6b23-3b62-1a400cba1b8a@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147CD89@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD147E550@ORSMSX113.amr.corp.intel.com> <583d2e38-94c1-9c38-d89d-de17e4e3f7c6@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD148A306@ORSMSX113.amr.corp.intel.com> <10aca7c2-340b-dbb9-6da9-750bd603c403@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD14EB9BC@ORSMSX113.amr.corp.intel.com> <39c952af-8e62-2774-11c6-cc1221964b64@oracle.com> <2E9E08D0-B232-4DFE-9EE6-468D06B715A5@oracle.com> <4003426e-1e73-a7b1-a5a3-5006f01fcab3@oracle.com> <0f35e51e-8297-2fe1-fd6f! ! ! ! -90d5c464e6bc@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD151C7F8@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152A195@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AD152B5C5@ORSMSX113.amr.corp.intel.com> <13D9EA95-E63B-4764-BC33-27098D1BE48C@oracle.com> <9ACD5B67AAC5594CB6268234CF29CF9AD15306CB@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AD1531734@ORSMSX113.amr.corp.intel.com> Hi Brian, Please find version 12 of the patch available at http://cr.openjdk.java.net/~ylu/8195160.12. I tested on my local Linux machine without any RDMA devices. The issue with the two test cases is resolved. Please review and let me know if you see anything else is missing. Thanks, Lucy From: nio-dev [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Lu, Yingqi Sent: Friday, September 28, 2018 4:26 PM To: Brian Burkhalter Cc: Aundhe, Shirish ; nio-dev at openjdk.java.net; Viswanathan, Sandhya ; Kaczmarek, Eric Subject: RE: adding rsockets support into JDK Hi Brian, Thank you for testing it out. It seems like the tests have issues handling the environment that does not have RDMA devices. I will look into this and get back to the mailing list by Monday. Thanks, Lucy From: Brian Burkhalter [mailto:brian.burkhalter at oracle.com] Sent: Friday, September 28, 2018 4:22 PM To: Lu, Yingqi > Cc: Alan Bateman >; nio-dev at openjdk.java.net; Viswanathan, Sandhya >; Aundhe, Shirish >; Kaczmarek, Eric > Subject: Re: adding rsockets support into JDK Hi Lucy, On Sep 25, 2018, at 12:58 PM, Lu, Yingqi > wrote: Please find webrev.11 is available athttp://cr.openjdk.java.net/~ylu/8195160.11/ [...] With the above changes, all test cases pass on my Linux CentOS 7 box. I get the failures [1, 2] when this version is run on my local build (Ubuntu 18.04 VM). Thanks, Brian [1] jdk/net/Sockets/rsocket/ServerSocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.serverSocket(RdmaNet.java:161) at jdk.net/jdk.internal.net.rdma.RdmaServerSocketChannelImpl.(RdmaServerSocketChannelImpl.java:84) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openServerSocketChannel(LinuxRdmaPollSelectorProvider.java:51) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openServerSocketChannel(RdmaPollSelectorProvider.java:67) at jdk.net/jdk.net.Sockets.openRdmaServerSocketChannel(Sockets.java:493) at SocketOptionTests.main(SocketOptionTests.java:55) [2] jdk/net/Sockets/rsocket/SocketChannel/SocketOptionTests.java java.net.SocketException: No such device at jdk.net/jdk.internal.net.rdma.RdmaNet.socket0(Native Method) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:157) at jdk.net/jdk.internal.net.rdma.RdmaNet.socket(RdmaNet.java:150) at jdk.net/jdk.internal.net.rdma.RdmaSocketChannelImpl.(RdmaSocketChannelImpl.java:96) at jdk.net/jdk.internal.net.rdma.LinuxRdmaPollSelectorProvider.openSocketChannel(LinuxRdmaPollSelectorProvider.java:46) at jdk.net/jdk.internal.net.rdma.RdmaPollSelectorProvider.openSocketChannel(RdmaPollSelectorProvider.java:63) at jdk.net/jdk.net.Sockets.openRdmaSocketChannel(Sockets.java:471) at SocketOptionTests.main(SocketOptionTests.java:54) -------------- next part -------------- An HTML attachment was scrubbed... URL: