From mikael.vidstedt at oracle.com Mon May 4 05:12:08 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Sun, 3 May 2020 22:12:08 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) Message-ID: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Please review this change which implements part of JEP 381: JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! Background: Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. Testing: A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. Cheers, Mikael [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ From Alan.Bateman at oracle.com Mon May 4 08:49:08 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 May 2020 09:49:08 +0100 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: On 04/05/2020 06:12, Mikael Vidstedt wrote: > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > I took a pass over the changes. I agree its a bit tedious. I'm sure there will be a few follow up issues as there are opportunities for cleanup in several areas. Just a few comments/questions from a first pass. ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option that was terminally deprecated in 14. The patch removes the implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). Do you want a someone to take a follow-on issue to remove the API? ResolverConfigurationImpl.localDomain0 can be removed. The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a residual reference to Solaris. JISAutoDetect - might be simpler to just initialize EUCJPName to "EUC_JP". Socket.setTrafficClass(int) swallows exceptions to workaround strange behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to that issue if you want. There is also cruft in the old plain SocketImpl that to work around eagerness to report "connection reset errors - I think we should just leave that because the old socket impl is not used by default and will be removed at some point. -Alan. From weijun.wang at oracle.com Mon May 4 14:22:38 2020 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 4 May 2020 22:22:38 +0800 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <1BA86CD8-2322-4D68-ADAE-D95311FEDF23@oracle.com> There are several security-related files (name.contains("security")) and they all look fine. --Max > On May 4, 2020, at 1:12 PM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From Roger.Riggs at oracle.com Mon May 4 14:59:30 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Mon, 4 May 2020 10:59:30 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <513265a5-8a7d-d2b9-9cfe-adcf87cca390@oracle.com> Hi Michael, Looks good. Maybe just a future cleanup to rename files, since the "...so..." is refering to solaris. src/java.base/unix/native/libjli/java_md_solinux.h src/java.base/unix/native/libjli/java_md_solinux.h Regards, Roger On 5/4/20 4:49 AM, Alan Bateman wrote: > On 04/05/2020 06:12, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was >> *extremely* mind-numbing, so I appreciate your help reviewing all the >> individual changes carefully. You may want to get that coffee cup >> filled up (or whatever keeps you awake)! >> > I took a pass over the changes. I agree its a bit tedious. I'm sure > there will be a few follow up issues as there are opportunities for > cleanup in several areas. Just a few comments/questions from a first > pass. > > ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option > that was terminally deprecated in 14. The patch removes the > implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). > Do you want a someone to take a follow-on issue to remove the API? > > ResolverConfigurationImpl.localDomain0 can be removed. > > The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a > residual reference to Solaris. > > JISAutoDetect - might be simpler to just initialize EUCJPName to > "EUC_JP". > > Socket.setTrafficClass(int) swallows exceptions to workaround strange > behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to > that issue if you want. There is also cruft in the old plain > SocketImpl that to work around eagerness to report "connection reset > errors - I think we should just leave that because the old socket impl > is not used by default and will be removed at some point. > > -Alan. From brent.christian at oracle.com Mon May 4 21:33:17 2020 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 4 May 2020 14:33:17 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: Hi, Looks fine to me. I have just one minor observation: src/java.base/share/native/libjli/emessages.h *** 92,102 **** #define JRE_ERROR5 "Error: Failed to start a %d-bit JVM process from a %d-bit JVM." ! #define JRE_ERROR6 "Error: Verify all necessary Java SE components have been installed.\n(Solaris SPARC 64-bit components must be installed after 32-bit components.)" #define JRE_ERROR7 "Error: Either 64-bit processes are not supported by this platform\nor the 64-bit components have not been installed." --- 91,101 ---- #define JRE_ERROR5 "Error: Failed to start a %d-bit JVM process from a %d-bit JVM." ! #define JRE_ERROR6 "Error: Verify all necessary Java SE components have been installed.\n" #define JRE_ERROR7 "Error: Either 64-bit processes are not supported by this platform\nor the 64-bit components have not been installed." The other error messages do not include a trailing newline. -Brent On 5/3/20 10:12 PM, Mikael Vidstedt wrote: > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From michael.x.mcmahon at oracle.com Tue May 5 11:16:54 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 5 May 2020 12:16:54 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel Message-ID: Hi, Could I get the following change reviewed please? The change is to add new static factory creation methods to java.nio.channels.SocketChannel and java.nio.channels.ServerSocketChannel which take a java.net.ProtocolFamily. These are basically the same as a similar method that was added to DatagramChannel back in 1.7 and which allow the creation of IPv4 only channels or IPv4/IPv6. The change will also facilitate the 'Unix domain' protocol family work in JEP-380 [4] The webrev is at [1] The CSR for the spec change is at [2] and the bugid is [3]. Thanks, Michael. [1] http://cr.openjdk.java.net/~michaelm/8241305/webrev.1/ [2] https://bugs.openjdk.java.net/browse/JDK-8242926 [3] https://bugs.openjdk.java.net/browse/JDK-8241305 [4] https://openjdk.java.net/jeps/380 From daniel.fuchs at oracle.com Tue May 5 12:42:37 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 5 May 2020 13:42:37 +0100 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <779ec619-f61c-be8a-793d-15777e783e8e@oracle.com> Hi Mikael, I spotted another place where a residual reference to Solaris remains in a comment: src/java.base/unix/native/libnet/PlainSocketImpl.c 857 #if defined(_AIX) 858 if (errno == EINVAL) { 859 // On Solaris setsockopt will set errno to EINVAL if the socket 860 // is closed. The default error message is then confusing Otherwise I had a look at the networking related files and the changes seemed reasonable to me. best regards, -- daniel On 04/05/2020 06:12, Mikael Vidstedt wrote: > Please review this change which implements part of JEP 381: > > JBS:https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev:http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ > JEP:https://bugs.openjdk.java.net/browse/JDK-8241787 > From Alan.Bateman at oracle.com Tue May 5 12:53:59 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 May 2020 13:53:59 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: On 05/05/2020 12:16, Michael McMahon wrote: > Hi, > > Could I get the following change reviewed please? > > The change is to add new static factory creation methods > to java.nio.channels.SocketChannel and > java.nio.channels.ServerSocketChannel > which take a java.net.ProtocolFamily. These are basically the same as > a similar method that was added to DatagramChannel back in 1.7 > and which allow the creation of IPv4 only channels or IPv4/IPv6. The javadoc and implementation looks good. Minor comment is that the requiresNonNull is not really needed in SocketChannel, ServerSocketChannel and DatagramSocket as they just delegate to the provider which is specified to throw NPE already. Also the second/subsequent lines of the @implSpec in SelectorProvider isn't misaligned. It might be simpler to just not indent those lines. The convention in this area is to organize the tests by class name. There is an etc directory for tests that cover many channels that might work. OpenDefault.java - the @summary says that it tests the "no-arg open method" but the test actually tests the socket address type returned by the adaptor's getLocalAddress and getInetAddress methods. It read is as adding test coverage to the existing APIs rather than the new APIs. A few comments: - I think we need to find a better name for this test, maybe LocalSocketAddressType ? - Can it be expanded to test getLocalAddress rather than just the socket adaptor methods? - The test names are openXXX but might be clearer if renamed to something like testSocketChannel, testServerSocketChannel, .. as it tests the local addresses that these channel report. OpenConnect.java - this is a good combo test and you might want to beef up the @summary to more fully describe what it does. You might want to think about renaming too, maybe OpenAndConnect. A few questions/comments: - Would it be possible to summarize the platform specific issues that you are running into? Most of the MLS elements seem to be DatagramChannel and it would be useful to get a summary of that issue (I suspect part of it is trying to connect a UDP connect to the wildcard address, something that doesn't make sense). - Why is this test /othervm and is /timeout=30 needed? - What is the purpose of cfactory and sfactory? I can't see how they could ever be null. If they are removed then it would allow the static openXXX methods to be cleaned up. At the use-sites it checks for null and I assume they should go away too. - I think linkLocalAddr should be removed or renamed to something like ipv6LinkLocalAddress to make it clearer that it returns true when the address is an IPv6 link local address. Alternatively might be clearer to just filter at the two use-sites. - connectNonBlocking doesn't look quite right. The connect method returns a boolean to indicate if the connection is established. If it returns false then you can register it with the Selector and select. - A minor point is that the parameters can be ProtocolFamily rather than StandardProtocolFamily. - There are a couple of places the exceptions have unusual names, e.g. catch (UnsupportedOperationException re), catch (Exception re).? Where these RuntimeException at some stage, maybe they can be renamed now. I skimmed through the ProtocolFamilies test too and have several comments but don't have time today to write all those down, I'll get to in the next few days. -Alan. From chris.hegarty at oracle.com Tue May 5 14:47:13 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 5 May 2020 15:47:13 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: <673A0A20-3920-4493-AF1C-42815BA1943F@oracle.com> Michael, > On 5 May 2020, at 12:16, Michael McMahon wrote: > > ... > > > [1] http://cr.openjdk.java.net/~michaelm/8241305/webrev.1/ I think this is very good. I already provided comments on the CSR/spec, and pushed any implementation/test review changes directly to the sandbox. -Chris. From chris.hegarty at oracle.com Tue May 5 15:00:22 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 5 May 2020 16:00:22 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: <17B8DE1A-EBB8-4614-96F3-519404E7D5E1@oracle.com> > On 5 May 2020, at 13:53, Alan Bateman wrote: > > ... > > Minor comment is that the requiresNonNull is not really needed in SocketChannel, ServerSocketChannel and DatagramSocket as they just delegate to the provider which is specified to throw NPE already. These are static methods that delegate to a pluggable provider. As such, it would be better for them to not rely on the null checking behaviour of the actual provider plugged in. A plugged-in provider may mistakenly throw an exception other than NPE, or delegate to a channel constructor which would end up constructing an instance of the channel to only then abandon it mid-initialization. On balance, it seems best to catch any potential bad null as early as possible, rather than waiting for it to (hopefully) be caught later. -Chris. From daniel.fuchs at oracle.com Wed May 6 15:56:56 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 6 May 2020 16:56:56 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: <1b776c0a-b0c1-6044-3b6d-75e0c10f52c3@oracle.com> Hi Michael, java/nio/channels/OpenConnect.java Why is the test using reflection? Is it so that you can run it on a platform that doesn't have the JEP implementation in? Otherwise I like the formatting of the data provider: very clear! It's a bit surprising to see that contrarily to SocketChannel, DatagramChannel seems to have system dependent behaviors, but I'm suspecting this is caused by passing wildcard addresses to connect? java/nio/channels/ProtocolFamilies.java 60 static final boolean preferIPv6 = 61 parseBoolean(getProperty("java.net.preferIPv6Addresses", "false")); You could use IPSupport::preferIPv6Addresses there - it's been added to IPSupport since that test was written. 101 if (expExType == UOE) { 102 assertThrows(UOE, () -> openSC(fam)); 103 } else { 104 openSC(fam); 105 } That's a strange pattern. I'd have expected something like: 101 if (expExType != null) { 102 assertThrows(expExType, () -> openSC(fam)); 103 } else { 104 openSC(fam); 105 } (this pattern appears at several places in this file) 395 private static Inet4Address getFirstLinkLocalIPv4Address() 396 throws Exception { 397 return NetworkConfiguration.probe() 398 .ip4Addresses() 399 .filter(a -> !a.isLoopbackAddress()) 400 .findFirst() 401 .orElse(null); 402 } The method above doesn't seem to be doing what its name implies it does? It doesn't return the first link local address but the first address that is not the loopback. Link local addresses have proved to be problematic on some macOS configuration. I'm curious to know why would this test needs them? best regards, -- daniel On 05/05/2020 12:16, Michael McMahon wrote: > Hi, > > Could I get the following change reviewed please? > > The change is to add new static factory creation methods > to java.nio.channels.SocketChannel and > java.nio.channels.ServerSocketChannel > which take a java.net.ProtocolFamily. These are basically the same as > a similar method that was added to DatagramChannel back in 1.7 > and which allow the creation of IPv4 only channels or IPv4/IPv6. > > The change will also facilitate the 'Unix domain' protocol family work > in JEP-380 [4] > > The webrev is at [1] The CSR for the spec change is at [2] > and the bugid is [3]. > > Thanks, > > Michael. > > > [1] http://cr.openjdk.java.net/~michaelm/8241305/webrev.1/ > > [2] https://bugs.openjdk.java.net/browse/JDK-8242926 > > [3] https://bugs.openjdk.java.net/browse/JDK-8241305 > > [4] https://openjdk.java.net/jeps/380 > From mikael.vidstedt at oracle.com Thu May 7 04:38:16 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:38:16 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <6854A858-ACA9-4572-868B-3745A2E30332@oracle.com> Alan, thank you for the review! New webrev coming. Meanwhile comments inline.. > On May 4, 2020, at 1:49 AM, Alan Bateman wrote: > > On 04/05/2020 06:12, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> > I took a pass over the changes. I agree its a bit tedious. I'm sure there will be a few follow up issues as there are opportunities for cleanup in several areas. Just a few comments/questions from a first pass. > > ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option that was terminally deprecated in 14. The patch removes the implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). Do you want a someone to take a follow-on issue to remove the API? I would very much appreciate if somebody could take that on! > ResolverConfigurationImpl.localDomain0 can be removed. Fixed. > The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a residual reference to Solaris. Fixed. > JISAutoDetect - might be simpler to just initialize EUCJPName to "EUC_JP?. I was going back and forth on this one - in a way preserving the method was nicely symmetric with getSJISName(), but in the end I removed the method in favor of initializing the variable directly. > Socket.setTrafficClass(int) swallows exceptions to workaround strange behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to that issue if you want. There is also cruft in the old plain SocketImpl that to work around eagerness to report "connection reset errors - I think we should just leave that because the old socket impl is not used by default and will be removed at some point. Anything I can do to keep the complexity of this patch down is appreciated. I have *not* changed anything, let me know if you want me to do something with this. Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 04:39:47 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:39:47 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <1BA86CD8-2322-4D68-ADAE-D95311FEDF23@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <1BA86CD8-2322-4D68-ADAE-D95311FEDF23@oracle.com> Message-ID: <369FDCBB-F56F-4840-8808-3E3F2CF05768@oracle.com> Thank you for reviewing, Max! Cheers, Mikael > On May 4, 2020, at 7:22 AM, Weijun Wang wrote: > > There are several security-related files (name.contains("security")) and they all look fine. > > --Max > > >> On May 4, 2020, at 1:12 PM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> > From mikael.vidstedt at oracle.com Thu May 7 04:44:49 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:44:49 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <3E46F5BA-F52A-417D-9B10-C2F371F9732D@oracle.com> > On May 4, 2020, at 2:33 PM, Brent Christian wrote: > > Hi, > > Looks fine to me. I have just one minor observation: > > src/java.base/share/native/libjli/emessages.h > > *** 92,102 **** > #define JRE_ERROR5 "Error: Failed to start a %d-bit JVM process from a %d-bit JVM." > ! #define JRE_ERROR6 "Error: Verify all necessary Java SE components have been installed.\n(Solaris SPARC 64-bit components must be installed after 32-bit components.)" > #define JRE_ERROR7 "Error: Either 64-bit processes are not supported by this platform\nor the 64-bit components have not been installed." > --- 91,101 ---- > #define JRE_ERROR5 "Error: Failed to start a %d-bit JVM process from a %d-bit JVM." > ! #define JRE_ERROR6 "Error: Verify all necessary Java SE components have been installed.\n" > #define JRE_ERROR7 "Error: Either 64-bit processes are not supported by this platform\nor the 64-bit components have not been installed." > > > The other error messages do not include a trailing newline. Nice catch! Fixed in the upcoming webrev. Thank you for this review - and the pre-review before it! :) Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 04:43:56 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:43:56 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <26996994-3e08-c312-c1fa-9b2ffd5b5d95@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <26996994-3e08-c312-c1fa-9b2ffd5b5d95@oracle.com> Message-ID: <3E353A59-B7E2-4924-9A94-25C2C42ACA2C@oracle.com> > On May 4, 2020, at 9:27 AM, naoto.sato at oracle.com wrote: > > Hi Mikael, > > I took a look at i18n related files. It looks good overall. > > One nit in java/nio/charset/Charset/DefaultCharsetTest.java: If the test is only applicable to linux (@requires os.family == "linux" in jtreg tag after the change), the condition isLinux() is no longer needed at line 73. Good catch! Fixed in the upcoming webrev. Thank you for the review! Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 04:43:17 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:43:17 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <513265a5-8a7d-d2b9-9cfe-adcf87cca390@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <513265a5-8a7d-d2b9-9cfe-adcf87cca390@oracle.com> Message-ID: I have always wondered what ?solinux? is supposed to mean - though not enough to actually ask anybody :) I?ll file a follow-up enhancement to cover renaming the files. Thank you for the review! Cheers, Mikael > On May 4, 2020, at 7:59 AM, Roger Riggs wrote: > > Hi Michael, > > Looks good. > > Maybe just a future cleanup to rename files, since the "...so..." is refering to solaris. > > src/java.base/unix/native/libjli/java_md_solinux.h > src/java.base/unix/native/libjli/java_md_solinux.h > > Regards, Roger > > > On 5/4/20 4:49 AM, Alan Bateman wrote: >> On 04/05/2020 06:12, Mikael Vidstedt wrote: >>> Please review this change which implements part of JEP 381: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>> >>> >>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>> >> I took a pass over the changes. I agree its a bit tedious. I'm sure there will be a few follow up issues as there are opportunities for cleanup in several areas. Just a few comments/questions from a first pass. >> >> ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option that was terminally deprecated in 14. The patch removes the implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). Do you want a someone to take a follow-on issue to remove the API? >> >> ResolverConfigurationImpl.localDomain0 can be removed. >> >> The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a residual reference to Solaris. >> >> JISAutoDetect - might be simpler to just initialize EUCJPName to "EUC_JP". >> >> Socket.setTrafficClass(int) swallows exceptions to workaround strange behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to that issue if you want. There is also cruft in the old plain SocketImpl that to work around eagerness to report "connection reset errors - I think we should just leave that because the old socket impl is not used by default and will be removed at some point. >> >> -Alan. > From mikael.vidstedt at oracle.com Thu May 7 04:45:34 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:45:34 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <779ec619-f61c-be8a-793d-15777e783e8e@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <779ec619-f61c-be8a-793d-15777e783e8e@oracle.com> Message-ID: > On May 5, 2020, at 5:42 AM, Daniel Fuchs wrote: > > Hi Mikael, > > I spotted another place where a residual reference to Solaris > remains in a comment: > src/java.base/unix/native/libnet/PlainSocketImpl.c > > 857 #if defined(_AIX) > 858 if (errno == EINVAL) { > 859 // On Solaris setsockopt will set errno to EINVAL if the socket > 860 // is closed. The default error message is then confusing Fixed in the upcoming webrev! > Otherwise I had a look at the networking related files and > the changes seemed reasonable to me. Thanks a lot for the review! Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 04:56:16 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 21:56:16 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> Message-ID: <3A65A5FD-BA6B-41C6-BF85-27746EF3E1A0@oracle.com> New webrev addressing the feedback/comments I have received: webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/corelibs/open/webrev/ incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/corelibs.incr/open/webrev/ I hope I caught everything. Some outstanding items: * File follow-up enhancement for the removal of SO_FLOW_SA and jdk.net.SocketFlow Would appreciate if somebody could help file this for me. * File follow-up enhancement for renaming the ?solinux? files * Get confirmation from Alan that the Socket.setTrafficClass(int) and SocketImpl stuff is good as-is Cheers, Mikael > On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/corelibs/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From Alan.Bateman at oracle.com Thu May 7 06:41:44 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 May 2020 07:41:44 +0100 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <3A65A5FD-BA6B-41C6-BF85-27746EF3E1A0@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <3A65A5FD-BA6B-41C6-BF85-27746EF3E1A0@oracle.com> Message-ID: On 07/05/2020 05:56, Mikael Vidstedt wrote: > : > > * File follow-up enhancement for the removal of SO_FLOW_SA and jdk.net.SocketFlow I've created JDK-8244582 to track this, we should try to this in the same release as JEP 381. > : > > > * Get confirmation from Alan that the Socket.setTrafficClass(int) and SocketImpl stuff is good as-is > JDK-8221487 tracks the Socket.setTrafficClass issue. I didn't create an issue to mop up issues in the old SocketImpl as it is not used (at least not by default) since JDK 13. The intention is to completely remove the old SocketImpl implementation, don't know which release yet but it's hardly seems worth spending time doing cleanup on this code. I've no doubt there will be cleanup in many areas of the JDK code after JEP 381 is pushed. -Alan. From mikael.vidstedt at oracle.com Thu May 7 21:24:33 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 7 May 2020 14:24:33 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <3A65A5FD-BA6B-41C6-BF85-27746EF3E1A0@oracle.com> Message-ID: > On May 6, 2020, at 11:41 PM, Alan Bateman wrote: > > On 07/05/2020 05:56, Mikael Vidstedt wrote: >> : >> >> * File follow-up enhancement for the removal of SO_FLOW_SA and jdk.net.SocketFlow > I've created JDK-8244582 to track this, we should try to this in the same release as JEP 381. Thank you! > >> : >> >> >> * Get confirmation from Alan that the Socket.setTrafficClass(int) and SocketImpl stuff is good as-is >> > JDK-8221487 tracks the Socket.setTrafficClass issue. I didn't create an issue to mop up issues in the old SocketImpl as it is not used (at least not by default) since JDK 13. The intention is to completely remove the old SocketImpl implementation, don't know which release yet but it's hardly seems worth spending time doing cleanup on this code. I've no doubt there will be cleanup in many areas of the JDK code after JEP 381 is pushed. Sounds good, thank you! Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 21:27:28 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 7 May 2020 14:27:28 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <32A4EA4A-D640-47F4-8A8A-1888BC016E14@vmware.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <513265a5-8a7d-d2b9-9cfe-adcf87cca390@oracle.com> <32A4EA4A-D640-47F4-8A8A-1888BC016E14@vmware.com> Message-ID: <0A8DB9F8-E535-48F1-B28A-B71257596C12@oracle.com> > On May 7, 2020, at 7:52 AM, Kumar Srinivasan wrote: > > Hi Mikael, > > I may have created solinux when the macosx port was merged and in an effort to reduce the CPP conditionals. > solinux = solaris + linux ie. Vanilla unix code vs Darwin code IIRC has Objective-C, MacOSX specific thread initialization etc. Thank you for the background - it all makes sense now! :) > I looked over the launcher related code looks ok to me. > > I did notice the \n endings for the messages that Brent pointed out. Thank you for the review! The line ending should be fixed in webrev.01. Cheers, Mikael > >> On May 6, 2020, at 9:43 PM, Mikael Vidstedt > wrote: >> >> >> I have always wondered what ?solinux? is supposed to mean - though not enough to actually ask anybody :) >> >> I?ll file a follow-up enhancement to cover renaming the files. >> >> Thank you for the review! >> >> Cheers, >> Mikael >> >>> On May 4, 2020, at 7:59 AM, Roger Riggs > wrote: >>> >>> Hi Michael, >>> >>> Looks good. >>> >>> Maybe just a future cleanup to rename files, since the "...so..." is refering to solaris. >>> >>> src/java.base/unix/native/libjli/java_md_solinux.h >>> src/java.base/unix/native/libjli/java_md_solinux.h >>> >>> Regards, Roger >>> >>> >>> On 5/4/20 4:49 AM, Alan Bateman wrote: >>>> On 04/05/2020 06:12, Mikael Vidstedt wrote: >>>>> Please review this change which implements part of JEP 381: >>>>> >>>>> JBS: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8244224&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=W5VvD1owIGoUcbcRkcwixXGPkFLjHUFof2v6cxMqchk%3D&reserved=0 >>>>> webrev: https://nam04.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~mikael%2Fwebrevs%2F8244224%2Fwebrev.00%2Fcorelibs%2Fopen%2Fwebrev%2F&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=tQZIjuF5cIPs%2FNqTRtY2WtmwAgwa0iv205wQ9vk0vOQ%3D&reserved=0 >>>>> JEP: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8241787&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=rDvBE%2BJ2F1IIFC9fbA92rs0KZNgYPg0JZM2ynqp7mcs%3D&reserved=0 >>>>> >>>>> >>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>> >>>> I took a pass over the changes. I agree its a bit tedious. I'm sure there will be a few follow up issues as there are opportunities for cleanup in several areas. Just a few comments/questions from a first pass. >>>> >>>> ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option that was terminally deprecated in 14. The patch removes the implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). Do you want a someone to take a follow-on issue to remove the API? >>>> >>>> ResolverConfigurationImpl.localDomain0 can be removed. >>>> >>>> The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a residual reference to Solaris. >>>> >>>> JISAutoDetect - might be simpler to just initialize EUCJPName to "EUC_JP". >>>> >>>> Socket.setTrafficClass(int) swallows exceptions to workaround strange behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to that issue if you want. There is also cruft in the old plain SocketImpl that to work around eagerness to report "connection reset errors - I think we should just leave that because the old socket impl is not used by default and will be removed at some point. >>>> >>>> -Alan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Fri May 8 08:20:40 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 8 May 2020 10:20:40 +0200 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports - (core libraries) In-Reply-To: <0A8DB9F8-E535-48F1-B28A-B71257596C12@oracle.com> References: <3905A7D9-C6DB-4A5C-BBBB-A16EEA3BB7F5@oracle.com> <513265a5-8a7d-d2b9-9cfe-adcf87cca390@oracle.com> <32A4EA4A-D640-47F4-8A8A-1888BC016E14@vmware.com> <0A8DB9F8-E535-48F1-B28A-B71257596C12@oracle.com> Message-ID: <41020a42-1334-8cb8-fc4b-a42952cffa3a@oracle.com> On 2020-05-07 23:27, Mikael Vidstedt wrote: > >> On May 7, 2020, at 7:52 AM, Kumar Srinivasan wrote: >> >> Hi Mikael, >> >> I may have created solinux when the macosx port was merged and in an effort to reduce the CPP conditionals. >> solinux = solaris + linux ie. Vanilla unix code vs Darwin code IIRC has Objective-C, MacOSX specific thread initialization etc. > Thank you for the background - it all makes sense now! :) If the file is truly only used on linux, it should be moved to java.base/linux/native/libjli instead. (And be renamed.) If it's still used on other unix platforms (read: AIX) it should probably be renamed java_md_unix.h instead, and kept in place. If this change should be done in a follow-up fix or as part of JEP 381, I leave to you to decide. (edit: *actually looking at the files* ... ) Hm. The include block looks like this: #if defined(_AIX) #include "java_md_aix.h" #endif #ifdef MACOSX #include "java_md_macosx.h" #else? /* !MACOSX */ #include "java_md_solinux.h" #endif /* MACOSX */ #endif /* JAVA_MD_H */ It would probably make sense to make this a three-pronged include with separate files for aix, macosx and linux, but that will probably require stuff to migrate from java_md_solinux.h to java_md_aix.h, or at the very least, proper testing on AIX. Recommend filing follow-up issue to sort this out. /Magnus > >> I looked over the launcher related code looks ok to me. >> >> I did notice the \n endings for the messages that Brent pointed out. > Thank you for the review! The line ending should be fixed in webrev.01. > > Cheers, > Mikael > >>> On May 6, 2020, at 9:43 PM, Mikael Vidstedt > wrote: >>> >>> >>> I have always wondered what ?solinux? is supposed to mean - though not enough to actually ask anybody :) >>> >>> I?ll file a follow-up enhancement to cover renaming the files. >>> >>> Thank you for the review! >>> >>> Cheers, >>> Mikael >>> >>>> On May 4, 2020, at 7:59 AM, Roger Riggs > wrote: >>>> >>>> Hi Michael, >>>> >>>> Looks good. >>>> >>>> Maybe just a future cleanup to rename files, since the "...so..." is refering to solaris. >>>> >>>> src/java.base/unix/native/libjli/java_md_solinux.h >>>> src/java.base/unix/native/libjli/java_md_solinux.h >>>> >>>> Regards, Roger >>>> >>>> >>>> On 5/4/20 4:49 AM, Alan Bateman wrote: >>>>> On 04/05/2020 06:12, Mikael Vidstedt wrote: >>>>>> Please review this change which implements part of JEP 381: >>>>>> >>>>>> JBS: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8244224&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=W5VvD1owIGoUcbcRkcwixXGPkFLjHUFof2v6cxMqchk%3D&reserved=0 >>>>>> webrev: https://nam04.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~mikael%2Fwebrevs%2F8244224%2Fwebrev.00%2Fcorelibs%2Fopen%2Fwebrev%2F&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=tQZIjuF5cIPs%2FNqTRtY2WtmwAgwa0iv205wQ9vk0vOQ%3D&reserved=0 >>>>>> JEP: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8241787&data=02%7C01%7Ckusrinivasan%40vmware.com%7Ce48bd44f0c484e6fd85608d7f243f1ff%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637244245954061434&sdata=rDvBE%2BJ2F1IIFC9fbA92rs0KZNgYPg0JZM2ynqp7mcs%3D&reserved=0 >>>>>> >>>>>> >>>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>>> >>>>> I took a pass over the changes. I agree its a bit tedious. I'm sure there will be a few follow up issues as there are opportunities for cleanup in several areas. Just a few comments/questions from a first pass. >>>>> >>>>> ExtendedSocketOption.SO_FLOW_SLA is the Solaris specific socket option that was terminally deprecated in 14. The patch removes the implementation but leave the API (SO_FLOW_SA and jdk.net.SocketFlow). Do you want a someone to take a follow-on issue to remove the API? >>>>> >>>>> ResolverConfigurationImpl.localDomain0 can be removed. >>>>> >>>>> The comment on mcast_join_leave in PlainDatagramSocketImpl.c has a residual reference to Solaris. >>>>> >>>>> JISAutoDetect - might be simpler to just initialize EUCJPName to "EUC_JP". >>>>> >>>>> Socket.setTrafficClass(int) swallows exceptions to workaround strange behaviour on Solaris. Tracked as JDK-8221487 so okay to leave it to that issue if you want. There is also cruft in the old plain SocketImpl that to work around eagerness to report "connection reset errors - I think we should just leave that because the old socket impl is not used by default and will be removed at some point. >>>>> >>>>> -Alan. From michael.x.mcmahon at oracle.com Fri May 8 09:41:39 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 8 May 2020 10:41:39 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: Hi Alan I think we have dealt with your comments below. As regards the question about platform specific differences shown by the OpenAndConnect test, yes much of it results in the case where the server binds to a wildcard address and then the client connects to that. It was a surprise to me that the API supports that at all, which it does. So, the test is really just documenting the differences. It does seem to be mostly windows that disallows it. Also, I didn't remove the null checks for protocol family. While it is a duplication with respect to the default provider, it does cover the case where a provider does not perform the check. I don't have a strong feeling about it, and don't mind removing it. I have updated the webrev at: http://cr.openjdk.java.net/~michaelm/8241305/webrev.2/ If you have any comments about the third test, let me know when you get a chance Thanks, Michael On 05/05/2020 13:53, Alan Bateman wrote: > On 05/05/2020 12:16, Michael McMahon wrote: >> Hi, >> >> Could I get the following change reviewed please? >> >> The change is to add new static factory creation methods >> to java.nio.channels.SocketChannel and >> java.nio.channels.ServerSocketChannel >> which take a java.net.ProtocolFamily. These are basically the same as >> a similar method that was added to DatagramChannel back in 1.7 >> and which allow the creation of IPv4 only channels or IPv4/IPv6. > The javadoc and implementation looks good. > > Minor comment is that the requiresNonNull is not really needed in > SocketChannel, ServerSocketChannel and DatagramSocket as they just > delegate to the provider which is specified to throw NPE already. Also > the second/subsequent lines of the @implSpec in SelectorProvider isn't > misaligned. It might be simpler to just not indent those lines. > > The convention in this area is to organize the tests by class name. > There is an etc directory for tests that cover many channels that > might work. > > OpenDefault.java - the @summary says that it tests the "no-arg open > method" but the test actually tests the socket address type returned > by the adaptor's getLocalAddress and getInetAddress methods. It read > is as adding test coverage to the existing APIs rather than the new > APIs. A few comments: > - I think we need to find a better name for this test, maybe > LocalSocketAddressType ? > - Can it be expanded to test getLocalAddress rather than just the > socket adaptor methods? > - The test names are openXXX but might be clearer if renamed to > something like testSocketChannel, testServerSocketChannel, .. as it > tests the local addresses that these channel report. > > OpenConnect.java - this is a good combo test and you might want to > beef up the @summary to more fully describe what it does. You might > want to think about renaming too, maybe OpenAndConnect. A few > questions/comments: > - Would it be possible to summarize the platform specific issues that > you are running into? Most of the MLS elements seem to be > DatagramChannel and it would be useful to get a summary of that issue > (I suspect part of it is trying to connect a UDP connect to the > wildcard address, something that doesn't make sense). > - Why is this test /othervm and is /timeout=30 needed? > - What is the purpose of cfactory and sfactory? I can't see how they > could ever be null. If they are removed then it would allow the static > openXXX methods to be cleaned up. At the use-sites it checks for null > and I assume they should go away too. > - I think linkLocalAddr should be removed or renamed to something like > ipv6LinkLocalAddress to make it clearer that it returns true when the > address is an IPv6 link local address. Alternatively might be clearer > to just filter at the two use-sites. > - connectNonBlocking doesn't look quite right. The connect method > returns a boolean to indicate if the connection is established. If it > returns false then you can register it with the Selector and select. > - A minor point is that the parameters can be ProtocolFamily rather > than StandardProtocolFamily. > - There are a couple of places the exceptions have unusual names, e.g. > catch (UnsupportedOperationException re), catch (Exception re).? Where > these RuntimeException at some stage, maybe they can be renamed now. > > I skimmed through the ProtocolFamilies test too and have several > comments but don't have time today to write all those down, I'll get > to in the next few days. > > -Alan. > > From Alan.Bateman at oracle.com Fri May 8 10:52:11 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 8 May 2020 11:52:11 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: On 08/05/2020 10:41, Michael McMahon wrote: > Hi Alan > > I think we have dealt with your comments below. As regards the > question about platform > specific differences shown by the OpenAndConnect test, yes much of it > results in the case > where the server binds to a wildcard address and then the client > connects to that. > It was a surprise to me that the API supports that at all, which it > does. So, the test > is really just documenting the differences. It does seem to be mostly > windows that disallows it. It looks a bit strange to try connect a DatagramChannel to a target that is any address. Daniel, Patrick and I have chatted about a few times. It comes up with tests that bind to the wildcard address and then use getLocalAddress to get the target address for the test. One possibility is to translate it to a specific address (as is done in SocketChannel and java.net.Socket) but that is something for a separate issue and not clear to me that we should adding a test for it here. Would it possible to summarize the other Windows specific issues that are being "documented" ? -Alan From michael.x.mcmahon at oracle.com Fri May 8 17:23:07 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 8 May 2020 18:23:07 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: <4cb939d5-2892-ac38-67f8-8a74bc3b56d6@oracle.com> On 08/05/2020 11:52, Alan Bateman wrote: > On 08/05/2020 10:41, Michael McMahon wrote: >> Hi Alan >> >> I think we have dealt with your comments below. As regards the >> question about platform >> specific differences shown by the OpenAndConnect test, yes much of it >> results in the case >> where the server binds to a wildcard address and then the client >> connects to that. >> It was a surprise to me that the API supports that at all, which it >> does. So, the test >> is really just documenting the differences. It does seem to be mostly >> windows that disallows it. > It looks a bit strange to try connect a DatagramChannel to a target > that is any address. Daniel, Patrick and I have chatted about a few > times. It comes up with tests that bind to the wildcard address and > then use getLocalAddress to get the target address for the test. One > possibility is to translate it to a specific address (as is done in > SocketChannel and java.net.Socket) but that is something for a > separate issue and not clear to me that we should adding a test for it > here. > > Would it possible to summarize the other Windows specific issues that > are being "documented" ? > > -Alan It's not just Windows and it's hard to discern patterns from the test, the way it is structured, but another one would be binding the first socket to an IPv4 address (loopback or actual local address) and binding the second socket to an IPv6 equivalent. This fails for SocketChannel on all platforms, and also for DatagramChannel on all platforms except MacOS. If you want I can cut the test back to only test positive scenarios that should work across all platforms. Michael From Alan.Bateman at oracle.com Sat May 9 08:06:29 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 9 May 2020 09:06:29 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <4cb939d5-2892-ac38-67f8-8a74bc3b56d6@oracle.com> References: <4cb939d5-2892-ac38-67f8-8a74bc3b56d6@oracle.com> Message-ID: <1015be23-aa9a-d323-59e4-932eb143120b@oracle.com> On 08/05/2020 18:23, Michael McMahon wrote: > : > > It's not just Windows and it's hard to discern patterns from the test, > the way it is structured, > but another one would be binding the first socket to an IPv4 address > (loopback or actual local address) > and binding the second socket to an IPv6 equivalent. This fails for > SocketChannel on all platforms, > and also for DatagramChannel on all platforms except MacOS. > > If you want I can cut the test back to only test positive scenarios > that should work across all platforms. Negative scenarios are good too but I think we have to weed out the test cases that don't make sense. I think the main issue with the matrix is that very row with saddr=null or saddr=xxANYLOCAL creates a nonsensical scenario for DatagramChannel.connect. Maybe you could put a special value "should pass on DC" element for these rows so that they are skipped? The other set of issues arises with the rows that have a value other than ALL or ~ALL in the "should pass on SC" column. The ML elements seem to be cases on Windows where a listener is bound to the wildcard address but the client cannot connect using the loopback address, is that right? I think we need a bit more information on what is going on there as I would expect that should work everywhere. (A subtle change in the patch is that it switch connect-to-wildcard to connect to the loopback rather than host address, but I don't think this explains it). There are a few other fishy. Is it possible that IP4LOCAL and IP6LOCAL are non loopback addresses on some test systems?? There are several left over comments such as "hangs L", and "true on macOS??" and I wonder if this might be the root of some of platform variance that you are running into. The rename/move of LocalSocketAddressType is good. You might want to double-check the addresses data provider as it's not clear (to me) why it adds InetAddress.getLocalHost and getLoopbackAddress to the stream, don't you end up with duplicates? I would expect you could siumplify it down to something like: ??????? NetworkConfiguration nc = NetworkConfiguration.probe(); ??????? return Stream.concat(nc.ip4Addresses(), nc.ip6Addresses()) ??????????????? .map(ia -> new Object[] { new InetSocketAddress(ia, 0) }) ??????????????? .iterator(); A minor is is that the "throws Exception" in each of the tests have ended up on their own line, no need for that I assume. -Alan. From Alan.Bateman at oracle.com Sat May 9 14:25:35 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 9 May 2020 15:25:35 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: Message-ID: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> On 08/05/2020 10:41, Michael McMahon wrote: > : > > If you have any comments about the third test, let me know when you > get a chance > I found a bit of time to look at the 3rd test, ProtocolFamilies. It provides good coverage but will need a round or two of cleanup. What is the purpose of running with -Djava.net.preferIPv6Addresses=true? That influences the ordering of lookups with InetAddress and I can't obviously see how it relates to this test. ia4 = getFirstLinkLocalIPv4Address().? The naming is problematic as it doesn't return a link-local address (it might do on a mis-configured system but we usually try to avoid such addresses). Instead it finds a local IPv4 address that is not a loopback address.? Maybe rename it findLocalIPv4Address and put a comment that it may return null? ia6 = getFirstLinkLocalIPv6Address().? Do you really want an IPv6 link local address or just a local address that is not the loopback? (I'm asking as its different to the IPv4 version on a number of points, including returning ::0 rather than null when an address is not found). Is dcOpenConnect needed in this test? This has the same problem as the other test where it is trying to connect a UDP socket to the wildcard address so doesn't make sense. It looks like the xxOpen tests are leaking file descriptors. Is this the reason why the default @run is /othervm? I'd prefer if the sockets were closed so that we at least have the option of running the test in agent VM mode. A minor comment is that I think I'd prefer if the parameters the xxOpen methods were "family" and something like "expectedException". The exception expected in the xxOpenBind tests with the openBind data provider is a bit confusing. It looks like the rows with "UOE" for the IPv4-only case are dups of the xxOpen tests? If you drop these rows then it would avoid special casing UOE and would allow you to drop "open" from the names too. defaultImpl() seems to test a custom SelectorProvider (not the default implementation) so maybe it should be renamed to testCustomProvider and SELECTOR_PROVIDER should be renamed too. Trivially, "nulls" and "uoe" might be better as testNulls and testUOE. Naming the unknown protocol family "STITCH" looks a bit strange, can this be something more obvious such as "CUSTOM_PF", "UNKNOWN_PF" or "BAD_PF"? -Alan. From michael.x.mcmahon at oracle.com Wed May 13 21:51:10 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 13 May 2020 22:51:10 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> Message-ID: I have updated the webrev for this at http://cr.openjdk.java.net/~michaelm/8241305/webrev.4/ based on the last round of comments. Thanks, Michael. On 09/05/2020 15:25, Alan Bateman wrote: > On 08/05/2020 10:41, Michael McMahon wrote: >> : >> >> If you have any comments about the third test, let me know when you >> get a chance >> > I found a bit of time to look at the 3rd test, ProtocolFamilies. It > provides good coverage but will need a round or two of cleanup. > > What is the purpose of running with > -Djava.net.preferIPv6Addresses=true? That influences the ordering of > lookups with InetAddress and I can't obviously see how it relates to > this test. > > ia4 = getFirstLinkLocalIPv4Address().? The naming is problematic as it > doesn't return a link-local address (it might do on a mis-configured > system but we usually try to avoid such addresses). Instead it finds a > local IPv4 address that is not a loopback address.? Maybe rename it > findLocalIPv4Address and put a comment that it may return null? > > ia6 = getFirstLinkLocalIPv6Address().? Do you really want an IPv6 link > local address or just a local address that is not the loopback? (I'm > asking as its different to the IPv4 version on a number of points, > including returning ::0 rather than null when an address is not found). > > Is dcOpenConnect needed in this test? This has the same problem as the > other test where it is trying to connect a UDP socket to the wildcard > address so doesn't make sense. > > It looks like the xxOpen tests are leaking file descriptors. Is this > the reason why the default @run is /othervm? I'd prefer if the sockets > were closed so that we at least have the option of running the test in > agent VM mode. A minor comment is that I think I'd prefer if the > parameters the xxOpen methods were "family" and something like > "expectedException". > > The exception expected in the xxOpenBind tests with the openBind data > provider is a bit confusing. It looks like the rows with "UOE" for the > IPv4-only case are dups of the xxOpen tests? If you drop these rows > then it would avoid special casing UOE and would allow you to drop > "open" from the names too. > > defaultImpl() seems to test a custom SelectorProvider (not the default > implementation) so maybe it should be renamed to testCustomProvider > and SELECTOR_PROVIDER should be renamed too. > > Trivially, "nulls" and "uoe" might be better as testNulls and testUOE. > Naming the unknown protocol family "STITCH" looks a bit strange, can > this be something more obvious such as "CUSTOM_PF", "UNKNOWN_PF" or > "BAD_PF"? > > -Alan. From Alan.Bateman at oracle.com Thu May 14 18:02:13 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 May 2020 19:02:13 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> Message-ID: <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> On 13/05/2020 22:51, Michael McMahon wrote: > I have updated the webrev for this at > http://cr.openjdk.java.net/~michaelm/8241305/webrev.4/ > based on the last round of comments. LocalSocketAddressType - looks good but not clear why it re-runs with preferIPv6Addresses=true. OpenAndConnect - looks much better, a few comments: The use of ignoreTest in each of the parameterized tests is a bit of anti-pattern. The data provider should filter the test cases so that the test aren't invoked with untestable parameters. That should avoid lots of skipped tests too. There are several places where exceptions are being caught and the error(Exception) method used to re-thrown them. I think error(Exception) can be removed and just let TestNG handle the failures. There are a few duplicate tests: L138 and L139, L144 and L145, L195 and L196. The tests that check for DONT_BIND would be a bit clear if they created "csa" after caddr != DONT_BIND, e.g. if (caddr != DONT_BIND) { ??? dc.bind(getSocketAddress(caddr); } ProtocolFamilies - looks much better, a few comments: isWindows can be removed. I think the data providers could be a bit clearer if they used if (hasIPv6 && !preferIPv4) { ... }. Do you intend to keep the commented out test cases in the openConnect data provider? getLocalIPv6Address looks like it could return a loopback address, filter step missing? BTW: if you change ia4 and ia6 to be InetAddress then it should eliminate the casts in all places. scOpenConnect, the connectOp isn't really needed, simpler to just call sc.connect if expectedException is null. dcOpen names the parameter "sfamily", typo? The other methods use "family". I think that is all I have for this round. -Alan. From michael.x.mcmahon at oracle.com Fri May 15 09:03:50 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 15 May 2020 10:03:50 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> Message-ID: On 14/05/2020 19:02, Alan Bateman wrote: > On 13/05/2020 22:51, Michael McMahon wrote: >> I have updated the webrev for this at >> http://cr.openjdk.java.net/~michaelm/8241305/webrev.4/ >> based on the last round of comments. > LocalSocketAddressType - looks good but not clear why it re-runs with > preferIPv6Addresses=true. > Ok, I overlooked that. Thanks > OpenAndConnect - looks much better, a few comments: > > The use of ignoreTest in each of the parameterized tests is a bit of > anti-pattern. The data provider should filter the test cases so that > the test aren't invoked with untestable parameters. That should avoid > lots of skipped tests too. > Initially I was thinking it wouldn't be any clearer because a predicate might need to be applied to each row of the table to determine if a local IPv4 address or local IPv6 address is present on this particular execution. But, it may be possible to split it into two separate tables for IPv4 and IPv6. If that works, I will do that. All remaining comments I agree with. I'll have another webrev later today Thanks again, Michael. > There are several places where exceptions are being caught and the > error(Exception) method used to re-thrown them. I think > error(Exception) can be removed and just let TestNG handle the failures. > > There are a few duplicate tests: L138 and L139, L144 and L145, L195 > and L196. > > The tests that check for DONT_BIND would be a bit clear if they > created "csa" after caddr != DONT_BIND, e.g. > > if (caddr != DONT_BIND) { > ??? dc.bind(getSocketAddress(caddr); > } > > > ProtocolFamilies - looks much better, a few comments: > > isWindows can be removed. > > I think the data providers could be a bit clearer if they used if > (hasIPv6 && !preferIPv4) { ... }. > Do you intend to keep the commented out test cases in the openConnect > data provider? > > getLocalIPv6Address looks like it could return a loopback address, > filter step missing? BTW: if you change ia4 and ia6 to be InetAddress > then it should eliminate the casts in all places. > > scOpenConnect, the connectOp isn't really needed, simpler to just call > sc.connect if expectedException is null. > > dcOpen names the parameter "sfamily", typo? The other methods use > "family". > > > I think that is all I have for this round. > > -Alan. > > From Alan.Bateman at oracle.com Fri May 15 10:45:35 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 May 2020 11:45:35 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> Message-ID: <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> On 15/05/2020 10:03, Michael McMahon wrote: > > Initially I was thinking it wouldn't be any clearer > because a predicate might need to be applied to each row of the table > to determine if a local IPv4 address or local IPv6 address is present > on this > particular execution. > > But, it may be possible to split it into two separate tables for IPv4 > and IPv6. > If that works, I will do that. There are several approaches but I think the main thing is that the data provider produces the parameters for scenarios that are testable. It can combine tables and/or filter cases that are untestable, doesn't have to be static array of cases. -Alan From michael.x.mcmahon at oracle.com Fri May 15 11:46:16 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 15 May 2020 12:46:16 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> Message-ID: On 15/05/2020 11:45, Alan Bateman wrote: > On 15/05/2020 10:03, Michael McMahon wrote: >> >> Initially I was thinking it wouldn't be any clearer >> because a predicate might need to be applied to each row of the table >> to determine if a local IPv4 address or local IPv6 address is present >> on this >> particular execution. >> >> But, it may be possible to split it into two separate tables for IPv4 >> and IPv6. >> If that works, I will do that. > There are several approaches but I think the main thing is that the > data provider produces the parameters for scenarios that are testable. > It can combine tables and/or filter cases that are untestable, doesn't > have to be static array of cases. > > -Alan Okay, webrev updated at: http://cr.openjdk.java.net/~michaelm/8241305/webrev.5/ Thanks, Michael. From brian.burkhalter at oracle.com Fri May 15 18:33:06 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 May 2020 11:33:06 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps Message-ID: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> Please review the proposed fix [1] of the issue [2]. This replaces the ?loopy? XBuffer.put(XBuffer src) fallback code while (src.hasRemaining()) dst.put(src.get()); with an implementation which behaves as if the source elements were first copied to a temporary array before being written into the target. The test fails before and passes after the implementation change is applied. I am aware of the flagged and commented out sections in the test. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.00/ [2] https://bugs.openjdk.java.net/browse/JDK-8245121 From paul.sandoz at oracle.com Fri May 15 18:58:55 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 May 2020 11:58:55 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> Message-ID: Hi Brian, That?s a pleasing result. - because of the fall back to the loop for small thresholds when the byte order differsI believe the issue can still occur in some cases, correct? In this case we would alas still need to use an explicit temporary buffer for overlap, that?s annoying, so much so it might just be better to remove the threshold (perhaps given copying between different orders is less common)? - can we cleanup the sub-type implementations? For direct buffers I think we can uses the fences and call to super. - for the test my preference would be to use a data provider to the combinatorial aspect, and you can avoid developing your own test logic for failure, and it?s easier to distinguish what combination of tests failed. Paul. > On May 15, 2020, at 11:33 AM, Brian Burkhalter wrote: > > Please review the proposed fix [1] of the issue [2]. This replaces the ?loopy? XBuffer.put(XBuffer src) fallback code > while (src.hasRemaining()) > dst.put(src.get()); > with an implementation which behaves as if the source elements were first copied to a temporary array before being written into the target. The test fails before and passes after the implementation change is applied. > > I am aware of the flagged and commented out sections in the test. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.00/ > [2] https://bugs.openjdk.java.net/browse/JDK-8245121 From brian.burkhalter at oracle.com Fri May 15 19:29:45 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 May 2020 12:29:45 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> Message-ID: <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> Hi Paul, > On May 15, 2020, at 11:58 AM, Paul Sandoz wrote: > > That?s a pleasing result. > > - because of the fall back to the loop for small thresholds when the byte order differsI believe the issue can still occur in some cases, correct? In this case we would alas still need to use an explicit temporary buffer for overlap, that?s annoying, so much so it might just be better to remove the threshold (perhaps given copying between different orders is less common)? The fallback occurs only if the length is below the threshold *and* base != srcBase. If base != srcBase, then either one buffer is heap and one is direct, or both are heap but refer to different arrays. Because of this second check I don?t think the issue can occur. > - can we cleanup the sub-type implementations? For direct buffers I think we can uses the fences and call to super. I?ll take a loop. It might be worth it to perform some benchmarking here. > - for the test my preference would be to use a data provider to the combinatorial aspect, and you can avoid developing your own test logic for failure, and it?s easier to distinguish what combination of tests failed. I did look over EqualsCompareTest. I came up with this approach as it seems to require less code although it is true that failures are not yet adequately described in the output. As a side note, this test has not yet found any failures when the backing memory of the two buffers is disjoint. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri May 15 19:42:52 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 May 2020 12:42:52 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> Message-ID: <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> > On May 15, 2020, at 12:29 PM, Brian Burkhalter wrote: > > Hi Paul, > >> On May 15, 2020, at 11:58 AM, Paul Sandoz > wrote: >> >> That?s a pleasing result. >> >> - because of the fall back to the loop for small thresholds when the byte order differsI believe the issue can still occur in some cases, correct? In this case we would alas still need to use an explicit temporary buffer for overlap, that?s annoying, so much so it might just be better to remove the threshold (perhaps given copying between different orders is less common)? > > The fallback occurs only if the length is below the threshold *and* base != srcBase. If base != srcBase, then either one buffer is heap and one is direct, or both are heap but refer to different arrays. Because of this second check I don?t think the issue can occur. > Doh, sorry I missed that. As you say it might be worth performing some benchmark, and be more informed whether the fallback to the loop is necessary. >> - can we cleanup the sub-type implementations? For direct buffers I think we can uses the fences and call to super. > > I?ll take a loop. It might be worth it to perform some benchmarking here. > >> - for the test my preference would be to use a data provider to the combinatorial aspect, and you can avoid developing your own test logic for failure, and it?s easier to distinguish what combination of tests failed. > > I did look over EqualsCompareTest. I came up with this approach as it seems to require less code although it is true that failures are not yet adequately described in the output. As a side note, this test has not yet found any failures when the backing memory of the two buffers is disjoint. > Ok. Given the overrides I guess this code is only being hit for certain cases, namely the byte buffer views. Paul. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri May 15 19:48:17 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 May 2020 20:48:17 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> Message-ID: <75513c26-6482-d7d2-af19-783160873f12@oracle.com> On 15/05/2020 12:46, Michael McMahon wrote: > : > > http://cr.openjdk.java.net/~michaelm/8241305/webrev.5/ Looks much better. LocalSocketAddressType.java Looks okay, just residuals of preferIPv6 in two places, I assume debugging from a previous iteration. ProtocolFamilies.java What is the reason for the "UOE" rows in the openBind data provider? If I read it correctly, it means the xxOpenBind tests duplicate the xxOpen tests for the UOE case. If the UOE rows can be dropped then it would simplify the xxOpenBind tests and avoid the outer try-catch (I assume the outer try-catch is in case UOE is not thrown but I think it can all go away). Minor nit but openBind uses "(hasIPv6 && !preferIPv4)" which is a bit clearer (in my view) that the inverse in the "open" data provider. OpenAndConnect.java The xxOpenAndConnect tests are much clearer in this iteration. Minor nit is that scOpenAndConnect looks like it print ssa twice, once as a SocketAddress, then its components. e they needed? I assume openConnect just need to tests that use IPv4 local addresses when IA4LOCAL != null. I think openConnectGen, openConnectV4Local, and openConnectV6Local would benefit from better names as they generate test cases and don't open anything. has_xxlocal also have unusual names, are they needed? I assume the openConnect data provider can use IA4LOCAL != null and IP6LOCAL != null. Another nit is that this test uses isNotLoopback, I thought the filters using !isLoopback(nif) in the other test was a bit clear. Also would be good to move it above or below isUp as they both test the network interface. -Alan From brian.burkhalter at oracle.com Fri May 15 20:46:48 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 15 May 2020 13:46:48 -0700 Subject: 8242477: (bf) MappedByteBuffer should clarify copying between overlapping mappings In-Reply-To: <8A25DCDC-73DE-4F74-BA7A-C342D1C14D54@oracle.com> References: <2F95A218-DF17-49A9-BD77-64CBC0D4F0A1@oracle.com> <874ktq9sgu.fsf@mid.deneb.enyo.de> <2b8f4185-7d48-c26f-bde1-168437a9791d@oracle.com> <36BCDCB6-027E-4DB0-87B0-E2A1CC7E7D62@oracle.com> <8A25DCDC-73DE-4F74-BA7A-C342D1C14D54@oracle.com> Message-ID: <1988C41E-C359-486F-8AC2-E682C2144FFA@oracle.com> > On Apr 24, 2020, at 8:14 AM, Brian Burkhalter wrote: > >> On Apr 24, 2020, at 2:24 AM, Alan Bateman > wrote: >> >>> The result of a bulk copy between >>> + * two direct buffers created via JNI is unspecified if they refer to >>> + * overlapping regions of memory. >>> * >>> *

Whether a byte buffer is direct or non-direct may be determined by >>> * invoking its {@link #isDirect isDirect} method. This method is provided so >>> >> I think this opens the door to questions about bulk copy operations on direct buffers by allocateDirect too because they can be sliced and duplicated in interesting ways. So easy to create scenarios where there is copying where the source and target overlaps. > > > I concur and have been able to test this. I think it might be possible to fix that case though. This inclines me to suggest perhaps constraining this issue to the MappedByteBuffer doc change only as first proposed and addressing the other overlap situations separately. If the fix for [1] proposed in [2] is accepted, then I think the prospective specification changes under discussion in this thread are no longer relevant or for that matter correct. The change for JDK-8245121 would standardize the expected result of a bulk copy performed with buffers having overlapping backing storage. The behavior would be the same irrespective of the type of buffer. Instead probably the specification of the bulk put buffer methods XBuffer::put(XBuffer src) should change to describe and make normative the behavior expected when the backing storage intersects. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8245121 [2] http://mail.openjdk.java.net/pipermail/nio-dev/2020-May/007265.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Sat May 16 08:06:57 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Sat, 16 May 2020 09:06:57 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <75513c26-6482-d7d2-af19-783160873f12@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> Message-ID: Thanks for the comments again. Just a couple of questions below. On 15/05/2020 20:48, Alan Bateman wrote: > > > On 15/05/2020 12:46, Michael McMahon wrote: >> : >> >> http://cr.openjdk.java.net/~michaelm/8241305/webrev.5/ > Looks much better. > > LocalSocketAddressType.java > > Looks okay, just residuals of preferIPv6 in two places, I assume > debugging from a previous iteration. > > > ProtocolFamilies.java > > What is the reason for the "UOE" rows in the openBind data provider? > If I read it correctly, it means the xxOpenBind tests duplicate the > xxOpen tests for the UOE case. If the UOE rows can be dropped then it > would simplify the xxOpenBind tests and avoid the outer try-catch (I > assume the outer try-catch is in case UOE is not thrown but I think it > can all go away). > I don't follow this. The reason for the UOE rows are to test for UOE being thrown when IPv6 is not available. Are you saying not to test this scenario? > Minor nit but openBind uses "(hasIPv6 && !preferIPv4)" which is a bit > clearer (in my view) that the inverse in the "open" data provider. > > > OpenAndConnect.java > > The xxOpenAndConnect tests are much clearer in this iteration. Minor > nit is that scOpenAndConnect looks like it print ssa twice, once as a > SocketAddress, then its components. > > e they needed? I assume openConnect just need to tests that use IPv4 > local addresses when IA4LOCAL != null. > I'm assuming the truncated word refers to previous sentence. > I think openConnectGen, openConnectV4Local, and openConnectV6Local > would benefit from better names as they generate test cases and don't > open anything. has_xxlocal also have unusual names, are they needed? I > assume the openConnect data provider can use IA4LOCAL != null and > IP6LOCAL != null. > That's fine. How about adding the suffix "Tests" to the names, eg openConnectV4LocalTests? > Another nit is that this test uses isNotLoopback, I thought the > filters using !isLoopback(nif) in the other test was a bit clear. Also > would be good to move it above or below isUp as they both test the > network interface. > I'll change it to use the same initialization code as the other test. It doesn't need to handle NetworkInterface directly and the isUp() method wasn't used anyway. I won't update the webrev until I hear back from you. Thanks, Michael. From Alan.Bateman at oracle.com Sat May 16 14:43:49 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 16 May 2020 15:43:49 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> Message-ID: <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> On 16/05/2020 09:06, Michael McMahon wrote: > > I don't follow this. The reason for the UOE rows are to test for UOE > being thrown when IPv6 is not > available. Are you saying not to test this scenario? The xxOpen tests will test that UOE is thrown by the open methods. I'm just pointing out that the test parameters generated for the xxOpenBind tests will test the same thing. If you delete L170-172 from the openBind data provider then it will allow the xxOpenBind tests to be simplified so that they test the bind method. I don't think there is any loss of test coverage because the second parameter is completely ignored by the tests that have expectedException=UOE. > >> : >> >> e they needed? I assume openConnect just need to tests that use IPv4 >> local addresses when IA4LOCAL != null. >> > I'm assuming the truncated word refers to previous sentence. Sorry about that, the comment was further down and related to the naming issues. If IA4LOCAL and IA6LOCAL are tested for null then it will allow has_ia4local and has_ia6local to go away. > : > > How about adding the suffix "Tests" to the names, eg > openConnectV4LocalTests? Might be clearer with "gen" or "generate" as they generate test cases, up to you. -Alan. From michael.x.mcmahon at oracle.com Sun May 17 09:10:24 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Sun, 17 May 2020 10:10:24 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> Message-ID: On 16/05/2020 15:43, Alan Bateman wrote: > On 16/05/2020 09:06, Michael McMahon wrote: >> >> I don't follow this. The reason for the UOE rows are to test for UOE >> being thrown when IPv6 is not >> available. Are you saying not to test this scenario? > The xxOpen tests will test that UOE is thrown by the open methods. > > I'm just pointing out that the test parameters generated for the > xxOpenBind tests will test the same thing. If you delete L170-172 from > the openBind data provider then it will allow the xxOpenBind tests to > be simplified so that they test the bind method. I don't think there > is any loss of test coverage because the second parameter is > completely ignored by the tests that have expectedException=UOE. > Right, got you now. >> >>> : >>> >>> e they needed? I assume openConnect just need to tests that use IPv4 >>> local addresses when IA4LOCAL != null. >>> >> I'm assuming the truncated word refers to previous sentence. > Sorry about that, the comment was further down and related to the > naming issues. If IA4LOCAL and IA6LOCAL are tested for null then it > will allow has_ia4local and has_ia6local to go away. > > OK >> : >> >> How about adding the suffix "Tests" to the names, eg >> openConnectV4LocalTests? > Might be clearer with "gen" or "generate" as they generate test cases, > up to you. > gen is already part of the name for one of the lists. So, I left it as Tests. If you have a strong preference for something different, I'll change it. New webrev is at: http://cr.openjdk.java.net/~michaelm/8241305/webrev.6/ Thanks, Michael. From Alan.Bateman at oracle.com Sun May 17 09:39:52 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 17 May 2020 10:39:52 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> Message-ID: <34be8798-5bb6-d526-5c59-7e1eb2a6cdfa@oracle.com> On 17/05/2020 10:10, Michael McMahon wrote: > > Right, got you now. Good but what about the the outer try-finally in xxOpenBind tests? If that is removed (sc, ssc, dc are not used) then it should make these tests much easier to maintain. Also will allow you to drop the "1" suffix from the variable names. > gen is already part of the name for one of the lists. So, I left it as > Tests. If you > have a strong preference for something different, I'll change it. Okay, I guess it's okay. -Alan. From michael.x.mcmahon at oracle.com Sun May 17 09:56:12 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Sun, 17 May 2020 10:56:12 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <34be8798-5bb6-d526-5c59-7e1eb2a6cdfa@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> <34be8798-5bb6-d526-5c59-7e1eb2a6cdfa@oracle.com> Message-ID: <786024f4-d107-e7f8-f8e9-b0bd48aa7b87@oracle.com> On 17/05/2020 10:39, Alan Bateman wrote: > On 17/05/2020 10:10, Michael McMahon wrote: >> >> Right, got you now. > Good but what about the the outer try-finally in xxOpenBind tests? If > that is removed (sc, ssc, dc are not used) then it should make these > tests much easier to maintain. Also will allow you to drop the "1" > suffix from the variable names. > Yes, sorry I should have noticed that. The webrev is updated in place with that fixed. Thanks, Michael. > >> gen is already part of the name for one of the lists. So, I left it >> as Tests. If you >> have a strong preference for something different, I'll change it. > Okay, I guess it's okay. > > -Alan. From Alan.Bateman at oracle.com Sun May 17 14:06:14 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 17 May 2020 15:06:14 +0100 Subject: RFR 8241305: Add protocol specific factory creation methods to SocketChannel and ServerSocketChannel In-Reply-To: <786024f4-d107-e7f8-f8e9-b0bd48aa7b87@oracle.com> References: <9fbced49-82f7-c85d-077c-7d23429cd52a@oracle.com> <3adc4ce2-186d-e859-f2dd-268b5c632161@oracle.com> <46a77e74-3c34-7878-028c-9133578402f7@oracle.com> <75513c26-6482-d7d2-af19-783160873f12@oracle.com> <86ffa1a1-cd0e-9971-7c09-141ea41d9866@oracle.com> <34be8798-5bb6-d526-5c59-7e1eb2a6cdfa@oracle.com> <786024f4-d107-e7f8-f8e9-b0bd48aa7b87@oracle.com> Message-ID: On 17/05/2020 10:56, Michael McMahon wrote: > > On 17/05/2020 10:39, Alan Bateman wrote: >> On 17/05/2020 10:10, Michael McMahon wrote: >>> >>> Right, got you now. >> Good but what about the the outer try-finally in xxOpenBind tests? If >> that is removed (sc, ssc, dc are not used) then it should make these >> tests much easier to maintain. Also will allow you to drop the "1" >> suffix from the variable names. >> > Yes, sorry I should have noticed that. The webrev is updated in place > with that fixed. Thanks, I think all 3 tests look much better now. Just a couple of nits that you might want to fix up before pushing this (I don't need another webrev): - The @summary on OpenAndConnect mentions "platforms" but these were all the problematic test scenarios that no longer exist so I think the summary statement can be cleaned up. - OpenAndConnect::isNotLoopback is no longer used so can be removed - Tidy up the imports as there are bunch of imports that aren't used (Predicate, Duration, parseBoolean, getProperty, ...). -Alan From Alan.Bateman at oracle.com Mon May 18 07:57:07 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 May 2020 08:57:07 +0100 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> Message-ID: <74e9c575-35fd-b641-f406-2504be9f6839@oracle.com> On 15/05/2020 19:33, Brian Burkhalter wrote: > Please review the proposed fix [1] of the issue [2]. This replaces the ?loopy? XBuffer.put(XBuffer src) fallback code > while (src.hasRemaining()) > dst.put(src.get()); > with an implementation which behaves as if the source elements were first copied to a temporary array before being written into the target. The test fails before and passes after the implementation change is applied. > The approach looks reasonable but I suspect this will need a reachability fence for at least src (maybe this too, I need to page some of the details). -Alan From brian.burkhalter at oracle.com Tue May 19 21:06:26 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 19 May 2020 14:06:26 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> Message-ID: <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> Here [1] is an updated version of the proposed patch. This version adds fences to the X-Buffer version of put($TypeBuffer) and modifies the direct and heap implementations to call super.put(). Specification verbiage is also added: * If this buffer and * the source buffer share the same backing memory, then the result will * be as if the source elements were first copied to an intermediate * location before being written into this buffer. If this work goes forward and this doc change is deemed necessary / worthwhile then a CSR will be filed. Some benchmarks were run to measure the performance of putting the contents of direct, heap, direct view, swapped direct view, heap view, and swapped heap view IntBuffers into direct and heap IntBuffers. The results are at [2]. On the average the proposed version appears to perform better, sometimes substantially, than the current version. A notable exception is for writing views of a byte buffer into a heap IntBuffer for a very small capacity. One would expect that this is due to falling through to the loopy implementation for small copy lengths, but testing did not bear this out. Some more investigation might be needed here. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.01 [2] http://cr.openjdk.java.net/~bpb/8245121/XBufferBench.csv > On May 15, 2020, at 12:42 PM, Paul Sandoz wrote: > > As you say it might be worth performing some benchmark, and be more informed whether the fallback to the loop is necessary. > On May 18, 2020, at 12:57 AM, Alan Bateman wrote: > > The approach looks reasonable but I suspect this will need a reachability fence for at least src (maybe this too, I need to page some of the details). -------------- next part -------------- An HTML attachment was scrubbed... URL: From claes.redestad at oracle.com Fri May 22 09:33:00 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 May 2020 11:33:00 +0200 Subject: RFR: 8245619: Remove unused methods in UnixNativeDispatcher Message-ID: <50837179-2579-acd0-7fa8-30e76fcecf6d@oracle.com> Hi, this patch removes four orphaned methods from UnixNativeDispatcher that are no longer used after the Solaris removal: Webrev: http://cr.openjdk.java.net/~redestad/8245619/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8245619 Testing: tier1 Thanks! /Claes From Alan.Bateman at oracle.com Fri May 22 09:36:08 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 May 2020 10:36:08 +0100 Subject: RFR: 8245619: Remove unused methods in UnixNativeDispatcher In-Reply-To: <50837179-2579-acd0-7fa8-30e76fcecf6d@oracle.com> References: <50837179-2579-acd0-7fa8-30e76fcecf6d@oracle.com> Message-ID: <085211a6-a86b-e71a-0d9e-642666a91ce0@oracle.com> On 22/05/2020 10:33, Claes Redestad wrote: > Hi, > > this patch removes four orphaned methods from UnixNativeDispatcher that > are no longer used after the Solaris removal: > > Webrev: http://cr.openjdk.java.net/~redestad/8245619/open.00/ > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245619 Looks good, I suspect there will many follow on changes to cleanup after the removal of the Solaris port. -Alan From claes.redestad at oracle.com Fri May 22 09:39:47 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 May 2020 11:39:47 +0200 Subject: RFR: 8245619: Remove unused methods in UnixNativeDispatcher In-Reply-To: <085211a6-a86b-e71a-0d9e-642666a91ce0@oracle.com> References: <50837179-2579-acd0-7fa8-30e76fcecf6d@oracle.com> <085211a6-a86b-e71a-0d9e-642666a91ce0@oracle.com> Message-ID: <09851f91-3dbd-8eda-76d4-851372170e7e@oracle.com> On 2020-05-22 11:36, Alan Bateman wrote: > On 22/05/2020 10:33, Claes Redestad wrote: >> Hi, >> >> this patch removes four orphaned methods from UnixNativeDispatcher that >> are no longer used after the Solaris removal: >> >> Webrev: http://cr.openjdk.java.net/~redestad/8245619/open.00/ >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245619 > Looks good, I suspect there will many follow on changes to cleanup after > the removal of the Solaris port. > > -Alan Thanks! /Claes From claes.redestad at oracle.com Fri May 22 10:41:46 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 May 2020 12:41:46 +0200 Subject: RFR: 8245623: Remove unused code in sun/nio/fs after Solaris removal Message-ID: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> Hi, a small cleanup of code left unused in nio/fs after the Solaris removal Bug: https://bugs.openjdk.java.net/browse/JDK-8245623 Webrev: http://cr.openjdk.java.net/~redestad/8245623/open.00/ Testing: tier1 Thanks! /Claes From Alan.Bateman at oracle.com Fri May 22 10:45:45 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 May 2020 11:45:45 +0100 Subject: RFR: 8245623: Remove unused code in sun/nio/fs after Solaris removal In-Reply-To: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> References: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> Message-ID: <15f1a2b6-66da-2b55-e971-2750e4da4456@oracle.com> On 22/05/2020 11:41, Claes Redestad wrote: > Hi, > > a small cleanup of code left unused in nio/fs after the Solaris removal > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245623 > Webrev: http://cr.openjdk.java.net/~redestad/8245623/open.00/ Okay although the static factory in UnixChannelFactory might come back for other usages in the future. -Alan From claes.redestad at oracle.com Fri May 22 10:48:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 May 2020 12:48:36 +0200 Subject: RFR: 8245623: Remove unused code in sun/nio/fs after Solaris removal In-Reply-To: <15f1a2b6-66da-2b55-e971-2750e4da4456@oracle.com> References: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> <15f1a2b6-66da-2b55-e971-2750e4da4456@oracle.com> Message-ID: <1d0148ea-b66d-7889-0e6f-cf5a4aae6076@oracle.com> On 2020-05-22 12:45, Alan Bateman wrote: > > > On 22/05/2020 11:41, Claes Redestad wrote: >> Hi, >> >> a small cleanup of code left unused in nio/fs after the Solaris removal >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245623 >> Webrev: http://cr.openjdk.java.net/~redestad/8245623/open.00/ > Okay although the static factory in UnixChannelFactory might come back > for other usages in the future. Ok. Should I drop the removal or keep it? /Claes From Alan.Bateman at oracle.com Fri May 22 10:52:13 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 May 2020 11:52:13 +0100 Subject: RFR: 8245623: Remove unused code in sun/nio/fs after Solaris removal In-Reply-To: <1d0148ea-b66d-7889-0e6f-cf5a4aae6076@oracle.com> References: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> <15f1a2b6-66da-2b55-e971-2750e4da4456@oracle.com> <1d0148ea-b66d-7889-0e6f-cf5a4aae6076@oracle.com> Message-ID: <7513a024-a1f5-e84a-f4a8-15faf5aae821@oracle.com> On 22/05/2020 11:48, Claes Redestad wrote: > On 2020-05-22 12:45, Alan Bateman wrote: >> >> >> On 22/05/2020 11:41, Claes Redestad wrote: >>> Hi, >>> >>> a small cleanup of code left unused in nio/fs after the Solaris removal >>> >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245623 >>> Webrev: http://cr.openjdk.java.net/~redestad/8245623/open.00/ >> Okay although the static factory in UnixChannelFactory might come >> back for other usages in the future. > > Ok. Should I drop the removal or keep it? It's okay to remove it, I'm just musing that it may come back at some point. -Alan From claes.redestad at oracle.com Fri May 22 10:53:44 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 May 2020 12:53:44 +0200 Subject: RFR: 8245623: Remove unused code in sun/nio/fs after Solaris removal In-Reply-To: <7513a024-a1f5-e84a-f4a8-15faf5aae821@oracle.com> References: <774220c7-6f7c-aff7-6677-0db5debce8a3@oracle.com> <15f1a2b6-66da-2b55-e971-2750e4da4456@oracle.com> <1d0148ea-b66d-7889-0e6f-cf5a4aae6076@oracle.com> <7513a024-a1f5-e84a-f4a8-15faf5aae821@oracle.com> Message-ID: On 2020-05-22 12:52, Alan Bateman wrote: > On 22/05/2020 11:48, Claes Redestad wrote: >> On 2020-05-22 12:45, Alan Bateman wrote: >>> >>> >>> On 22/05/2020 11:41, Claes Redestad wrote: >>>> Hi, >>>> >>>> a small cleanup of code left unused in nio/fs after the Solaris removal >>>> >>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8245623 >>>> Webrev: http://cr.openjdk.java.net/~redestad/8245623/open.00/ >>> Okay although the static factory in UnixChannelFactory might come >>> back for other usages in the future. >> >> Ok. Should I drop the removal or keep it? > It's okay to remove it, I'm just musing that it may come back at some > point. Ok, thanks! /Claes From Alan.Bateman at oracle.com Sat May 23 08:18:22 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 23 May 2020 09:18:22 +0100 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> Message-ID: On 19/05/2020 22:06, Brian Burkhalter wrote: > Here [1] is an updated version of the proposed patch. This version > adds fences to the X-Buffer version of put($TypeBuffer) and modifies > the direct and heap implementations to call super.put(). > > Specification verbiage is also added: > * If this buffer and * the source buffer share the same backing > memory, then the result will * be as if the source elements were first > copied to an intermediate * location before being written into this > buffer. > If this work goes forward and this doc change is deemed necessary / > worthwhile then a CSR will be filed. > The updated implementation with the fences looks good. For the javadoc then I think it might be more consistent with the existing API docs if you replaced "backing memory" with "backing array or memory". It does add new testable assertions so a CSR is needed. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Tue May 26 18:35:59 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 May 2020 11:35:59 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> Message-ID: <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> This is looking good, Strictly speaking the fences are only required for the direct buffer, but I think I prefer them being placed where the unsafe usages are for tighter scope. I would be inclined to drop the Bits.JNI_COPY_TO_ARRAY_THRESHOLD, since it never applies to direct buffers. I bet the swapping. Maybe can be determined with follow evaluation after this issue. Test generally looks good: 47 static final Random RND = new MyRandom(SEED); // XXX RandomFactory 48 49 static class MyRandom extends Random { 50 MyRandom(long seed) { 51 super(seed); 52 } The additional methods are never used. No need to extend? (If there was a need the methods could use next accepting bits values). I still hold out hope you can convert it to a testng test, with data providers and asserts, this is all good common practice for writing tests. The nested loops in test() effectively capture the test combinations input for two test cases. Let me know if you need some advice/help on this. Paul. > On May 19, 2020, at 2:06 PM, Brian Burkhalter wrote: > > Here [1] is an updated version of the proposed patch. This version adds fences to the X-Buffer version of put($TypeBuffer) and modifies the direct and heap implementations to call super.put(). > > Specification verbiage is also added: > * If this buffer and > * the source buffer share the same backing memory, then the result will > * be as if the source elements were first copied to an intermediate > * location before being written into this buffer. > If this work goes forward and this doc change is deemed necessary / worthwhile then a CSR will be filed. > > Some benchmarks were run to measure the performance of putting the contents of direct, heap, direct view, swapped direct view, heap view, and swapped heap view IntBuffers into direct and heap IntBuffers. The results are at [2]. On the average the proposed version appears to perform better, sometimes substantially, than the current version. A notable exception is for writing views of a byte buffer into a heap IntBuffer for a very small capacity. One would expect that this is due to falling through to the loopy implementation for small copy lengths, but testing did not bear this out. Some more investigation might be needed here. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.01 > [2] http://cr.openjdk.java.net/~bpb/8245121/XBufferBench.csv >> On May 15, 2020, at 12:42 PM, Paul Sandoz > wrote: >> >> As you say it might be worth performing some benchmark, and be more informed whether the fallback to the loop is necessary. > > > >> On May 18, 2020, at 12:57 AM, Alan Bateman > wrote: >> >> The approach looks reasonable but I suspect this will need a reachability fence for at least src (maybe this too, I need to page some of the details). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue May 26 21:05:14 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 26 May 2020 14:05:14 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> Message-ID: <45E379D1-6EAE-400D-8D9E-C0FBD5584335@oracle.com> > On May 23, 2020, at 1:18 AM, Alan Bateman wrote: > > For the javadoc then I think it might be more consistent with the existing API docs if you replaced "backing memory" with "backing array or memory". It does add new testable assertions so a CSR is needed. I filed a CSR for this [1]. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8245866 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Tue May 26 21:20:42 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 26 May 2020 14:20:42 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> Message-ID: <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> > On May 26, 2020, at 11:35 AM, Paul Sandoz wrote: > > This is looking good, > > Strictly speaking the fences are only required for the direct buffer, but I think I prefer them being placed where the unsafe usages are for tighter scope. Actually since the specific direct and heap implementations are modified simply to invoke super.put($TypeBuffer) if the target buffer is read-write, then within the X-Buffer method we don?t really know whether the destination or source is direct without an explicit check. I suppose one could make use of the fences contingent on the value of isDirect(). > I would be inclined to drop the Bits.JNI_COPY_TO_ARRAY_THRESHOLD, since it never applies to direct buffers. I bet the swapping. Maybe can be determined with follow evaluation after this issue. You mean dropping the else branch and converting the else if branch to an else? > Test generally looks good: > > 47 static final Random RND = new MyRandom(SEED); // XXX RandomFactory > 48 > 49 static class MyRandom extends Random { > 50 MyRandom(long seed) { > 51 super(seed); > 52 } > The additional methods are never used. No need to extend? (If there was a need the methods could use next accepting bits values). It was done this way to make it simple to grab the appropriate next method based on the type: 163 nextType = lookup.findVirtual(MyRandom.class, 164 "next" + typeName, MethodType.methodType(elementType)); Also, the method Random.next(int) is protected. > I still hold out hope you can convert it to a testng test, with data providers and asserts, this is all good common practice for writing tests. The nested loops in test() effectively capture the test combinations input for two test cases. Let me know if you need some advice/help on this. I?ll take another look. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Tue May 26 21:27:25 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 May 2020 14:27:25 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> Message-ID: <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> > On May 26, 2020, at 2:20 PM, Brian Burkhalter wrote: > > >> On May 26, 2020, at 11:35 AM, Paul Sandoz > wrote: >> >> This is looking good, >> >> Strictly speaking the fences are only required for the direct buffer, but I think I prefer them being placed where the unsafe usages are for tighter scope. > > Actually since the specific direct and heap implementations are modified simply to invoke super.put($TypeBuffer) if the target buffer is read-write, then within the X-Buffer method we don?t really know whether the destination or source is direct without an explicit check. I suppose one could make use of the fences contingent on the value of isDirect(). > One approach is to wrap the super call for fences when direct, but I think there is no need. >> I would be inclined to drop the Bits.JNI_COPY_TO_ARRAY_THRESHOLD, since it never applies to direct buffers. I bet the swapping. Maybe can be determined with follow evaluation after this issue. > > You mean dropping the else branch and converting the else if branch to an else? > Yes. >> Test generally looks good: >> >> 47 static final Random RND = new MyRandom(SEED); // XXX RandomFactory >> 48 >> 49 static class MyRandom extends Random { >> 50 MyRandom(long seed) { >> 51 super(seed); >> 52 } >> The additional methods are never used. No need to extend? (If there was a need the methods could use next accepting bits values). > > It was done this way to make it simple to grab the appropriate next method based on the type: > 163 nextType = lookup.findVirtual(MyRandom.class, > 164 "next" + typeName, MethodType.methodType(elementType)); > Also, the method Random.next(int) is protected. > Ah, sorry, I missed that, you can change the implementation to use next()? rather than rolling your own logic. Paul. >> I still hold out hope you can convert it to a testng test, with data providers and asserts, this is all good common practice for writing tests. The nested loops in test() effectively capture the test combinations input for two test cases. Let me know if you need some advice/help on this. > > I?ll take another look. > > Thanks, > > Brian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Wed May 27 12:36:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 May 2020 13:36:31 +0100 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <45E379D1-6EAE-400D-8D9E-C0FBD5584335@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <45E379D1-6EAE-400D-8D9E-C0FBD5584335@oracle.com> Message-ID: On 26/05/2020 22:05, Brian Burkhalter wrote: > : > > I filed a CSR for this [1]. > I tweaked the summary to make it a bit clearer what this issue is about and added myself as Reviewer. -Alan From brian.burkhalter at oracle.com Wed May 27 15:03:07 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 27 May 2020 08:03:07 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <45E379D1-6EAE-400D-8D9E-C0FBD5584335@oracle.com> Message-ID: <465C579B-7D21-4727-A462-D81C69EEEE29@oracle.com> > On May 27, 2020, at 5:36 AM, Alan Bateman wrote: > > On 26/05/2020 22:05, Brian Burkhalter wrote: >> : >> >> I filed a CSR for this [1]. >> > I tweaked the summary to make it a bit clearer what this issue is about and added myself as Reviewer. Thanks. I?ll mark it final at the end of the day. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu May 28 19:45:27 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 28 May 2020 12:45:27 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> Message-ID: <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> New version uploaded here [1]. This includes the doc change outlined in the CSR [2] and the changes mentioned below. > On May 26, 2020, at 2:27 PM, Paul Sandoz wrote: > >>> I would be inclined to drop the Bits.JNI_COPY_TO_ARRAY_THRESHOLD, since it never applies to direct buffers. I bet the swapping. Maybe can be determined with follow evaluation after this issue. >> >> You mean dropping the else branch and converting the else if branch to an else? >> > > Yes. Done in this update. > > [?] > >>> I still hold out hope you can convert it to a testng test, with data providers and asserts, this is all good common practice for writing tests. The nested loops in test() effectively capture the test combinations input for two test cases. Let me know if you need some advice/help on this. >> >> I?ll take another look. I took a stab at changing the test to use DataProviders and Asserts in this update. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.02/ [2] https://bugs.openjdk.java.net/browse/JDK-8245866 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Thu May 28 20:40:39 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 May 2020 13:40:39 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> Message-ID: <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> > On May 28, 2020, at 12:45 PM, Brian Burkhalter wrote: > > New version uploaded here [1]. This includes the doc change outlined in the CSR [2] and the changes mentioned below. > >> On May 26, 2020, at 2:27 PM, Paul Sandoz > wrote: >> >>>> I would be inclined to drop the Bits.JNI_COPY_TO_ARRAY_THRESHOLD, since it never applies to direct buffers. I bet the swapping. Maybe can be determined with follow evaluation after this issue. >>> >>> You mean dropping the else branch and converting the else if branch to an else? >>> >> >> Yes. > > Done in this update. > Ok. >> >> [?] >> >>>> I still hold out hope you can convert it to a testng test, with data providers and asserts, this is all good common practice for writing tests. The nested loops in test() effectively capture the test combinations input for two test cases. Let me know if you need some advice/help on this. >>> >>> I?ll take another look. > > I took a stab at changing the test to use DataProviders and Asserts in this update. > That?s a good start, but I think the use of iterators in this case makes it more complicated than need be. Iterators are a pain to directly compose, its hard to see the cross product. A data provider can return Object[][], basically an array of test case arguments, if you are not concerned about the pre-allocation of the number of cases (which I think is so here). So you can use your favorite nested for loops, or nested streams and flatMap if you wish to do something lazily, to construct the test argument set. For the testPairs method you can separate out the arguments, no need for an array. Paul. > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8245121/webrev.02/ > [2] https://bugs.openjdk.java.net/browse/JDK-8245866 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Thu May 28 23:56:13 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 28 May 2020 16:56:13 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> Message-ID: <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> > On May 28, 2020, at 1:40 PM, Paul Sandoz wrote: > >> I took a stab at changing the test to use DataProviders and Asserts in this update. >> > > That?s a good start, but I think the use of iterators in this case makes it more complicated than need be. Iterators are a pain to directly compose, its hard to see the cross product. > > A data provider can return Object[][], basically an array of test case arguments, if you are not concerned about the pre-allocation of the number of cases (which I think is so here). So you can use your favorite nested for loops, or nested streams and flatMap if you wish to do something lazily, to construct the test argument set. An updated version is at http://cr.openjdk.java.net/~bpb/8245121/webrev.03/. This gets rid of the Iterators in favor of nested for loops. It?s about 90 lines shorter than the previous version. I might try nested streams and flatmap later for educational purposes. > For the testPairs method you can separate out the arguments, no need for an array. Fixed. Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri May 29 00:21:59 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 May 2020 17:21:59 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> Message-ID: Much better! Here?s a tip, rather than creating a fixed size array upfront: - create a List to add argument list combinations to - add with args.add(new Object[] {proxy}) - at the end do args.toArray(Object[][]::new) For: 281 List proxies1 = getProxies(type); 282 List proxies2 = getProxies(type); 283 for (BufferProxy proxy1 : proxies1) { 284 for (BufferProxy proxy2 : proxies2) { 285 data[index][0] = proxy1; 286 data[index++][1] = proxy2; 287 } 288 } Can you reuse proxy1 since the list is not modified and the proxy is stateless w.r.t. Buffers? Is it necessary to test when proxy1 == proxy2 i.e. the diagonal? It?s useful to try the streams approach as an exercise, I can imagine some people might find nested flatMaps a little jarring. The nested loops are fine here. Paul. > On May 28, 2020, at 4:56 PM, Brian Burkhalter wrote: > > >> On May 28, 2020, at 1:40 PM, Paul Sandoz > wrote: >> >>> I took a stab at changing the test to use DataProviders and Asserts in this update. >>> >> >> That?s a good start, but I think the use of iterators in this case makes it more complicated than need be. Iterators are a pain to directly compose, its hard to see the cross product. >> >> A data provider can return Object[][], basically an array of test case arguments, if you are not concerned about the pre-allocation of the number of cases (which I think is so here). So you can use your favorite nested for loops, or nested streams and flatMap if you wish to do something lazily, to construct the test argument set. > > An updated version is at > > http://cr.openjdk.java.net/~bpb /8245121/webrev.03/. > > This gets rid of the Iterators in favor of nested for loops. It?s about 90 lines shorter than the previous version. I might try nested streams and flatmap later for educational purposes. > >> For the testPairs method you can separate out the arguments, no need for an array. > > Fixed. > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri May 29 00:57:47 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 28 May 2020 17:57:47 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> Message-ID: <9EC7A15D-AD97-40DA-90ED-6B905F5CD03E@oracle.com> > On May 28, 2020, at 5:21 PM, Paul Sandoz wrote: > > Here?s a tip, rather than creating a fixed size array upfront: > > - create a List to add argument list combinations to > - add with args.add(new Object[] {proxy}) > - at the end do args.toArray(Object[][]::new) Cool - thanks! > For: > > 281 List proxies1 = getProxies(type); > 282 List proxies2 = getProxies(type); > 283 for (BufferProxy proxy1 : proxies1) { > 284 for (BufferProxy proxy2 : proxies2) { > 285 data[index][0] = proxy1; > 286 data[index++][1] = proxy2; > 287 } > 288 } > > Can you reuse proxy1 since the list is not modified and the proxy is stateless w.r.t. Buffers? Is it necessary to test when proxy1 == proxy2 i.e. the diagonal? Good point. Yes I think it can be re-used as there is no relevant state. I don?t think it?s necessary to check for proxy1 == proxy2: we want to test the case where both are the same type, kind, and order. > It?s useful to try the streams approach as an exercise, I can imagine some people might find nested flatMaps a little jarring. The nested loops are fine here. The updated version is at http://cr.openjdk.java.net/~bpb/8245121/webrev.04/ It keeps getting shorter! Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri May 29 16:25:12 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 May 2020 09:25:12 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <9EC7A15D-AD97-40DA-90ED-6B905F5CD03E@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> <9EC7A15D-AD97-40DA-90ED-6B905F5CD03E@oracle.com> Message-ID: <8708C721-D731-49EB-84BB-0737B2D7FD9D@oracle.com> There we go :-) I found it took me a couple rounds of writing tests to settle on a few useful patterns. Generally it would be useful if we had a test library for creating combinations, permutations, zipping etc. (CC?ing Pavel) I have one other request and one comment, both of which you can choose to ignore if you wanna wrap this up. - change the implementations of MyRandom.next$Type to use next() rather than your use your own shifting logic. - typeToAttr can be constructed using Map.of: Map.of( byte.class, TypeAttr.of(ByteBuffer.class, Byte.BYTES, ?Byte?), char.class, ? ?, ) In fact TypeAttr really really wants to be a record (nominal tuple). I believe we can use preview features in tests, and it is worthwhile to do so (gaining more coverage). Paul. > On May 28, 2020, at 5:57 PM, Brian Burkhalter wrote: > > >> On May 28, 2020, at 5:21 PM, Paul Sandoz > wrote: >> >> Here?s a tip, rather than creating a fixed size array upfront: >> >> - create a List to add argument list combinations to >> - add with args.add(new Object[] {proxy}) >> - at the end do args.toArray(Object[][]::new) > > Cool - thanks! > >> For: >> >> 281 List proxies1 = getProxies(type); >> 282 List proxies2 = getProxies(type); >> 283 for (BufferProxy proxy1 : proxies1) { >> 284 for (BufferProxy proxy2 : proxies2) { >> 285 data[index][0] = proxy1; >> 286 data[index++][1] = proxy2; >> 287 } >> 288 } >> >> Can you reuse proxy1 since the list is not modified and the proxy is stateless w.r.t. Buffers? Is it necessary to test when proxy1 == proxy2 i.e. the diagonal? > > Good point. Yes I think it can be re-used as there is no relevant state. I don?t think it?s necessary to check for proxy1 == proxy2: we want to test the case where both are the same type, kind, and order. > >> It?s useful to try the streams approach as an exercise, I can imagine some people might find nested flatMaps a little jarring. The nested loops are fine here. > > The updated version is at > > http://cr.openjdk.java.net/~bpb/ 8245121/webrev.04/ > > It keeps getting shorter! > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Fri May 29 19:11:18 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 29 May 2020 12:11:18 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: <8708C721-D731-49EB-84BB-0737B2D7FD9D@oracle.com> References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> <9EC7A15D-AD97-40DA-90ED-6B905F5CD03E@oracle.com> <8708C721-D731-49EB-84BB-0737B2D7FD9D@oracle.com> Message-ID: > On May 29, 2020, at 9:25 AM, Paul Sandoz wrote: > > I have one other request and one comment, both of which you can choose to ignore if you wanna wrap this up. > > - change the implementations of MyRandom.next$Type to use next() rather than your use your own shifting logic. > > - typeToAttr can be constructed using Map.of: > Map.of( > byte.class, TypeAttr.of(ByteBuffer.class, Byte.BYTES, ?Byte?), > char.class, ? > ?, > ) > In fact TypeAttr really really wants to be a record (nominal tuple). Here?s an updated version which addresses all of the above points: http://cr.openjdk.java.net/~bpb/8245121/webrev.05/ > I believe we can use preview features in tests, and it is worthwhile to do so (gaining more coverage). Yes one can use preview features provided the following tags are in the test header: * @compile --enable-preview -source ${jdk.version} .java * @run testng/othervm --enable-preview The ??enable-preview? option requires ?othervm." Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri May 29 19:18:04 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 May 2020 12:18:04 -0700 Subject: 8245121: (bf) XBuffer.put(Xbuffer src) can give unexpected result when storage overlaps In-Reply-To: References: <3847C57F-422C-4888-A28F-97EB7D24891B@oracle.com> <33D59F6A-4D36-4D93-98EA-126779B5DB59@oracle.com> <1629FA4D-8578-4D4A-856D-B2203F5AB980@oracle.com> <9879B537-A609-41DA-A42D-F11B82ADCCE1@oracle.com> <03C1E3DC-0AEA-4BE8-9B01-4E09B7836357@oracle.com> <86BDC491-F719-41ED-A66E-D0CD0AC7D183@oracle.com> <7707D65F-B6D5-49AC-9A79-3968BF08F718@oracle.com> <766385FD-9C9D-458D-BD18-7DCE2A3BE073@oracle.com> <44B229C8-D95C-40C3-BB18-3A6732255B85@oracle.com> <4A41A109-DED2-4796-A143-305B58CF15B3@oracle.com> <9EC7A15D-AD97-40DA-90ED-6B905F5CD03E@oracle.com> <8708C721-D731-49EB-84BB-0737B2D7FD9D@oracle.com> Message-ID: Very nice +1 Paul. > On May 29, 2020, at 12:11 PM, Brian Burkhalter wrote: > > >> On May 29, 2020, at 9:25 AM, Paul Sandoz > wrote: >> >> I have one other request and one comment, both of which you can choose to ignore if you wanna wrap this up. >> >> - change the implementations of MyRandom.next$Type to use next() rather than your use your own shifting logic. >> >> - typeToAttr can be constructed using Map.of: >> Map.of( >> byte.class, TypeAttr.of(ByteBuffer.class, Byte.BYTES, ?Byte?), >> char.class, ? >> ?, >> ) >> In fact TypeAttr really really wants to be a record (nominal tuple). > > Here?s an updated version which addresses all of the above points: > > http://cr.openjdk.java.net/~bpb/8245121/webrev.05/ >> I believe we can use preview features in tests, and it is worthwhile to do so (gaining more coverage). > > Yes one can use preview features provided the following tags are in the test header: > > * @compile --enable-preview -source ${jdk.version} .java > * @run testng/othervm --enable-preview > > The ??enable-preview? option requires ?othervm." > > Thanks, > > Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: