From daniel.fuchs at oracle.com Mon Apr 1 09:51:56 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 1 Apr 2019 10:51:56 +0100 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: Hi Arthur, Not directly related to Alex's original question but... On 30/03/2019 00:03, Arthur Eubanks wrote: > We have some ipv6 patches as well in this area as well (as well as other > patches to support ipv6 only environments) that we're trying to > upstream. I don't understand the code myself, but it might be useful to > take a look: > http://cr.openjdk.java.net/~aeubanks/jdwpipv6/webrev.00/index.html SocketTransportService.java: On my machine at least, InetAddress.getByName("localhost") and InetAddress.getLocalHost() are quite different: InetAddress.getByName("localhost") will return the loopback (127.0.0.1) InetAddress.getLocalHost() returns the local hostname (dhcp-XXX-XXX...) If I'm not mistaken your changes in HostPort would transform something that previously returned the loopback (no host specified) with something that returns the local host name ("*" specified). So I'm not sure these changes are quite right. Maybe Alex will be able to confirm. best regards, -- daniel From chris.hegarty at oracle.com Mon Apr 1 11:50:42 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 1 Apr 2019 12:50:42 +0100 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: Alex, On 29/03/2019 22:07, Alex Menkov wrote: > (added net-dev as suggested by Alan) > > Net gurus, please assist in reviewing socket-related code. > > New webrev: > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ Specifically on SocketTransportService.java. What Arthur has proposed is better ( changing to lastIndexOf alone is not sufficient ). Or is your assumption that the IPv6 literal is not enclosed in square brackets? If keeping Arthur's `static class HostPort` please make the fields final. >> Would it make sense to support the preference properties? >> java.net.preferIPv4Stack >> java.net.preferIPv6Addresses I'm not sure about this, especially given the property name prefixes. I need to think a little more on it. There is quite a bit of new native code, is it possible to rewrite any of this in Java, e.g. reading of the system properties ( if that is to remain )? -Chris. From chris.hegarty at oracle.com Mon Apr 1 12:02:31 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 1 Apr 2019 13:02:31 +0100 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: <919bbc04-d908-9198-0ec6-2ea04cad0fbc@oracle.com> On 01/04/2019 12:50, Chris Hegarty wrote: > ... > Specifically on SocketTransportService.java. What Arthur has > proposed is better ( changing to lastIndexOf alone is not > sufficient ). Or is your assumption that the IPv6 literal is > not enclosed in square brackets? More specifically, what syntax are you proposing to pass IPv6 addresses from the command line? -Chris From chris.hegarty at oracle.com Mon Apr 1 13:11:40 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 1 Apr 2019 14:11:40 +0100 Subject: [ipv6] Issue Tracking Message-ID: Arthur, others, In order to help track the all the IPv6 related work, we have the following issues: Improved IPv6 Support ( for implementation issues ) https://bugs.openjdk.java.net/browse/JDK-8221681 Update JDK networking tests for IPv6-only environments ( test-only ) https://bugs.openjdk.java.net/browse/JDK-8220499 We can use these "umbrella" issues to keep track of the individual bugs and fixes, which can be either sub-tasks or linked-issues. Keeping a reasonably clean separation between test-only changes and "real" implementation changes should help with tracking ( since there will be likely a number of pushes/fixes that provide incremental improvements ). Again, for discussion, at least on net-dev, lets use the `[ipv6]` prefix in the subject line. Regards, -Chris. From chris.hegarty at oracle.com Mon Apr 1 13:40:33 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 1 Apr 2019 14:40:33 +0100 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: On 01/04/2019 10:51, Daniel Fuchs wrote: > Hi Arthur, > > Not directly related to Alex's original question but... > > On 30/03/2019 00:03, Arthur Eubanks wrote: >> We have some ipv6 patches as well in this area as well (as well as >> other patches to support ipv6 only environments) that we're trying to >> upstream. I don't understand the code myself, but it might be useful >> to take a look: >> http://cr.openjdk.java.net/~aeubanks/jdwpipv6/webrev.00/index.html > > SocketTransportService.java: > > On my machine at least, InetAddress.getByName("localhost") and > InetAddress.getLocalHost() are quite different: > > > InetAddress.getByName("localhost") will return the loopback (127.0.0.1) > InetAddress.getLocalHost() returns the local hostname (dhcp-XXX-XXX...) > > If I'm not mistaken your changes in HostPort would transform something > that previously returned the loopback (no host specified) with something > that returns the local host name ("*" specified). > > > So I'm not sure these changes are quite right. > Maybe Alex will be able to confirm. We need to consult the documentation for the command-line tools that exercise this code, or the higher-level API spec for the expected behavior of `*`, port-only specified, etc. -Chris. From aeubanks at google.com Mon Apr 1 15:47:39 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 1 Apr 2019 08:47:39 -0700 Subject: [ipv6] Issue Tracking In-Reply-To: References: Message-ID: Sounds good, thanks. On Mon, Apr 1, 2019 at 6:11 AM Chris Hegarty wrote: > Arthur, others, > > In order to help track the all the IPv6 related work, we have the > following issues: > > Improved IPv6 Support ( for implementation issues ) > https://bugs.openjdk.java.net/browse/JDK-8221681 > > Update JDK networking tests for IPv6-only environments ( test-only ) > https://bugs.openjdk.java.net/browse/JDK-8220499 > > We can use these "umbrella" issues to keep track of the individual > bugs and fixes, which can be either sub-tasks or linked-issues. Keeping > a reasonably clean separation between test-only changes and > "real" implementation changes should help with tracking ( since there > will be likely a number of pushes/fixes that provide incremental > improvements ). > > Again, for discussion, at least on net-dev, lets use the `[ipv6]` > prefix in the subject line. > > Regards, > -Chris. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Mon Apr 1 16:36:31 2019 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Apr 2019 09:36:31 -0700 Subject: [ipv6] Issue Tracking In-Reply-To: References: Message-ID: Sorry for never having had the time to finish https://bugs.openjdk.java.net/browse/JDK-8179037 Improve IPv6 support On Mon, Apr 1, 2019 at 6:12 AM Chris Hegarty wrote: > Arthur, others, > > In order to help track the all the IPv6 related work, we have the > following issues: > > Improved IPv6 Support ( for implementation issues ) > https://bugs.openjdk.java.net/browse/JDK-8221681 > > Update JDK networking tests for IPv6-only environments ( test-only ) > https://bugs.openjdk.java.net/browse/JDK-8220499 > > We can use these "umbrella" issues to keep track of the individual > bugs and fixes, which can be either sub-tasks or linked-issues. Keeping > a reasonably clean separation between test-only changes and > "real" implementation changes should help with tracking ( since there > will be likely a number of pushes/fixes that provide incremental > improvements ). > > Again, for discussion, at least on net-dev, lets use the `[ipv6]` > prefix in the subject line. > > Regards, > -Chris. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Mon Apr 1 17:36:25 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 1 Apr 2019 10:36:25 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: Hi Daniel, Chris, Unfortunately docs are out-dated (I plan to update it in JDK13). In JDK9 https://bugs.openjdk.java.net/browse/JDK-8041435 changed the behavior - empty address (i.e. only port is specified) means "local connections only", "*" means "all available connections". Then in JDK10 new "allow" option was introduced: https://bugs.openjdk.java.net/browse/JDK-8061228 (it allows to specify list of addresses/subnets to accept connections from). --alex On 04/01/2019 06:40, Chris Hegarty wrote: > > On 01/04/2019 10:51, Daniel Fuchs wrote: >> Hi Arthur, >> >> Not directly related to Alex's original question but... >> >> On 30/03/2019 00:03, Arthur Eubanks wrote: >>> We have some ipv6 patches as well in this area as well (as well as >>> other patches to support ipv6 only environments) that we're trying to >>> upstream. I don't understand the code myself, but it might be useful >>> to take a look: >>> http://cr.openjdk.java.net/~aeubanks/jdwpipv6/webrev.00/index.html >> >> SocketTransportService.java: >> >> On my machine at least, InetAddress.getByName("localhost") and >> InetAddress.getLocalHost() are quite different: >> >> >> InetAddress.getByName("localhost") will return the loopback (127.0.0.1) >> InetAddress.getLocalHost() returns the local hostname (dhcp-XXX-XXX...) >> >> If I'm not mistaken your changes in HostPort would transform something >> that previously returned the loopback (no host specified) with something >> that returns the local host name ("*" specified). >> >> >> So I'm not sure these changes are quite right. >> Maybe Alex will be able to confirm. > > We need to consult the documentation for the command-line tools that > exercise this code, or the higher-level API spec for the expected > behavior of `*`, port-only specified, etc. > > -Chris. From alexey.menkov at oracle.com Mon Apr 1 18:21:41 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 1 Apr 2019 11:21:41 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> Hi Chris, On 04/01/2019 04:50, Chris Hegarty wrote: > Alex, > > On 29/03/2019 22:07, Alex Menkov wrote: >> (added net-dev as suggested by Alan) >> >> Net gurus, please assist in reviewing socket-related code. >> >> New webrev: >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ > > Specifically on SocketTransportService.java. What Arthur has > proposed is better ( changing to lastIndexOf alone is not > sufficient ). Or is your assumption that the IPv6 literal is > not enclosed in square brackets? I didn't know about enclosing IPv6 in square brackets, but looks like that's standard way to alleviate conflict between IPv6 address and colon as port separator. Will update the fix to handle them in both JDI connectors (SocketTransportService.java) and debugger agent (socketTransport.c) > > If keeping Arthur's `static class HostPort` please make the > fields final. > > >> Would it make sense to support the preference properties? > >>??? java.net.preferIPv4Stack > >>??? java.net.preferIPv6Addresses > > I'm not sure about this, especially given the property name > prefixes. I need to think a little more on it. In the initial version of the fix I didn't check the properties. The rationale here is backward compatibility - is address is empty, old debuggers tries to connect to IPv4 address only. But handling this properties we will better handle clients with properties set (as jdb or any other debugger which uses JDI connectors are affected by the properties). BTW fix provided by Arthur implements listening on localhost differently - it creates several sockets and binds them to both IPv4 and IPv6 addresses. But the problem here (and that's the reason I decide to not implement it this way) - how to handle the case when we successfully bind on one address (for example IPv4), but fail to bind on other (for example the port is busy for IPv6 stack). Arthur's version just fail in the case (i.e. the whole Java process terminates) and I don't think this is good behavior. > > There is quite a bit of new native code, is it possible to > rewrite any of this in Java, e.g. reading of the system > properties ( if that is to remain )? socketTransport.c is a debugger agent which is completely native. --alex > > -Chris. From alexey.menkov at oracle.com Mon Apr 1 18:27:04 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 1 Apr 2019 11:27:04 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: <919bbc04-d908-9198-0ec6-2ea04cad0fbc@oracle.com> References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <919bbc04-d908-9198-0ec6-2ea04cad0fbc@oracle.com> Message-ID: <8318116e-7e45-d6a0-25ce-03bad40c5c27@oracle.com> On 04/01/2019 05:02, Chris Hegarty wrote: > > > On 01/04/2019 12:50, Chris Hegarty wrote: >> ... >> Specifically on SocketTransportService.java. What Arthur has >> proposed is better ( changing to lastIndexOf alone is not >> sufficient ). Or is your assumption that the IPv6 literal is >> not enclosed in square brackets? > > More specifically, what syntax are you proposing to pass IPv6 > addresses from the command line? > For JDI connectors (SocketTransportService.java) there are 2 separate connector arguments: "localAddress" and "port". For JDWP agent (socketTransport.c) arguments are provided as a single command line argument: java -agentlib:jdwp=transport=dt_socket,server=y,address=
: MyApp --alex > -Chris From aeubanks at google.com Mon Apr 1 20:52:29 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 1 Apr 2019 13:52:29 -0700 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> Message-ID: There are a bunch of tests with "-Djava.net.preferIPv4Stack=true", which should clearly fail in an IPv6 only environment. Would skipping these when IPv4 is not supported be okay? I see that some tests with "-Djava.net.preferIPv6Addresses=true" will pass even if the IPv6 part of the test failed. On Fri, Mar 29, 2019 at 12:59 PM Arthur Eubanks wrote: > I had just started on this yesterday. > I'll look at which of the tests we modified in our patches with ipv4/6 > detection. I'm worried that some of them may be disabling test coverage > that shouldn't be disabled, but I'll have to look in more detail. Also, we > don't have a JDK at head which works in our environments, so we might miss > some failures until we've decided to have a JDK at head that works in our > environments and run all of the networking tests. > > Thanks for the webrev, I'll definitely use that. > > On Fri, Mar 29, 2019 at 8:10 AM Chris Hegarty > wrote: > >> Arthur, >> >> Several tests require the presence of either: 1) IPv4, or 2) IPv6, or >> 3) IPv4&IPv6. A test library utility class could provide such. >> >> Here is a rough idea about how to determine Internet Protocol support. >> http://cr.openjdk.java.net/~chegar/8220673/webrev.00/ >> >> It might be enough to experiment with and see how far it gets us? >> >> -Chris. >> >> P.S. Let's use the `[ipv6]` subject prefix to track related work >> in this area. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Mon Apr 1 21:13:39 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 1 Apr 2019 14:13:39 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> Message-ID: <56dc7f42-a316-f660-4a88-3a2f799f4447@oracle.com> Hi Daniel, As far as I see you are right. Attach to address with empty hostname will try to connect to "external" address instead of loopback. --alex On 04/01/2019 02:51, Daniel Fuchs wrote: > Hi Arthur, > > Not directly related to Alex's original question but... > > On 30/03/2019 00:03, Arthur Eubanks wrote: >> We have some ipv6 patches as well in this area as well (as well as >> other patches to support ipv6 only environments) that we're trying to >> upstream. I don't understand the code myself, but it might be useful >> to take a look: >> http://cr.openjdk.java.net/~aeubanks/jdwpipv6/webrev.00/index.html > > SocketTransportService.java: > > On my machine at least, InetAddress.getByName("localhost") and > InetAddress.getLocalHost() are quite different: > > > InetAddress.getByName("localhost") will return the loopback (127.0.0.1) > InetAddress.getLocalHost() returns the local hostname (dhcp-XXX-XXX...) > > If I'm not mistaken your changes in HostPort would transform something > that previously returned the loopback (no host specified) with something > that returns the local host name ("*" specified). > > > So I'm not sure these changes are quite right. > Maybe Alex will be able to confirm. > > best regards, > > -- daniel From alexey.menkov at oracle.com Tue Apr 2 23:14:31 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 2 Apr 2019 16:14:31 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> Message-ID: Updated webrev: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/ - added support for addresses enclosed in square brackets; - updated SocketTransportService.java to handle empty hostname the same way as JDWP agent (listen/attach to loopback address); Has to update nsk/jdi/ListeningConnector/startListening/startlis001.java (all other com/sun/jdi, nsk/jdi, nsk/jdwp, nsk/jdb test are passed). --alex On 04/01/2019 11:21, Alex Menkov wrote: > Hi Chris, > > On 04/01/2019 04:50, Chris Hegarty wrote: >> Alex, >> >> On 29/03/2019 22:07, Alex Menkov wrote: >>> (added net-dev as suggested by Alan) >>> >>> Net gurus, please assist in reviewing socket-related code. >>> >>> New webrev: >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ >> >> Specifically on SocketTransportService.java. What Arthur has >> proposed is better ( changing to lastIndexOf alone is not >> sufficient ). Or is your assumption that the IPv6 literal is >> not enclosed in square brackets? > > I didn't know about enclosing IPv6 in square brackets, but looks like > that's standard way to alleviate conflict between IPv6 address and colon > as port separator. > Will update the fix to handle them in both JDI connectors > (SocketTransportService.java) and debugger agent (socketTransport.c) > >> >> If keeping Arthur's `static class HostPort` please make the >> fields final. >> >> ?>> Would it make sense to support the preference properties? >> ?>>??? java.net.preferIPv4Stack >> ?>>??? java.net.preferIPv6Addresses >> >> I'm not sure about this, especially given the property name >> prefixes. I need to think a little more on it. > > In the initial version of the fix I didn't check the properties. > The rationale here is backward compatibility - is address is empty, old > debuggers tries to connect to IPv4 address only. > But handling this properties we will better handle clients with > properties set (as jdb or any other debugger which uses JDI connectors > are affected by the properties). > > BTW fix provided by Arthur implements listening on localhost differently > - it creates several sockets and binds them to both IPv4 and IPv6 > addresses. But the problem here (and that's the reason I decide to not > implement it this way) - how to handle the case when we successfully > bind on one address (for example IPv4), but fail to bind on other (for > example the port is busy for IPv6 stack). Arthur's version just fail in > the case (i.e. the whole Java process terminates) and I don't think this > is good behavior. > > >> >> There is quite a bit of new native code, is it possible to >> rewrite any of this in Java, e.g. reading of the system >> properties ( if that is to remain )? > > socketTransport.c is a debugger agent which is completely native. > > --alex > >> >> -Chris. From nhoward at twitter.com Wed Apr 3 16:46:30 2019 From: nhoward at twitter.com (Nora Howard) Date: Wed, 3 Apr 2019 10:46:30 -0600 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX Message-ID: I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. On OSX 10.12 or later, when no mDNS services are registered, getaddrinfo fails on looking up localhost addresses[1]. The jdk?s current implementation falls back to getifaddrs after getaddrinfo fails[2]. When getaddrinfo fails in this way, it takes 5 seconds to fail [3]. This means that calls to InetAddress.getLocalHost() may take up to 5 seconds to run. I?d like to change the jdk?s behavior in this instance so that instead of falling back to calling getifaddrs after getaddrinfo fails, it instead calls getifaddrs first if we?re on OSX and the hostname is localhost?s. Doing this will eliminate the 5 second delay looking up localhost on machines with this network setup. When looking up non-localhost addresses, it?ll add a call to os:gethostname and a comparison. I?d like to get some feedback about whether this approach sounds reasonable, and some guidance about next steps. I?ve read the ?how to contribute? page[4], and the ?code review? guide[5]. I?ve put together a patch that I?ve tested locally by turning off mDNS services on my local OSX machine and exercising the behavior. I?d like to submit patches that target dev trunk as well as the jdk 8 and 11 branches. References 1. > On a Macintosh running 10.12 or later, if there are no apps that have a > mDNS services registered (for instance, iTunes) and there are no services > selected in the Sharing System Preferences, then the getaddrinfo will fail. > (https://bugs.openjdk.java.net/browse/JDK-8170910) 2. http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 (The lookupIfLocalhost function checks the passed hostname against os::get_host_name, and if they match, looks up via getifaddrs.) 3. This comment describes how to reproduce the issue. I?ve reproduced it locally on jdk 8, 11 and 13 ea. https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 4. https://openjdk.java.net/contribute/ 5. https://openjdk.java.net/guide/codeReview.html Thanks, -- Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Thu Apr 4 08:41:07 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 04 Apr 2019 09:41:07 +0100 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: References: Message-ID: <5CA5C323.4020508@oracle.com> Hi Nora, At first sight, the approach sounds reasonable to me. I'd like to see the patch and also do you have detailed instructions on how to reproduce the issue (the 5 second delay)? Thanks, Michael. On 03/04/2019, 17:46, Nora Howard wrote: > I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. > > On OSX 10.12 or later, when no mDNS services are registered, > getaddrinfo fails on looking up localhost addresses[1]. The jdk?s > current implementation falls back to getifaddrs after getaddrinfo > fails[2]. When getaddrinfo fails in this way, it takes 5 seconds to > fail [3]. This means that calls to InetAddress.getLocalHost() may take > up to 5 seconds to run. > > I?d like to change the jdk?s behavior in this instance so that instead > of falling back to calling getifaddrs after getaddrinfo fails, it > instead calls getifaddrs first if we?re on OSX and the hostname is > localhost?s. > > Doing this will eliminate the 5 second delay looking up localhost on > machines with this network setup. When looking up non-localhost > addresses, it?ll add a call to os:gethostname and a comparison. > > I?d like to get some feedback about whether this approach sounds > reasonable, and some guidance about next steps. I?ve read the ?how to > contribute? page[4], and the ?code review? guide[5]. > > I?ve put together a patch that I?ve tested locally by turning off mDNS > services on my local OSX machine and exercising the behavior. I?d like > to submit patches that target dev trunk as well as the jdk 8 and 11 > branches. > > References > > 1. > > On a Macintosh running 10.12 or later, if there are no apps that > have a mDNS services registered (for instance, iTunes) and there > are no services selected in the Sharing System Preferences, then > the getaddrinfo will fail. > > (https://bugs.openjdk.java.net/browse/JDK-8170910) > > 2. > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 > (The lookupIfLocalhost function checks the passed hostname against > os::get_host_name, and if they match, looks up via getifaddrs.) > > 3. This comment describes how to reproduce the issue. I?ve reproduced > it locally on jdk 8, 11 and 13 ea. > https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 > > > 4. https://openjdk.java.net/contribute/ > > 5. https://openjdk.java.net/guide/codeReview.html > > Thanks, > -- > Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From nhoward at twitter.com Thu Apr 4 17:38:22 2019 From: nhoward at twitter.com (Nora Howard) Date: Thu, 4 Apr 2019 11:38:22 -0600 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: <5CA5C323.4020508@oracle.com> References: <5CA5C323.4020508@oracle.com> Message-ID: Here's the diff http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ To setup an env to reproduce the issue, on a OSX machine with >= 10.12, turn off services that register with mDNS. For my laptop, I had to turn off iTunes and General System Preferences, disable "Allow Handoff between this Mac and your iCloud services". You might have to restart. The reporter on the ticket said to run `dns-sd -lo -B _services._dns-sd._udp local` to see whether you have any running. The file I used for exercising the behavior looks like this import java.net.InetAddress; public class Test { public static void main(String[] args) throws Exception { for (int i=0;i<4;i++){ long start = System.currentTimeMillis(); InetAddress localHost = InetAddress.getLocalHost(); System.out.println(localHost); System.out.println(System.currentTimeMillis() - start); }}} The output I get on my laptop when I've set things up this way looks like this on JDK 13 EA. The first call takes the full 5 seconds and subsequent ones hit a cache. nhoward.local/10.0.0.3 5044 nhoward.local/10.0.0.3 0 nhoward.local/10.0.0.3 0 nhoward.local/10.0.0.3 0 On JDK 8, each request takes 5 seconds. JDK 8's address lookup cache works differently. It's localhost name cache expiry is 5 seconds. On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon wrote: > Hi Nora, > > At first sight, the approach sounds reasonable to me. I'd like to see the > patch > and also do you have detailed instructions on how to reproduce the issue > (the 5 second delay)? > > Thanks, > Michael. > > On 03/04/2019, 17:46, Nora Howard wrote: > > I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. > > On OSX 10.12 or later, when no mDNS services are registered, getaddrinfo > fails on looking up localhost addresses[1]. The jdk?s current > implementation falls back to getifaddrs after getaddrinfo fails[2]. When > getaddrinfo fails in this way, it takes 5 seconds to fail [3]. This means > that calls to InetAddress.getLocalHost() may take up to 5 seconds to run. > > I?d like to change the jdk?s behavior in this instance so that instead of > falling back to calling getifaddrs after getaddrinfo fails, it instead > calls getifaddrs first if we?re on OSX and the hostname is localhost?s. > > Doing this will eliminate the 5 second delay looking up localhost on > machines with this network setup. When looking up non-localhost addresses, > it?ll add a call to os:gethostname and a comparison. > > I?d like to get some feedback about whether this approach sounds > reasonable, and some guidance about next steps. I?ve read the ?how to > contribute? page[4], and the ?code review? guide[5]. > > I?ve put together a patch that I?ve tested locally by turning off mDNS > services on my local OSX machine and exercising the behavior. I?d like to > submit patches that target dev trunk as well as the jdk 8 and 11 branches. > > References > > 1. > >> On a Macintosh running 10.12 or later, if there are no apps that have a >> mDNS services registered (for instance, iTunes) and there are no services >> selected in the Sharing System Preferences, then the getaddrinfo will fail. >> > (https://bugs.openjdk.java.net/browse/JDK-8170910) > > 2. > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 > (The lookupIfLocalhost function checks the passed hostname against > os::get_host_name, and if they match, looks up via getifaddrs.) > > 3. This comment describes how to reproduce the issue. I?ve reproduced it > locally on jdk 8, 11 and 13 ea. > https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 > > 4. https://openjdk.java.net/contribute/ > > 5. https://openjdk.java.net/guide/codeReview.html > > Thanks, > -- > Nora > > -- Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Fri Apr 5 16:12:51 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Fri, 05 Apr 2019 17:12:51 +0100 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: References: <5CA5C323.4020508@oracle.com> Message-ID: <5CA77E83.70606@oracle.com> I've spent some time trying to reproduce this on 10.13 but have not been able. I have disabled all the services as suggested (including some additional) such that the dns-sd command produces no output "records" and also by disabling unicast DNS access. I wonder if there has been a change in 10.13 to workaround this problem as I don't see any delay or external MDNS lookups happening. . Could you send me the output of "scutil --dns" of your laptop? At the same time, I will ask some colleagues here who might be running 10.12 to try it out. Thanks, Michael On 04/04/2019, 18:38, Nora Howard wrote: > Here's the diff http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ > > > To setup an env to reproduce the issue, on a OSX machine with >= > 10.12, turn off services that register with mDNS. For my laptop, I had > to turn off iTunes and General System Preferences, disable "Allow > Handoff between this Mac and your iCloud services". You might have to > restart. The reporter on the ticket said to run `dns-sd -lo -B > _services._dns-sd._udp local` to see whether you have any running. > > The file I used for exercising the behavior looks like this > > import java.net.InetAddress; > public class Test { > public static void main(String[] args) throws Exception { > for (int i=0;i<4;i++){ > long start = System.currentTimeMillis(); > InetAddress localHost = InetAddress.getLocalHost(); > System.out.println(localHost); > System.out.println(System.currentTimeMillis() - start); > }}} > > The output I get on my laptop when I've set things up this way looks > like this on JDK 13 EA. The first call takes the full 5 seconds and > subsequent ones hit a cache. > > nhoward.local/10.0.0.3 > 5044 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > > On JDK 8, each request takes 5 seconds. JDK 8's address lookup cache > works differently. It's localhost name cache expiry is 5 seconds. > > > > On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon > > > wrote: > > Hi Nora, > > At first sight, the approach sounds reasonable to me. I'd like to > see the patch > and also do you have detailed instructions on how to reproduce the > issue (the 5 second delay)? > > Thanks, > Michael. > > On 03/04/2019, 17:46, Nora Howard wrote: >> I'd initially sent this to jdk-dev, but I was asked to move it to >> net-dev. >> >> On OSX 10.12 or later, when no mDNS services are registered, >> getaddrinfo fails on looking up localhost addresses[1]. The jdk?s >> current implementation falls back to getifaddrs after getaddrinfo >> fails[2]. When getaddrinfo fails in this way, it takes 5 seconds >> to fail [3]. This means that calls to InetAddress.getLocalHost() >> may take up to 5 seconds to run. >> >> I?d like to change the jdk?s behavior in this instance so that >> instead of falling back to calling getifaddrs after getaddrinfo >> fails, it instead calls getifaddrs first if we?re on OSX and the >> hostname is localhost?s. >> >> Doing this will eliminate the 5 second delay looking up localhost >> on machines with this network setup. When looking up >> non-localhost addresses, it?ll add a call to os:gethostname and a >> comparison. >> >> I?d like to get some feedback about whether this approach sounds >> reasonable, and some guidance about next steps. I?ve read the >> ?how to contribute? page[4], and the ?code review? guide[5]. >> >> I?ve put together a patch that I?ve tested locally by turning off >> mDNS services on my local OSX machine and exercising the >> behavior. I?d like to submit patches that target dev trunk as >> well as the jdk 8 and 11 branches. >> >> References >> >> 1. >> >> On a Macintosh running 10.12 or later, if there are no apps >> that have a mDNS services registered (for instance, iTunes) >> and there are no services selected in the Sharing System >> Preferences, then the getaddrinfo will fail. >> >> (https://bugs.openjdk.java.net/browse/JDK-8170910) >> >> 2. >> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 >> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 >> (The lookupIfLocalhost function checks the passed hostname >> against os::get_host_name, and if they match, looks up via >> getifaddrs.) >> >> 3. This comment describes how to reproduce the issue. I?ve >> reproduced it locally on jdk 8, 11 and 13 ea. >> https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 >> >> >> 4. https://openjdk.java.net/contribute/ >> >> 5. https://openjdk.java.net/guide/codeReview.html >> >> Thanks, >> -- >> Nora > > > > -- > Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From nhoward at twitter.com Fri Apr 5 19:15:48 2019 From: nhoward at twitter.com (Nora Howard) Date: Fri, 5 Apr 2019 13:15:48 -0600 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: <5CA77E83.70606@oracle.com> References: <5CA5C323.4020508@oracle.com> <5CA77E83.70606@oracle.com> Message-ID: Oh, another element is that to see the behavior, your local hostname has to not have an entry in /etc/hosts. Adding it is one of the workarounds. My /etc/hosts has a localhost entry that does not include the name my machine ends up with after boot. I'm also on 10.13, so I don't think that's the issue per se. When I run scutil --dns. I get a resolver that points to en01 with some names on my VPN, and a number of mdns registrations that all look like this, where the resolver number, the domain and the order change, but the rest of the fields stay the same. resolver #2 domain : local options : mdns timeout : 5 flags : Request A records reach : 0x00000000 (Not Reachable) order : 300000 On Fri, Apr 5, 2019 at 10:14 AM Michael McMahon < michael.x.mcmahon at oracle.com> wrote: > I've spent some time trying to reproduce this on 10.13 but have not been > able. > I have disabled all the services as suggested (including some additional) > such that the dns-sd command produces no output "records" and also by > disabling unicast DNS access. > > I wonder if there has been a change in 10.13 to workaround this problem > as I don't see any delay or external MDNS lookups happening. > . > Could you send me the output of "scutil --dns" of your laptop? > At the same time, I will ask some colleagues here who might be running > 10.12 > to try it out. > > Thanks, > Michael > > On 04/04/2019, 18:38, Nora Howard wrote: > > Here's the diff http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ > > To setup an env to reproduce the issue, on a OSX machine with >= 10.12, > turn off services that register with mDNS. For my laptop, I had to turn off > iTunes and General System Preferences, disable "Allow Handoff between this > Mac and your iCloud services". You might have to restart. The reporter on > the ticket said to run `dns-sd -lo -B _services._dns-sd._udp local` to see > whether you have any running. > > The file I used for exercising the behavior looks like this > > import java.net.InetAddress; > public class Test { > public static void main(String[] args) throws Exception { > for (int i=0;i<4;i++){ > long start = System.currentTimeMillis(); > InetAddress localHost = InetAddress.getLocalHost(); > System.out.println(localHost); > System.out.println(System.currentTimeMillis() - start); > }}} > > The output I get on my laptop when I've set things up this way looks like > this on JDK 13 EA. The first call takes the full 5 seconds and subsequent > ones hit a cache. > > nhoward.local/10.0.0.3 > 5044 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > > On JDK 8, each request takes 5 seconds. JDK 8's address lookup cache works > differently. It's localhost name cache expiry is 5 seconds. > > > > On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon < > michael.x.mcmahon at oracle.com> wrote: > >> Hi Nora, >> >> At first sight, the approach sounds reasonable to me. I'd like to see the >> patch >> and also do you have detailed instructions on how to reproduce the issue >> (the 5 second delay)? >> >> Thanks, >> Michael. >> >> On 03/04/2019, 17:46, Nora Howard wrote: >> >> I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. >> >> On OSX 10.12 or later, when no mDNS services are registered, getaddrinfo >> fails on looking up localhost addresses[1]. The jdk?s current >> implementation falls back to getifaddrs after getaddrinfo fails[2]. When >> getaddrinfo fails in this way, it takes 5 seconds to fail [3]. This means >> that calls to InetAddress.getLocalHost() may take up to 5 seconds to run. >> >> I?d like to change the jdk?s behavior in this instance so that instead of >> falling back to calling getifaddrs after getaddrinfo fails, it instead >> calls getifaddrs first if we?re on OSX and the hostname is localhost?s. >> >> Doing this will eliminate the 5 second delay looking up localhost on >> machines with this network setup. When looking up non-localhost addresses, >> it?ll add a call to os:gethostname and a comparison. >> >> I?d like to get some feedback about whether this approach sounds >> reasonable, and some guidance about next steps. I?ve read the ?how to >> contribute? page[4], and the ?code review? guide[5]. >> >> I?ve put together a patch that I?ve tested locally by turning off mDNS >> services on my local OSX machine and exercising the behavior. I?d like to >> submit patches that target dev trunk as well as the jdk 8 and 11 branches. >> >> References >> >> 1. >> >>> On a Macintosh running 10.12 or later, if there are no apps that have a >>> mDNS services registered (for instance, iTunes) and there are no services >>> selected in the Sharing System Preferences, then the getaddrinfo will fail. >>> >> (https://bugs.openjdk.java.net/browse/JDK-8170910) >> >> 2. >> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 >> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 >> (The lookupIfLocalhost function checks the passed hostname against >> os::get_host_name, and if they match, looks up via getifaddrs.) >> >> 3. This comment describes how to reproduce the issue. I?ve reproduced it >> locally on jdk 8, 11 and 13 ea. >> https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 >> >> 4. https://openjdk.java.net/contribute/ >> >> 5. https://openjdk.java.net/guide/codeReview.html >> >> Thanks, >> -- >> Nora >> >> > > -- > Nora > > -- Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Apr 8 11:34:32 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 Apr 2019 12:34:32 +0100 Subject: NIO based SocketImpl to replace legacy PlainSocketImpl In-Reply-To: <179696de-0340-8b1e-b802-2959ad59858b@oracle.com> References: <3225f8f0-08f2-c9bb-6dc4-b7eb7cfc4aa4@oracle.com> <179696de-0340-8b1e-b802-2959ad59858b@oracle.com> Message-ID: <42f4dbae-0e46-0137-03b9-bea5ad2ef772@oracle.com> Just a quick summary on where we are on this effort. I've submitted the JEP [1] so it can be considered for the roadmap. There wasn't any additional feedback from the draft that I sent here in February. In the mean-time, the "preparatory" changes have been pushed to jdk/jdk: JDK-8220493: Prepare Socket/ServerSocket for alternative platform SocketImpl JDK-8221259: New tests for java.net.Socket to exercise long standing behavior JDK-8220738: (sc) Move ServerSocketChannelImpl remaining native method to Net JDK-8217500: (sc) Move SocketChannelImpl's remaining native methods to Net JDK_8221252: (sc) SocketChannel and its socket adaptor need to handle connection reset JDK-8217461: (sc) Add Net.available to return the number of bytes in the socket input buffer JDK-8217451: ExtendedSocketOptions should encapsulate support for SO_FLOW_SLA JDK-8219446: Specify behaviour of timeout accepting methods of Socket and ServerSocket if timeout is negative and the CSR [2] has been removed to "Provisional". Getting the enabling changes pushed in advance means the changes are significantly reduced and are mostly one class with the new implementation. A snapshot of the patch from the niosocketimpl-branch is here: ? http://cr.openjdk.java.net/~alanb/8221481/0/webrev/index.html If the JEP becomes a candidate then then changes will need Reviewers. We will also need to decide on the system property to configure the JDK to use the legacy implementation. At this time, you can run with -Djdk.net.usePlainSocketImpl=true to use the legacy PlainSocketImpl. It probably not wroth spending time on anything more sophisticated. In addition to the above patch, the niosocketimpl-branch has updates to several existing socket tests to run them a second time with -Djdk.net.usePlainSocketImpl. I left them out of the above snapshot to avoid clutter as it's a one-line to each test description. As documented in the JEP, the motive is to avoid the legacy implementation bit rotting before it is removed. The niosocketimpl-branch also has Sergey Kuksenko's updates to the socket micros which may make it to the main line in advance of the changes. -Alan. [1] http://openjdk.java.net/jeps/8218559 [2] https://bugs.openjdk.java.net/browse/JDK-8222028 From chris.hegarty at oracle.com Tue Apr 9 11:17:35 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 9 Apr 2019 12:17:35 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> Message-ID: <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Arthur, > On 1 Apr 2019, at 21:52, Arthur Eubanks wrote: > > There are a bunch of tests with "-Djava.net.preferIPv4Stack=true", which should clearly fail in an IPv6 only environment. Right. > Would skipping these when IPv4 is not supported be okay? Yes, I believe so, but we will need to check what specific functionality that the test is exercising. In many cases it's just the TCP/UDP stack without IPv6 support, to exercise more code paths. These cases should be fine to be excluded for IPv6-only environments. I'll play a little with the jtreg @requires functionality, but depending on the test structure and the existing @run tags, it may be more straight forward to just check for the presence of the system property and platform support ( wrapped up into a convenient test library function ), returning early from the test where it doesn't make sense to run on that configuration. > I see that some tests with "-Djava.net.preferIPv6Addresses=true" will pass even if the IPv6 part of the test failed. Do you have a list of these? > On Fri, Mar 29, 2019 at 12:59 PM Arthur Eubanks > wrote: > ... Also, we don't have a JDK at head which works in our environments, so we might miss some failures until we've decided to have a JDK at head that works in our environments and run all of the networking tests. Right, there is a chicken and egg problem here, but you are correct that we need to make progress on an IPv6-only JDK build. If not yet integrated, at least a generally agreed patch / approach. -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Tue Apr 9 16:01:25 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 09 Apr 2019 17:01:25 +0100 Subject: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: References: <5CA5C323.4020508@oracle.com> <5CA77E83.70606@oracle.com> Message-ID: <5CACC1D5.80507@oracle.com> Hi, So, I reproduced the issue, albeit with one configuration change that I'm not sure whether is common or not. Basically, I could see that lookups to any host with a ".local" suffix would use MDNS and if the local services are not active, then I see the packets being sent on the wire, no responses and a delay of 5 seconds before the lookup fails. Initially, this was not affecting localhost lookups which still use normal DNS and even when DNS not available, that was failing immediately and therefore no delay. To make InetAddress.getLocalHost() fail with a 5 second delay, I had to add ".local" to the DNS search path of the system. That has the effect of attempting a MDNS lookup after the DNS lookup fails, and this delays the failure for 5 seconds. One question is if this is a common network configuration, or is there some other way to cause it to happen? Looking at the patch again, I think perhaps (assuming we want to make a change here) a different approach might be better. Quite a lot of work is happening in the lookupIfLocalhost() function and I think we should try to limit this being called to only the local hostname lookup path. This might require boolean flags being being passed down from the InetAddress.getLocalHost() call to indicate whether this is a local address lookup or not. Thanks, Michael On 05/04/2019, 20:15, Nora Howard wrote: > Oh, another element is that to see the behavior, your local hostname > has to not have an entry in /etc/hosts. Adding it is one of the > workarounds. My /etc/hosts has a localhost entry that does not include > the name my machine ends up with after boot. > > I'm also on 10.13, so I don't think that's the issue per se. > > When I run scutil --dns. I get a resolver that points to en01 with > some names on my VPN, and a number of mdns registrations that all look > like this, where the resolver number, the domain and the order change, > but the rest of the fields stay the same. > resolver #2 > domain : local > options : mdns > timeout : 5 > flags : Request A records > reach : 0x00000000 (Not Reachable) > order : 300000 > > > > On Fri, Apr 5, 2019 at 10:14 AM Michael McMahon > > > wrote: > > I've spent some time trying to reproduce this on 10.13 but have > not been able. > I have disabled all the services as suggested (including some > additional) > such that the dns-sd command produces no output "records" and also by > disabling unicast DNS access. > > I wonder if there has been a change in 10.13 to workaround this > problem > as I don't see any delay or external MDNS lookups happening. > . > Could you send me the output of "scutil --dns" of your laptop? > At the same time, I will ask some colleagues here who might be > running 10.12 > to try it out. > > Thanks, > Michael > > On 04/04/2019, 18:38, Nora Howard wrote: >> Here's the diff >> http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ >> >> >> To setup an env to reproduce the issue, on a OSX machine with >= >> 10.12, turn off services that register with mDNS. For my laptop, >> I had to turn off iTunes and General System Preferences, disable >> "Allow Handoff between this Mac and your iCloud services". You >> might have to restart. The reporter on the ticket said to run >> `dns-sd -lo -B _services._dns-sd._udp local` to see whether you >> have any running. >> >> The file I used for exercising the behavior looks like this >> >> import java.net.InetAddress; >> public class Test { >> public static void main(String[] args) throws Exception { >> for (int i=0;i<4;i++){ >> long start = System.currentTimeMillis(); >> InetAddress localHost = InetAddress.getLocalHost(); >> System.out.println(localHost); >> System.out.println(System.currentTimeMillis() - start); >> }}} >> >> The output I get on my laptop when I've set things up this way >> looks like this on JDK 13 EA. The first call takes the full 5 >> seconds and subsequent ones hit a cache. >> >> nhoward.local/10.0.0.3 >> 5044 >> nhoward.local/10.0.0.3 >> 0 >> nhoward.local/10.0.0.3 >> 0 >> nhoward.local/10.0.0.3 >> 0 >> >> On JDK 8, each request takes 5 seconds. JDK 8's address lookup >> cache works differently. It's localhost name cache expiry is 5 >> seconds. >> >> >> >> On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon >> > > wrote: >> >> Hi Nora, >> >> At first sight, the approach sounds reasonable to me. I'd >> like to see the patch >> and also do you have detailed instructions on how to >> reproduce the issue (the 5 second delay)? >> >> Thanks, >> Michael. >> >> On 03/04/2019, 17:46, Nora Howard wrote: >>> I'd initially sent this to jdk-dev, but I was asked to move >>> it to net-dev. >>> >>> On OSX 10.12 or later, when no mDNS services are registered, >>> getaddrinfo fails on looking up localhost addresses[1]. The >>> jdk?s current implementation falls back to getifaddrs after >>> getaddrinfo fails[2]. When getaddrinfo fails in this way, it >>> takes 5 seconds to fail [3]. This means that calls to >>> InetAddress.getLocalHost() may take up to 5 seconds to run. >>> >>> I?d like to change the jdk?s behavior in this instance so >>> that instead of falling back to calling getifaddrs after >>> getaddrinfo fails, it instead calls getifaddrs first if >>> we?re on OSX and the hostname is localhost?s. >>> >>> Doing this will eliminate the 5 second delay looking up >>> localhost on machines with this network setup. When looking >>> up non-localhost addresses, it?ll add a call to >>> os:gethostname and a comparison. >>> >>> I?d like to get some feedback about whether this approach >>> sounds reasonable, and some guidance about next steps. I?ve >>> read the ?how to contribute? page[4], and the ?code review? >>> guide[5]. >>> >>> I?ve put together a patch that I?ve tested locally by >>> turning off mDNS services on my local OSX machine and >>> exercising the behavior. I?d like to submit patches that >>> target dev trunk as well as the jdk 8 and 11 branches. >>> >>> References >>> >>> 1. >>> >>> On a Macintosh running 10.12 or later, if there are no >>> apps that have a mDNS services registered (for instance, >>> iTunes) and there are no services selected in the >>> Sharing System Preferences, then the getaddrinfo will fail. >>> >>> (https://bugs.openjdk.java.net/browse/JDK-8170910) >>> >>> 2. >>> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 >>> http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 >>> (The lookupIfLocalhost function checks the passed hostname >>> against os::get_host_name, and if they match, looks up via >>> getifaddrs.) >>> >>> 3. This comment describes how to reproduce the issue. I?ve >>> reproduced it locally on jdk 8, 11 and 13 ea. >>> https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 >>> >>> >>> 4. https://openjdk.java.net/contribute/ >>> >>> 5. https://openjdk.java.net/guide/codeReview.html >>> >>> Thanks, >>> -- >>> Nora >> >> >> >> -- >> Nora > > > > -- > Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Wed Apr 10 17:51:08 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Wed, 10 Apr 2019 10:51:08 -0700 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: > > There are a bunch of tests with "-Djava.net.preferIPv4Stack=true", which > should clearly fail in an IPv6 only environment. > > > Right. > > Would skipping these when IPv4 is not supported be okay? > > > Yes, I believe so, but we will need to check what specific functionality > that the test is exercising. In many cases it's just the TCP/UDP stack > without IPv6 support, to exercise more code paths. These cases should be > fine to be excluded for IPv6-only environments. > > I'll play a little with the jtreg @requires functionality, but depending > on the test structure and the existing @run tags, it may be more > straight forward to just check for the presence of the system property > and platform support ( wrapped up into a convenient test library > function ), returning early from the test where it doesn't make sense to > run on that configuration. > So at the beginning of each networking test that tests multiple configurations, there is something like `if (IPSupport.currentConfigurationNotSupported()) { return; }`? That seems slightly ugly, but maybe that's the best way. > > I see that some tests with "-Djava.net.preferIPv6Addresses=true" will pass > even if the IPv6 part of the test failed. > > > Do you have a list of these? > test/jdk/java/nio/channels/DatagramChannel/BindNull.java is the only one I see. > > > On Fri, Mar 29, 2019 at 12:59 PM Arthur Eubanks > wrote: > >> ... Also, we don't have a JDK at head which works in our environments, so >> we might miss some failures until we've decided to have a JDK at head that >> works in our environments and run all of the networking tests. >> > > Right, there is a chicken and egg problem here, but you are correct that > we need to make progress on an IPv6-only JDK build. If not yet > integrated, at least a generally agreed patch / approach. > > -Chris. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Wed Apr 10 20:39:15 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Wed, 10 Apr 2019 13:39:15 -0700 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: Here's a prototype webrev to see if the approach is okay with you. If it looks good, I'll continue with the remaining tests I can find. (should I start a new thread for the webrev?) http://cr.openjdk.java.net/~aeubanks/8220673/webrev.00 On Wed, Apr 10, 2019 at 10:51 AM Arthur Eubanks wrote: > There are a bunch of tests with "-Djava.net.preferIPv4Stack=true", which >> should clearly fail in an IPv6 only environment. >> >> >> Right. >> >> Would skipping these when IPv4 is not supported be okay? >> >> >> Yes, I believe so, but we will need to check what specific functionality >> that the test is exercising. In many cases it's just the TCP/UDP stack >> without IPv6 support, to exercise more code paths. These cases should be >> fine to be excluded for IPv6-only environments. >> >> I'll play a little with the jtreg @requires functionality, but depending >> on the test structure and the existing @run tags, it may be more >> straight forward to just check for the presence of the system property >> and platform support ( wrapped up into a convenient test library >> function ), returning early from the test where it doesn't make sense to >> run on that configuration. >> > So at the beginning of each networking test that tests multiple > configurations, there is something like `if > (IPSupport.currentConfigurationNotSupported()) { return; }`? > That seems slightly ugly, but maybe that's the best way. > >> >> I see that some tests with "-Djava.net.preferIPv6Addresses=true" will >> pass even if the IPv6 part of the test failed. >> >> >> Do you have a list of these? >> > test/jdk/java/nio/channels/DatagramChannel/BindNull.java is the only one I > see. > >> >> >> On Fri, Mar 29, 2019 at 12:59 PM Arthur Eubanks >> wrote: >> >>> ... Also, we don't have a JDK at head which works in our environments, >>> so we might miss some failures until we've decided to have a JDK at head >>> that works in our environments and run all of the networking tests. >>> >> >> Right, there is a chicken and egg problem here, but you are correct that >> we need to make progress on an IPv6-only JDK build. If not yet >> integrated, at least a generally agreed patch / approach. >> >> -Chris. >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Wed Apr 10 21:03:33 2019 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 10 Apr 2019 23:03:33 +0200 Subject: AW: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: <5CACC1D5.80507@oracle.com> References: <5CA5C323.4020508@oracle.com> <5CA77E83.70606@oracle.com> <5CACC1D5.80507@oracle.com> Message-ID: <5cae5a25.1c69fb81.113e.0304@mx.google.com> Maybe instead of having a MacOS specific algorithm it would be a good idea to have getLocalHostName generally look up the hostnames from the Array with a trailing ?.? and only if that Fails have it search with search Suffix? Also I think we talked before that removing AI_CANONNAME would also help. (Especially since it does a reverse lookup itself anyway). Gruss Bernd Von: Michael McMahon Gesendet: Dienstag, 9. April 2019 18:35 An: Nora Howard Cc: net-dev at openjdk.java.net Betreff: Re: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX Hi, So, I reproduced the issue, albeit with one configuration change that I'm not sure whether is common or not. Basically, I could see that lookups to any host with a ".local" suffix would use MDNS and if the local services are not active, then I see the packets being sent on the wire, no responses and a delay of 5 seconds before the lookup fails. Initially, this was not affecting localhost lookups which still use normal DNS and even when DNS not available, that was failing immediately and therefore no delay. To make InetAddress.getLocalHost() fail with a 5 second delay, I had to add ".local" to the DNS search path of the system. That has the effect of attempting a MDNS lookup after the DNS lookup fails, and this delays the failure for 5 seconds. One question is if this is a common network configuration, or is there some other way to cause it to happen? Looking at the patch again, I think perhaps (assuming we want to make a change here) a different approach might be better. Quite a lot of work is happening in the lookupIfLocalhost() function and I think we should try to limit this being called to only the local hostname lookup path. This might require boolean flags being being passed down from the InetAddress.getLocalHost() call to indicate whether this is a local address lookup or not. Thanks, Michael On 05/04/2019, 20:15, Nora Howard wrote: Oh, another element is that to see the behavior, your local hostname has to not have an entry in /etc/hosts. Adding it is one of the workarounds. My /etc/hosts has a localhost entry that does not include the name my machine ends up with after boot. I'm also on 10.13, so I don't think that's the issue per se. When I run scutil --dns. I get a resolver that points to en01 with some names on my VPN, and a number of mdns registrations that all look like this, where the resolver number, the domain and the order change, but the rest of the fields stay the same. resolver #2 ? domain?? : local ? options? : mdns ? timeout? : 5 ? flags??? : Request A records ? reach??? : 0x00000000 (Not Reachable) ? order??? : 300000 On Fri, Apr 5, 2019 at 10:14 AM Michael McMahon wrote: I've spent some time trying to reproduce this on 10.13 but have not been able. I have disabled all the services as suggested (including some additional) such that the dns-sd command produces no output "records" and also by disabling unicast DNS access. I wonder if there has been a change in 10.13 to workaround this problem as I don't see any delay or external MDNS lookups happening. . Could you send me the output of "scutil --dns" of your laptop? At the same time, I will ask some colleagues here who might be running 10.12 to try it out. Thanks, Michael On 04/04/2019, 18:38, Nora Howard wrote: Here's the diff http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ To setup an env to reproduce the issue, on a OSX machine with >= 10.12, turn off services that register with mDNS. For my laptop, I had to turn off iTunes and General System Preferences, disable "Allow Handoff between this Mac and your iCloud services". You might have to restart. The reporter on the ticket said to run `dns-sd -lo -B _services._dns-sd._udp local` to see whether you have any running. The file I used for exercising the behavior looks like this import java.net.InetAddress; public class Test { ? public static void main(String[] args) throws Exception { ????? for (int i=0;i<4;i++){ ????????? long start = System.currentTimeMillis(); ????????? InetAddress localHost = InetAddress.getLocalHost(); ????????? System.out.println(localHost); ????????? System.out.println(System.currentTimeMillis() - start); }}} The output I get on my laptop when I've set things up this way looks like this on JDK 13 EA. The first call takes the full 5 seconds and subsequent ones hit a cache. nhoward.local/10.0.0.3 5044 nhoward.local/10.0.0.3 0 nhoward.local/10.0.0.3 0 nhoward.local/10.0.0.3 0 On JDK 8, each request takes 5 seconds. JDK 8's address lookup cache works differently. It's localhost name cache expiry is 5 seconds. On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon wrote: Hi Nora, At first sight, the approach sounds reasonable to me. I'd like to see the patch and also do you have detailed instructions on how to reproduce the issue (the 5 second delay)? Thanks, Michael. On 03/04/2019, 17:46, Nora Howard wrote: I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. On OSX 10.12 or later, when no mDNS services are registered, getaddrinfo fails on looking up localhost addresses[1]. The jdk?s current implementation falls back to getifaddrs after getaddrinfo fails[2]. When getaddrinfo fails in this way, it takes 5 seconds to fail [3]. This means that calls to InetAddress.getLocalHost() may take up to 5 seconds to run. I?d like to change the jdk?s behavior in this instance so that instead of falling back to calling getifaddrs after getaddrinfo fails, it instead calls getifaddrs first if we?re on OSX and the hostname is localhost?s. Doing this will eliminate the 5 second delay looking up localhost on machines with this network setup. When looking up non-localhost addresses, it?ll add a call to os:gethostname and a comparison. I?d like to get some feedback about whether this approach sounds reasonable, and some guidance about next steps. I?ve read the ?how to contribute? page[4], and the ?code review? guide[5]. I?ve put together a patch that I?ve tested locally by turning off mDNS services on my local OSX machine and exercising the behavior. I?d like to submit patches that target dev trunk as well as the jdk 8 and 11 branches. References 1. On a Macintosh running 10.12 or later, if there are no apps that have a mDNS services registered (for instance, iTunes) and there are no services selected in the Sharing System Preferences, then the getaddrinfo will fail. (https://bugs.openjdk.java.net/browse/JDK-8170910) 2. http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 (The lookupIfLocalhost function checks the passed hostname against os::get_host_name, and if they match, looks up via getifaddrs.) 3. This comment describes how to reproduce the issue. I?ve reproduced it locally on jdk 8, 11 and 13 ea. https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 4. https://openjdk.java.net/contribute/ 5. https://openjdk.java.net/guide/codeReview.html Thanks, -- Nora -- Nora -- Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From norman.maurer at googlemail.com Thu Apr 11 12:40:53 2019 From: norman.maurer at googlemail.com (Norman Maurer) Date: Thu, 11 Apr 2019 14:40:53 +0200 Subject: ServerSocket.isBound() continue to return true after close() is called. Message-ID: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> Hi there, While working on netty I just noticed that a ServerSocket will keep return true when isBound() is called even after it was closed. Is this by design? I was bit surprised by this honestly as after the socket is closed there is no way it will accept any more connections. If this is by design should we at least update the java docs ? https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/ServerSocket.html#isBound() In contrast for DatagramSocket it is directly called out: https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/DatagramSocket.html#isBound() So I wonder if it was mainly done the same way for a ServerSocket to be more consistent here or if it is just an oversight. This code shows the behaviour. ServerSocket socket = new ServerSocket(0); socket.close(); if (socket.isBound()) { throw new AssertionError(); } Thanks, Norman -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Thu Apr 11 13:05:13 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 11 Apr 2019 14:05:13 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: Hi Arthur, This looks like a good cleanup to me. On 10/04/2019 21:39, Arthur Eubanks wrote: > Here's a prototype webrev to see if the approach is okay with you. If it > looks good, I'll continue with the remaining tests I can find. (should I > start a new thread for the webrev?) > > http://cr.openjdk.java.net/~aeubanks/8220673/webrev.00 DatagramChannel/BindNull.java 48 if (IPSupport.hasIPv4()) { 49 try (DatagramChannel dc = DatagramChannel.open(StandardProtocolFamily.INET6)) { 50 dc.bind(null); 51 } 52 } I guess you meant `IPSupport.hasIPv6()` here? We're putting a lot of trust in the implementation of this IPSupport utility. Have you been able to verify visually - by looking at the tests logs and trying out different configuration on your local machine - that it is behaving as expected? best regards, -- daniel From michael.x.mcmahon at oracle.com Thu Apr 11 15:12:35 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 11 Apr 2019 16:12:35 +0100 Subject: ServerSocket.isBound() continue to return true after close() is called. In-Reply-To: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> References: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> Message-ID: <5CAF5963.3020106@oracle.com> Norman The specification on what happens to all socket types was updated many years ago in bug id 6505016, but it looks like ServerSocket::isBound was missed from that effort. I think we should probably update the spec to reflect current behavior and be consistent with the change above. There will be some small spec updates to ServerSocket coming which originated from the SocketImpl replacement work that Alan mentioned recently and I think we can include this small change probably with one of those. Michael. On 11/04/2019, 13:40, Norman Maurer wrote: > Hi there, > > While working on netty I just noticed that a ServerSocket will keep > return true when isBound() is called even after it was closed. Is this > by design? I was bit surprised by this honestly as after the socket is > closed there is no way it will accept any more connections. > > If this is by design should we at least update the java docs ? > > https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/ServerSocket.html#isBound() > > > In contrast for DatagramSocket it is directly called out: > > https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/DatagramSocket.html#isBound() > > > So I wonder if it was mainly done the same way for a ServerSocket to > be more consistent here or if it is just an oversight. > > This code shows the behaviour. > > ServerSocket socket =newServerSocket(0); > > socket.close(); > if(socket.isBound()) { > throw newAssertionError(); > } > > > > Thanks, > Norman > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norman.maurer at googlemail.com Thu Apr 11 15:27:26 2019 From: norman.maurer at googlemail.com (Norman Maurer) Date: Thu, 11 Apr 2019 17:27:26 +0200 Subject: ServerSocket.isBound() continue to return true after close() is called. In-Reply-To: <5CAF5963.3020106@oracle.com> References: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> <5CAF5963.3020106@oracle.com> Message-ID: <8A9BC985-E2A4-45B6-B2D6-D648E7B409F0@googlemail.com> Ok thanks? update to the java docs sounds good to me. I was just suprised by the behaviour :) > On 11. Apr 2019, at 17:12, Michael McMahon wrote: > > Norman > > The specification on what happens to all socket types was updated many years ago > in bug id 6505016, but it looks like ServerSocket::isBound was missed from that effort. > I think we should probably update the spec to reflect current behavior and be consistent > with the change above. There will be some small spec updates to ServerSocket coming > which originated from the SocketImpl replacement work that Alan mentioned recently > and I think we can include this small change probably with one of those. > > Michael. > > On 11/04/2019, 13:40, Norman Maurer wrote: >> >> Hi there, >> >> While working on netty I just noticed that a ServerSocket will keep return true when isBound() is called even after it was closed. Is this by design? I was bit surprised by this honestly as after the socket is closed there is no way it will accept any more connections. >> >> If this is by design should we at least update the java docs ? >> >> https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/ServerSocket.html#isBound() >> >> In contrast for DatagramSocket it is directly called out: >> >> https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/net/DatagramSocket.html#isBound() >> >> So I wonder if it was mainly done the same way for a ServerSocket to be more consistent here or if it is just an oversight. >> >> This code shows the behaviour. >> >> ServerSocket socket = new ServerSocket(0); >> >> socket.close(); >> if (socket.isBound()) { >> throw new AssertionError(); >> } >> >> >> >> Thanks, >> Norman >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Apr 11 16:01:53 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Apr 2019 17:01:53 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: Arthur, Daniel, > On 10 Apr 2019, at 21:39, Arthur Eubanks wrote: > > Here's a prototype webrev to see if the approach is okay with you. If it looks good, I'll continue with the remaining tests I can find. (should I start a new thread for the webrev?) > > http://cr.openjdk.java.net/~aeubanks/8220673/webrev.00 This is not too bad. Given the typical usage, one `@run` followed by a second `@run + system property`, then we kinda cannot easily use an `@requires`. Also, it would be good to dump/print the IP configuration, as determined by IPSupport, if the test will not be run. This will be useful when digging into test output jtr files. An alternative, rather than simple returning ( when the configuration does not support the test ) maybe throw jtreg.SkippedException [1], to indicate that the test has been skipped (rather than run successfully). An example of this is LevelTransitionTest [2]. That said, I would still like to experiment a little with @requires for the cases where there is a single @run tag. > On 11 Apr 2019, at 14:05, Daniel Fuchs wrote: > ... > We're putting a lot of trust in the implementation of this > IPSupport utility. Yes, this is a good point. What?s nice about this is that there is just one body of code that provides the functionality ( and it is all in Java, not native). I'm interested to see how this performs in Arthur's experiments, and I need to do a little more testing myself. It is important that a test leaves enough breadcrumbs behind if it decides that it will not run. -Chris [1] https://openjdk.java.net/jtreg/faq.html#what-if-a-test-does-not-apply-in-a-given-situation [2] https://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/compiler/tiered/LevelTransitionTest.java -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Thu Apr 11 16:15:25 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 11 Apr 2019 17:15:25 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: <8108bc3b-0298-1e38-6e40-91c4752beadd@oracle.com> On 11/04/2019 17:01, Chris Hegarty wrote: > Yes, this is a good point. What?s nice about this is that there is just > one body of code that provides the functionality ( and it is all in > Java, not native). I'm interested to see how this performs in Arthur's > experiments, and I need to do a little more testing myself. It is > important that a test leaves enough breadcrumbs behind if it decides > that it will not run. > Ah - so possibly we could have a higher level method in IPSupport: that would do something like: public static void ensureConfigurationIsValid() throws jtreg.SkippedException { if (!IPSupport.currentConfigurationIsValid()) { // log the configuration, provides diagnostic // as to why it is invalid ... String message = "invalid configuration for this test: " + ...; throws new SkippedException(message); } } Is that what you have in mind Chris? best regards, -- daniel From chris.hegarty at oracle.com Thu Apr 11 16:18:53 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Apr 2019 17:18:53 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: <8108bc3b-0298-1e38-6e40-91c4752beadd@oracle.com> References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> <8108bc3b-0298-1e38-6e40-91c4752beadd@oracle.com> Message-ID: <0A49AA84-964D-4C65-B077-64DF51AB3914@oracle.com> Daniel, > On 11 Apr 2019, at 17:15, Daniel Fuchs wrote: > > On 11/04/2019 17:01, Chris Hegarty wrote: >> Yes, this is a good point. What?s nice about this is that there is just >> one body of code that provides the functionality ( and it is all in >> Java, not native). I'm interested to see how this performs in Arthur's >> experiments, and I need to do a little more testing myself. It is >> important that a test leaves enough breadcrumbs behind if it decides >> that it will not run. > > Ah - so possibly we could have a higher level method in IPSupport: > that would do something like: > > public static void ensureConfigurationIsValid() > throws jtreg.SkippedException { > > if (!IPSupport.currentConfigurationIsValid()) { > // log the configuration, provides diagnostic > // as to why it is invalid > ... > String message = "invalid configuration for this test: " + ...; > throws new SkippedException(message); > } > > } > > Is that what you have in mind Chris? No, but that looks much better than what was in my head ;-) I like it Daniel, as its a one line change to the test source. -Chris. From michael.x.mcmahon at oracle.com Thu Apr 11 17:06:36 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 11 Apr 2019 18:06:36 +0100 Subject: AW: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: <5cae5a25.1c69fb81.113e.0304@mx.google.com> References: <5CA5C323.4020508@oracle.com> <5CA77E83.70606@oracle.com> <5CACC1D5.80507@oracle.com> <5cae5a25.1c69fb81.113e.0304@mx.google.com> Message-ID: <5CAF741C.1000802@oracle.com> On 10/04/2019, 22:03, Bernd Eckenfels wrote: > > Maybe instead of having a MacOS specific algorithm it would be a good > idea to have getLocalHostName generally look up the hostnames from the > Array with a trailing ?.? and only if that Fails have it search with > search Suffix? > Interesting idea. Though I think it doesn't change the fact that you are querying the name service for a name it won't resolve, and in this MDNS case, would end up waiting for 5 seconds for the result. > Also I think we talked before that removing AI_CANONNAME would also > help. (Especially since it does a reverse lookup itself anyway). > AI_CANONNAME doesn't do a reverse lookup. It just asks for the canonical name to be returned as part of the regular forward lookup, but which we don't use. This shouldn't make any difference to the lookup because if the name supplied is an alias, it still has to be resolved to get the IP address of the desired name whether you use the AI_CANONNAME flag or not. Typically, the result for the alias and the cname record is returned in the same response from the local DNS server. You're probably thinking of the InetAddress.getCanonicalHostName() method which does do a reverse lookup (via getnameinfo()) Michael > > *Von: *Michael McMahon > *Gesendet: *Dienstag, 9. April 2019 18:35 > *An: *Nora Howard > *Cc: *net-dev at openjdk.java.net > *Betreff: *Re: JDK-8170910: Avoid 5 second localhost lookup behavior > on OSX > > Hi, > > So, I reproduced the issue, albeit with one configuration change that > I'm not sure whether is common or not. > > Basically, I could see that lookups to any host with a ".local" suffix > would use MDNS and if the local services are not active, then I see > the packets being sent on the wire, no responses and a delay of 5 > seconds before the lookup fails. Initially, this was not affecting > localhost lookups which still use normal DNS and even when DNS not > available, that was failing immediately and therefore no delay. > > To make InetAddress.getLocalHost() fail with a 5 second delay, I had > to add ".local" to the DNS search path of the system. That has the > effect of attempting a MDNS lookup after the DNS lookup fails, and > this delays the failure for 5 seconds. > > One question is if this is a common network configuration, or is there > some other way to cause it to happen? > > Looking at the patch again, I think perhaps (assuming we want to make > a change here) a different approach might be better. Quite a lot of > work is happening in the lookupIfLocalhost() function and I think we > should try to limit this being called to only the local hostname > lookup path. This might require boolean flags being being passed down > from the InetAddress.getLocalHost() call to indicate whether this is a > local address lookup or not. > > Thanks, > Michael > > > On 05/04/2019, 20:15, Nora Howard wrote: > > Oh, another element is that to see the behavior, your local > hostname has to not have an entry in /etc/hosts. Adding it is one > of the workarounds. My /etc/hosts has a localhost entry that does > not include the name my machine ends up with after boot. > > I'm also on 10.13, so I don't think that's the issue per se. > > When I run scutil --dns. I get a resolver that points to en01 with > some names on my VPN, and a number of mdns registrations that all > look like this, where the resolver number, the domain and the > order change, but the rest of the fields stay the same. > > resolver #2 > domain : local > options : mdns > timeout : 5 > flags : Request A records > reach : 0x00000000 (Not Reachable) > order : 300000 > > On Fri, Apr 5, 2019 at 10:14 AM Michael McMahon > > wrote: > > I've spent some time trying to reproduce this on 10.13 but > have not been able. > I have disabled all the services as suggested (including some > additional) > such that the dns-sd command produces no output "records" and > also by > disabling unicast DNS access. > > I wonder if there has been a change in 10.13 to workaround > this problem > as I don't see any delay or external MDNS lookups happening. > . > Could you send me the output of "scutil --dns" of your laptop? > At the same time, I will ask some colleagues here who might be > running 10.12 > to try it out. > > Thanks, > Michael > > On 04/04/2019, 18:38, Nora Howard wrote: > > Here's the diff > http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ > > > To setup an env to reproduce the issue, on a OSX machine > with >= 10.12, turn off services that register with mDNS. > For my laptop, I had to turn off iTunes and General System > Preferences, disable "Allow Handoff between this Mac and > your iCloud services". You might have to restart. The > reporter on the ticket said to run `dns-sd -lo -B > _services._dns-sd._udp local` to see whether you have any > running. > > The file I used for exercising the behavior looks like this > > import java.net.InetAddress; > public class Test { > public static void main(String[] args) throws Exception { > for (int i=0;i<4;i++){ > long start = System.currentTimeMillis(); > InetAddress localHost = InetAddress.getLocalHost(); > System.out.println(localHost); > System.out.println(System.currentTimeMillis() - > start); > }}} > > The output I get on my laptop when I've set things up this > way looks like this on JDK 13 EA. The first call takes the > full 5 seconds and subsequent ones hit a cache. > > nhoward.local/10.0.0.3 > 5044 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > > On JDK 8, each request takes 5 seconds. JDK 8's address > lookup cache works differently. It's localhost name cache > expiry is 5 seconds. > > On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon > > wrote: > > Hi Nora, > > At first sight, the approach sounds reasonable to me. > I'd like to see the patch > and also do you have detailed instructions on how to > reproduce the issue (the 5 second delay)? > > Thanks, > Michael. > > On 03/04/2019, 17:46, Nora Howard wrote: > > I'd initially sent this to jdk-dev, but I was > asked to move it to net-dev. > > On OSX 10.12 or later, when no mDNS services are > registered, getaddrinfo fails on looking up > localhost addresses[1]. The jdk?s current > implementation falls back to getifaddrs after > getaddrinfo fails[2]. When getaddrinfo fails in > this way, it takes 5 seconds to fail [3]. This > means that calls to InetAddress.getLocalHost() may > take up to 5 seconds to run. > > I?d like to change the jdk?s behavior in this > instance so that instead of falling back to > calling getifaddrs after getaddrinfo fails, it > instead calls getifaddrs first if we?re on OSX and > the hostname is localhost?s. > > Doing this will eliminate the 5 second delay > looking up localhost on machines with this network > setup. When looking up non-localhost addresses, > it?ll add a call to os:gethostname and a comparison. > > I?d like to get some feedback about whether this > approach sounds reasonable, and some guidance > about next steps. I?ve read the ?how to > contribute? page[4], and the ?code review? guide[5]. > > I?ve put together a patch that I?ve tested locally > by turning off mDNS services on my local OSX > machine and exercising the behavior. I?d like to > submit patches that target dev trunk as well as > the jdk 8 and 11 branches. > > > References > > 1. > > On a Macintosh running 10.12 or later, if > there are no apps that have a mDNS services > registered (for instance, iTunes) and there > are no services selected in the Sharing System > Preferences, then the getaddrinfo will fail. > > (https://bugs.openjdk.java.net/browse/JDK-8170910) > > 2. > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 > (The lookupIfLocalhost function checks the passed > hostname against os::get_host_name, and if they > match, looks up via getifaddrs.) > > 3. This comment describes how to reproduce the > issue. I?ve reproduced it locally on jdk 8, 11 and > 13 ea. > https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 > > > 4. https://openjdk.java.net/contribute/ > > 5. https://openjdk.java.net/guide/codeReview.html > > Thanks, > > -- > > Nora > > > > -- > > Nora > > > > -- > > Nora > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at oracle.com Thu Apr 11 18:42:16 2019 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 11 Apr 2019 14:42:16 -0400 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: Hi, If you need different @requires for different runs, there is a jtreg feature to define separate tests using separate comment blocks. /*... ?*@requires x ?*@run ... ?*/ /* ?* @requires !x ?* @run ... ?*/ $.02, Roger On 04/11/2019 12:01 PM, Chris Hegarty wrote: > Arthur, Daniel, > >> On 10 Apr 2019, at 21:39, Arthur Eubanks > > wrote: >> >> Here's a prototype webrev to see if the approach is okay with you. If >> it looks good, I'll continue with the remaining tests I can find. >> (should I start a new thread for the webrev?) >> >> http://cr.openjdk.java.net/~aeubanks/8220673/webrev.00 >> > > This is not too bad. Given the typical usage, one `@run` followed by a > second `@run + system property`, then we kinda cannot easily use an > `@requires`. Also, it would be good to dump/print the IP configuration, > as determined by IPSupport, if the test will not be run. This will be > useful when digging into test output jtr files. > > An alternative, rather than simple returning ( when the configuration > does not support the test ) maybe throw jtreg.SkippedException [1], to > indicate that the test has been skipped (rather than run successfully). > An example of this is LevelTransitionTest [2]. > > That said, I would still like to experiment a little with @requires for > the cases where there is a single @run tag. > > >> On 11 Apr 2019, at 14:05, Daniel Fuchs > > wrote: >> ... >> We're putting a lot of trust in the implementation of this >> IPSupport utility. > > Yes, this is a good point. What?s nice about this is that there is just > one body of code that provides the functionality ( and it is all in > Java, not native). I'm interested to see how this performs in Arthur's > experiments, and I need to do a little more testing myself. It is > important that a test leaves enough breadcrumbs behind if it decides > that it will not run. > > -Chris > > [1] > https://openjdk.java.net/jtreg/faq.html#what-if-a-test-does-not-apply-in-a-given-situation > [2] > https://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/compiler/tiered/LevelTransitionTest.java From nhoward at twitter.com Thu Apr 11 19:17:38 2019 From: nhoward at twitter.com (Nora Howard) Date: Thu, 11 Apr 2019 13:17:38 -0600 Subject: AW: JDK-8170910: Avoid 5 second localhost lookup behavior on OSX In-Reply-To: <5CAF741C.1000802@oracle.com> References: <5CA5C323.4020508@oracle.com> <5CA77E83.70606@oracle.com> <5CACC1D5.80507@oracle.com> <5cae5a25.1c69fb81.113e.0304@mx.google.com> <5CAF741C.1000802@oracle.com> Message-ID: > > One question is if this is a common network configuration, or is there > some other way to cause it to happen? > It's common enough that when I looked up "lookupAllHostAddr OSX", there's a fair number of blog posts and stackoverflow posts about the issue that point to the /etc/hosts workaround. Looking at the patch again, I think perhaps (assuming we want to make a > change here) a different approach might be better. Quite a lot of work is > happening in the lookupIfLocalhost() function and I think we should try to > limit this being called to only the local hostname lookup path. This might > require boolean flags being being passed down from the > InetAddress.getLocalHost() call to indicate whether this is a local address > lookup or not. I considered creating a new code path or passing a flag, but I wasn't sure whether the additional code complexity of introducing either of those was worth it or not. Also, adding a new flag wouldn't help if there are cases where the addr lookup is called with the local host name but not through the getLocalHost(). I'm not sure how common that case is. One of my concerns when I looked through what adding a flag would entail, was that in order to pass a flag down, I think I'd need to add knowledge of the flag to a number of the classes and interfaces that collaborate with InetAddress. On Thu, Apr 11, 2019 at 11:19 AM Michael McMahon < michael.x.mcmahon at oracle.com> wrote: > > > On 10/04/2019, 22:03, Bernd Eckenfels wrote: > > Maybe instead of having a MacOS specific algorithm it would be a good idea > to have getLocalHostName generally look up the hostnames from the Array > with a trailing ?.? and only if that Fails have it search with search > Suffix? > > > > > Interesting idea. Though I think it doesn't change the fact that you are > querying the name service for > a name it won't resolve, and in this MDNS case, would end up waiting for 5 > seconds for the result. > > Also I think we talked before that removing AI_CANONNAME would also help. > (Especially since it does a reverse lookup itself anyway). > > > AI_CANONNAME doesn't do a reverse lookup. It just asks for the canonical > name to be returned as part of the > regular forward lookup, but which we don't use. > > This shouldn't make any difference to the lookup because if the name > supplied is an alias, it still has to > be resolved to get the IP address of the desired name whether you use the > AI_CANONNAME flag or not. > Typically, the result for the alias and the cname record is returned in > the same response from the local DNS server. > > You're probably thinking of the InetAddress.getCanonicalHostName() method > which does do a reverse lookup > (via getnameinfo()) > > Michael > > > > > *Von: *Michael McMahon > *Gesendet: *Dienstag, 9. April 2019 18:35 > *An: *Nora Howard > *Cc: *net-dev at openjdk.java.net > *Betreff: *Re: JDK-8170910: Avoid 5 second localhost lookup behavior on > OSX > > > > Hi, > > So, I reproduced the issue, albeit with one configuration change that I'm > not sure whether is common or not. > > Basically, I could see that lookups to any host with a ".local" suffix > would use MDNS and if the local services are not active, then I see the > packets being sent on the wire, no responses and a delay of 5 seconds > before the lookup fails. Initially, this was not affecting localhost > lookups which still use normal DNS and even when DNS not available, that > was failing immediately and therefore no delay. > > To make InetAddress.getLocalHost() fail with a 5 second delay, I had to > add ".local" to the DNS search path of the system. That has the effect of > attempting a MDNS lookup after the DNS lookup fails, and this delays the > failure for 5 seconds. > > One question is if this is a common network configuration, or is there > some other way to cause it to happen? > > Looking at the patch again, I think perhaps (assuming we want to make a > change here) a different approach might be better. Quite a lot of work is > happening in the lookupIfLocalhost() function and I think we should try to > limit this being called to only the local hostname lookup path. This might > require boolean flags being being passed down from the > InetAddress.getLocalHost() call to indicate whether this is a local address > lookup or not. > > Thanks, > Michael > > > On 05/04/2019, 20:15, Nora Howard wrote: > > Oh, another element is that to see the behavior, your local hostname has > to not have an entry in /etc/hosts. Adding it is one of the workarounds. My > /etc/hosts has a localhost entry that does not include the name my machine > ends up with after boot. > > > > I'm also on 10.13, so I don't think that's the issue per se. > > > > When I run scutil --dns. I get a resolver that points to en01 with some > names on my VPN, and a number of mdns registrations that all look like > this, where the resolver number, the domain and the order change, but the > rest of the fields stay the same. > > resolver #2 > domain : local > options : mdns > timeout : 5 > flags : Request A records > reach : 0x00000000 (Not Reachable) > order : 300000 > > > > > > > > On Fri, Apr 5, 2019 at 10:14 AM Michael McMahon < > michael.x.mcmahon at oracle.com> wrote: > > I've spent some time trying to reproduce this on 10.13 but have not been > able. > I have disabled all the services as suggested (including some additional) > such that the dns-sd command produces no output "records" and also by > disabling unicast DNS access. > > I wonder if there has been a change in 10.13 to workaround this problem > as I don't see any delay or external MDNS lookups happening. > . > Could you send me the output of "scutil --dns" of your laptop? > At the same time, I will ask some colleagues here who might be running > 10.12 > to try it out. > > Thanks, > Michael > > On 04/04/2019, 18:38, Nora Howard wrote: > > Here's the diff http://cr.openjdk.java.net/~tonyp/8170910/webrev.0/ > > > > To setup an env to reproduce the issue, on a OSX machine with >= 10.12, > turn off services that register with mDNS. For my laptop, I had to turn off > iTunes and General System Preferences, disable "Allow Handoff between this > Mac and your iCloud services". You might have to restart. The reporter on > the ticket said to run `dns-sd -lo -B _services._dns-sd._udp local` to see > whether you have any running. > > > > The file I used for exercising the behavior looks like this > > > > import java.net.InetAddress; > public class Test { > public static void main(String[] args) throws Exception { > for (int i=0;i<4;i++){ > long start = System.currentTimeMillis(); > InetAddress localHost = InetAddress.getLocalHost(); > System.out.println(localHost); > System.out.println(System.currentTimeMillis() - start); > }}} > > > > The output I get on my laptop when I've set things up this way looks like > this on JDK 13 EA. The first call takes the full 5 seconds and subsequent > ones hit a cache. > > > > nhoward.local/10.0.0.3 > 5044 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > nhoward.local/10.0.0.3 > 0 > > > > On JDK 8, each request takes 5 seconds. JDK 8's address lookup cache works > differently. It's localhost name cache expiry is 5 seconds. > > > > > > > > On Thu, Apr 4, 2019 at 2:41 AM Michael McMahon < > michael.x.mcmahon at oracle.com> wrote: > > Hi Nora, > > At first sight, the approach sounds reasonable to me. I'd like to see the > patch > and also do you have detailed instructions on how to reproduce the issue > (the 5 second delay)? > > Thanks, > Michael. > > On 03/04/2019, 17:46, Nora Howard wrote: > > I'd initially sent this to jdk-dev, but I was asked to move it to net-dev. > > > > On OSX 10.12 or later, when no mDNS services are registered, getaddrinfo > fails on looking up localhost addresses[1]. The jdk?s current > implementation falls back to getifaddrs after getaddrinfo fails[2]. When > getaddrinfo fails in this way, it takes 5 seconds to fail [3]. This means > that calls to InetAddress.getLocalHost() may take up to 5 seconds to run. > > I?d like to change the jdk?s behavior in this instance so that instead of > falling back to calling getifaddrs after getaddrinfo fails, it instead > calls getifaddrs first if we?re on OSX and the hostname is localhost?s. > > Doing this will eliminate the 5 second delay looking up localhost on > machines with this network setup. When looking up non-localhost addresses, > it?ll add a call to os:gethostname and a comparison. > > I?d like to get some feedback about whether this approach sounds > reasonable, and some guidance about next steps. I?ve read the ?how to > contribute? page[4], and the ?code review? guide[5]. > > I?ve put together a patch that I?ve tested locally by turning off mDNS > services on my local OSX machine and exercising the behavior. I?d like to > submit patches that target dev trunk as well as the jdk 8 and 11 branches. > > > References > > 1. > > On a Macintosh running 10.12 or later, if there are no apps that have a > mDNS services registered (for instance, iTunes) and there are no services > selected in the Sharing System Preferences, then the getaddrinfo will fail. > > (https://bugs.openjdk.java.net/browse/JDK-8170910) > > > > 2. > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet6AddressImpl.c#l252 > http://hg.openjdk.java.net/jdk/jdk/file/cd3b7ad53265/src/java.base/unix/native/libnet/Inet4AddressImpl.c#l132 > (The lookupIfLocalhost function checks the passed hostname against > os::get_host_name, and if they match, looks up via getifaddrs.) > > > > 3. This comment describes how to reproduce the issue. I?ve reproduced it > locally on jdk 8, 11 and 13 ea. > https://bugs.openjdk.java.net/browse/JDK-8170910?focusedCommentId=14038262&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14038262 > > > > 4. https://openjdk.java.net/contribute/ > > > > 5. https://openjdk.java.net/guide/codeReview.html > > Thanks, > > -- > > Nora > > > > -- > > Nora > > > > -- > > Nora > > > > -- Nora -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Apr 12 00:18:19 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 11 Apr 2019 17:18:19 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> Message-ID: <1a836ba1-f8b9-7c0e-a8fb-1e6da25b5046@oracle.com> Hi Alex, Great debugging feature! While I'm still reading all the details, could you, please, fix some minor format issues? http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html + * If host is a literal IPv6 address, it may be in square brackets. Extra space before "square". http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html ?I'd suggest to unify comments before functions: ? - start comment with a capital latter and ended with a dot ? - use comment format like this: ?? /* ??? */ ?Examples of comments that need this change: 262 /* result must be release with dbgsysFreeAddrInfo */ => /* * Result must be release with dbgsysFreeAddrInfo. ?*/ 325 // input is sockaddr just because all clients have it => /* * Input is sockaddr just because all clients have it. */ 1129 /* reads boolean system value, 1130 * sets *result to trueValue if the ptoperty is "true", 1131 * to falseValue if the property if "false", 1132 * doesn't change *result if the property is not set or failed to read. 1133 */ => /* * Read boolean system value andset result to: * - trueValue if the property is "true" * - falseValue if the property is "false" * * Return JNI_OK if result is set, return JNI_ERR otherwise. */ . . . 293 * use IPv6 socket (to accept IPv6 and mapped Ipv4),... 342 * (with AF_INET6 Ipv4 addresses are not parsed even with AI_V4MAPPED and AI_ALL flags) ...345 hints.ai_family = AF_UNSPEC; // IPv6 or mapped Ipv4 ... 360 } else { // Ipv4 address Replace Ipv4 with IPv4 for unification with IPv6 For unification replace: convertIpv4ToIpv6 => convertIPv4ToIPv6 297 hints.ai_flags |= AI_PASSIVE 298 | (allowOnlyIPv4 ? 0 : AI_V4MAPPED | AI_ALL); Better to have just one line 1135 JNIEnv* env, . . . 1165 JNIEnv* jniEnv = NULL; A suggestion is to use the same name for JNIEnv*: ? 1135 JNIEnv* jni, . . . ? 1165 JNIEnv* jni = NULL; ? Reformat: 608 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 609 || (pass == 1 && ai->ai_family != preferredAddressFamily)) and 828 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 829 || (pass == 1 && ai->ai_family != preferredAddressFamily)) => if ((pass == 0 && ai->ai_family == preferredAddressFamily) || (pass == 1 && ai->ai_family != preferredAddressFamily)) ? Even better, replace it with logical XOR: if ((pass == 0 ^^ ai->ai_family == preferredAddressFamily) http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/ListeningConnector/startListening/startlis001.java.frames.html 102 /* Check that listening address returned by ListeningConnector.startListening() 103 * matches the address which was set via connector's arguments. 104 * Empty host address causes listening for local connections only (loopback interface) 105 * */ Dot is missed at the end. Replace "* */" with "*/". http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/jdk/com/sun/jdi/JdwpAllowTest.java.frames.html 162 // generate allow address by changing random bit in the local address 163 // and calculate 2 masks (prefix length) - one is matches original local address 164 // and another doesn't. Replace with /* style of comment. 249 positiveTest("PositiveMaskTest(" + test.localAddress + ")", test.allowAddress + "/" + test.prefixLengthGood); 250 positiveTest("NegativeMaskTest(" + test.localAddress + ")", test.allowAddress + "/" + test.prefixLengthBad); ? A suggestion to move second argument to additional line: positiveTest("PositiveMaskTest(" + test.localAddress + ")", test.allowAddress + "/" + test.prefixLengthGood); positiveTest("NegativeMaskTest(" + test.localAddress + ")", test.allowAddress + "/" + test.prefixLengthBad); Thanks, Serguei On 4/2/19 4:14 PM, Alex Menkov wrote: > Updated webrev: > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/ > > - added support for addresses enclosed in square brackets; > - updated SocketTransportService.java to handle empty hostname the > same way as JDWP agent (listen/attach to loopback address); > Has to update nsk/jdi/ListeningConnector/startListening/startlis001.java > (all other com/sun/jdi, nsk/jdi, nsk/jdwp, nsk/jdb test are passed). > > --alex > > On 04/01/2019 11:21, Alex Menkov wrote: >> Hi Chris, >> >> On 04/01/2019 04:50, Chris Hegarty wrote: >>> Alex, >>> >>> On 29/03/2019 22:07, Alex Menkov wrote: >>>> (added net-dev as suggested by Alan) >>>> >>>> Net gurus, please assist in reviewing socket-related code. >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ >>> >>> Specifically on SocketTransportService.java. What Arthur has >>> proposed is better ( changing to lastIndexOf alone is not >>> sufficient ). Or is your assumption that the IPv6 literal is >>> not enclosed in square brackets? >> >> I didn't know about enclosing IPv6 in square brackets, but looks like >> that's standard way to alleviate conflict between IPv6 address and >> colon as port separator. >> Will update the fix to handle them in both JDI connectors >> (SocketTransportService.java) and debugger agent (socketTransport.c) >> >>> >>> If keeping Arthur's `static class HostPort` please make the >>> fields final. >>> >>> ?>> Would it make sense to support the preference properties? >>> ?>>??? java.net.preferIPv4Stack >>> ?>>??? java.net.preferIPv6Addresses >>> >>> I'm not sure about this, especially given the property name >>> prefixes. I need to think a little more on it. >> >> In the initial version of the fix I didn't check the properties. >> The rationale here is backward compatibility - is address is empty, >> old debuggers tries to connect to IPv4 address only. >> But handling this properties we will better handle clients with >> properties set (as jdb or any other debugger which uses JDI >> connectors are affected by the properties). >> >> BTW fix provided by Arthur implements listening on localhost >> differently - it creates several sockets and binds them to both IPv4 >> and IPv6 addresses. But the problem here (and that's the reason I >> decide to not implement it this way) - how to handle the case when we >> successfully bind on one address (for example IPv4), but fail to bind >> on other (for example the port is busy for IPv6 stack). Arthur's >> version just fail in the case (i.e. the whole Java process >> terminates) and I don't think this is good behavior. >> >> >>> >>> There is quite a bit of new native code, is it possible to >>> rewrite any of this in Java, e.g. reading of the system >>> properties ( if that is to remain )? >> >> socketTransport.c is a debugger agent which is completely native. >> >> --alex >> >>> >>> -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Fri Apr 12 08:44:51 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 12 Apr 2019 09:44:51 +0100 Subject: ServerSocket.isBound() continue to return true after close() is called. In-Reply-To: <8A9BC985-E2A4-45B6-B2D6-D648E7B409F0@googlemail.com> References: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> <5CAF5963.3020106@oracle.com> <8A9BC985-E2A4-45B6-B2D6-D648E7B409F0@googlemail.com> Message-ID: The following JIRA issue has been filed to track this: https://bugs.openjdk.java.net/browse/JDK-8222363 -Chris. P.S. my bad! I missed this case when working on 6505016 ( in 2007! ) > On 11 Apr 2019, at 16:27, Norman Maurer wrote: > > Ok thanks? update to the java docs sounds good to me. I was just suprised by the behaviour :) > >> On 11. Apr 2019, at 17:12, Michael McMahon > wrote: >> >> Norman >> >> The specification on what happens to all socket types was updated many years ago >> in bug id 6505016, but it looks like ServerSocket::isBound was missed from that effort. >> I think we should probably update the spec to reflect current behavior and be consistent >> with the change above. There will be some small spec updates to ServerSocket coming >> which originated from the SocketImpl replacement work that Alan mentioned recently >> and I think we can include this small change probably with one of those. >> >> Michael. >> >> On 11/04/2019, 13:40, Norman Maurer wrote: >>> >>> Hi there, >>> >>> While working on netty I just noticed that a ServerSocket will keep return true when isBound() is called even after it was closed. Is this by design? I was bit surprised by this honestly as after the socket is closed there is no way it will accept any more connections. -------------- next part -------------- An HTML attachment was scrubbed... URL: From norman.maurer at googlemail.com Fri Apr 12 09:22:20 2019 From: norman.maurer at googlemail.com (Norman Maurer) Date: Fri, 12 Apr 2019 11:22:20 +0200 Subject: ServerSocket.isBound() continue to return true after close() is called. In-Reply-To: References: <624DB82A-0483-4B19-9322-F8FFA8C08C46@googlemail.com> <5CAF5963.3020106@oracle.com> <8A9BC985-E2A4-45B6-B2D6-D648E7B409F0@googlemail.com> Message-ID: No worries. Thanks a lot! > On 12. Apr 2019, at 10:44, Chris Hegarty wrote: > > The following JIRA issue has been filed to track this: > https://bugs.openjdk.java.net/browse/JDK-8222363 > > -Chris. > > P.S. my bad! I missed this case when working on 6505016 ( in 2007! ) > >> On 11 Apr 2019, at 16:27, Norman Maurer > wrote: >> >> Ok thanks? update to the java docs sounds good to me. I was just suprised by the behaviour :) >> >>> On 11. Apr 2019, at 17:12, Michael McMahon > wrote: >>> >>> Norman >>> >>> The specification on what happens to all socket types was updated many years ago >>> in bug id 6505016, but it looks like ServerSocket::isBound was missed from that effort. >>> I think we should probably update the spec to reflect current behavior and be consistent >>> with the change above. There will be some small spec updates to ServerSocket coming >>> which originated from the SocketImpl replacement work that Alan mentioned recently >>> and I think we can include this small change probably with one of those. >>> >>> Michael. >>> >>> On 11/04/2019, 13:40, Norman Maurer wrote: >>>> >>>> Hi there, >>>> >>>> While working on netty I just noticed that a ServerSocket will keep return true when isBound() is called even after it was closed. Is this by design? I was bit surprised by this honestly as after the socket is closed there is no way it will accept any more connections. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Fri Apr 12 23:58:17 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 12 Apr 2019 16:58:17 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: <1a836ba1-f8b9-7c0e-a8fb-1e6da25b5046@oracle.com> References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> <1a836ba1-f8b9-7c0e-a8fb-1e6da25b5046@oracle.com> Message-ID: updated webrev: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/ changes (vs webrev.02) are non-functional (added/edited comments, code reformatting, renaming convertIpv4ToIpv6 function to convertIPv4ToIPv6, renaming env variable to jniEnv (env is already used in one of the functions)). About pass/preferredAddressFamily conditions - there is no "logical xor" in C/C++. Also I think that the current condition is clearer. --alex On 04/11/2019 17:18, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > Great debugging feature! > While I'm still reading all the details, could you, please, fix some > minor format issues? > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html > > + * If host is a literal IPv6 address, it may be in square > brackets. Extra space before "square". > > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html > > ?I'd suggest to unify comments before functions: > ? - start comment with a capital latter and ended with a dot > ? - use comment format like this: > ?? /* > ??? */ > > ?Examples of comments that need this change: > > 262 /* result must be release with dbgsysFreeAddrInfo */ => /* * Result > must be release with dbgsysFreeAddrInfo. ?*/ > > 325 // input is sockaddr just because all clients have it => > /* * Input is sockaddr just because all clients have it. */ > > 1129 /* reads boolean system value, > 1130 * sets *result to trueValue if the ptoperty is "true", > 1131 * to falseValue if the property if "false", > 1132 * doesn't change *result if the property is not set or failed to read. > 1133 */ => /* * Read boolean system value andset result to: * - > trueValue if the property is "true" > * - falseValue if the property is "false" * * Return JNI_OK if result is set, return JNI_ERR otherwise. > */ > > . . . > > 293 * use IPv6 socket (to accept IPv6 and mapped Ipv4),... > 342 * (with AF_INET6 Ipv4 addresses are not parsed even with AI_V4MAPPED > and AI_ALL flags) ...345 hints.ai_family = AF_UNSPEC; // IPv6 or mapped > Ipv4 ... 360 } else { // Ipv4 address Replace Ipv4 with IPv4 for > unification with IPv6 > For unification replace: convertIpv4ToIpv6 => convertIPv4ToIPv6 > > 297 hints.ai_flags |= AI_PASSIVE > 298 | (allowOnlyIPv4 ? 0 : AI_V4MAPPED | AI_ALL); Better to have just > one line > > > 1135 JNIEnv* env, . . . 1165 JNIEnv* jniEnv = NULL; > > A suggestion is to use the same name for JNIEnv*: > > ? 1135 JNIEnv* jni, . . . > ? 1165 JNIEnv* jni = NULL; > > > ? Reformat: > > 608 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 609 || > (pass == 1 && ai->ai_family != preferredAddressFamily)) and > > 828 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 829 || > (pass == 1 && ai->ai_family != preferredAddressFamily)) => if ((pass == > 0 && ai->ai_family == preferredAddressFamily) || (pass == 1 && > ai->ai_family != preferredAddressFamily)) > > ? Even better, replace it with logical XOR: > > if ((pass == 0 ^^ ai->ai_family == preferredAddressFamily) > > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/ListeningConnector/startListening/startlis001.java.frames.html > > 102 /* Check that listening address returned by > ListeningConnector.startListening() > 103 * matches the address which was set via connector's arguments. > 104 * Empty host address causes listening for local connections only > (loopback interface) > 105 * */ Dot is missed at the end. Replace "* */" with "*/". > > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/jdk/com/sun/jdi/JdwpAllowTest.java.frames.html > > 162 // generate allow address by changing random bit in the local address > 163 // and calculate 2 masks (prefix length) - one is matches original > local address > 164 // and another doesn't. Replace with /* style of comment. > > > 249 positiveTest("PositiveMaskTest(" + test.localAddress + ")", > test.allowAddress + "/" + test.prefixLengthGood); > 250 positiveTest("NegativeMaskTest(" + test.localAddress + ")", > test.allowAddress + "/" + test.prefixLengthBad); > > ? A suggestion to move second argument to additional line: > > positiveTest("PositiveMaskTest(" + test.localAddress + ")", > test.allowAddress + "/" + test.prefixLengthGood); > positiveTest("NegativeMaskTest(" + test.localAddress + ")", > test.allowAddress + "/" + test.prefixLengthBad); > > Thanks, > Serguei > > > On 4/2/19 4:14 PM, Alex Menkov wrote: >> Updated webrev: >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/ >> >> - added support for addresses enclosed in square brackets; >> - updated SocketTransportService.java to handle empty hostname the >> same way as JDWP agent (listen/attach to loopback address); >> Has to update nsk/jdi/ListeningConnector/startListening/startlis001.java >> (all other com/sun/jdi, nsk/jdi, nsk/jdwp, nsk/jdb test are passed). >> >> --alex >> >> On 04/01/2019 11:21, Alex Menkov wrote: >>> Hi Chris, >>> >>> On 04/01/2019 04:50, Chris Hegarty wrote: >>>> Alex, >>>> >>>> On 29/03/2019 22:07, Alex Menkov wrote: >>>>> (added net-dev as suggested by Alan) >>>>> >>>>> Net gurus, please assist in reviewing socket-related code. >>>>> >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ >>>> >>>> Specifically on SocketTransportService.java. What Arthur has >>>> proposed is better ( changing to lastIndexOf alone is not >>>> sufficient ). Or is your assumption that the IPv6 literal is >>>> not enclosed in square brackets? >>> >>> I didn't know about enclosing IPv6 in square brackets, but looks like >>> that's standard way to alleviate conflict between IPv6 address and >>> colon as port separator. >>> Will update the fix to handle them in both JDI connectors >>> (SocketTransportService.java) and debugger agent (socketTransport.c) >>> >>>> >>>> If keeping Arthur's `static class HostPort` please make the >>>> fields final. >>>> >>>> ?>> Would it make sense to support the preference properties? >>>> ?>>??? java.net.preferIPv4Stack >>>> ?>>??? java.net.preferIPv6Addresses >>>> >>>> I'm not sure about this, especially given the property name >>>> prefixes. I need to think a little more on it. >>> >>> In the initial version of the fix I didn't check the properties. >>> The rationale here is backward compatibility - is address is empty, >>> old debuggers tries to connect to IPv4 address only. >>> But handling this properties we will better handle clients with >>> properties set (as jdb or any other debugger which uses JDI >>> connectors are affected by the properties). >>> >>> BTW fix provided by Arthur implements listening on localhost >>> differently - it creates several sockets and binds them to both IPv4 >>> and IPv6 addresses. But the problem here (and that's the reason I >>> decide to not implement it this way) - how to handle the case when we >>> successfully bind on one address (for example IPv4), but fail to bind >>> on other (for example the port is busy for IPv6 stack). Arthur's >>> version just fail in the case (i.e. the whole Java process >>> terminates) and I don't think this is good behavior. >>> >>> >>>> >>>> There is quite a bit of new native code, is it possible to >>>> rewrite any of this in Java, e.g. reading of the system >>>> properties ( if that is to remain )? >>> >>> socketTransport.c is a debugger agent which is completely native. >>> >>> --alex >>> >>>> >>>> -Chris. > From serguei.spitsyn at oracle.com Sat Apr 13 01:52:21 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 12 Apr 2019 18:52:21 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> <1a836ba1-f8b9-7c0e-a8fb-1e6da25b5046@oracle.com> Message-ID: <6dedcf93-6afe-8ea9-f0dd-f8da8825910a@oracle.com> Hi Alex, Thank you for the update! One more round of minor formatting change requests for better readability. :) #1: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html +??????????????? port = Integer.decode(hostPort.substring(splitIndex+1)); . . . +??????????? } else if (hostPort.charAt(0) == ?[? && hostPort.charAt(splitIndex-1) == ?]?) { ?Need spaces around ?+? and ?-? signs. #2: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdwp.agent/windows/native/libdt_socket/socket_md.c.udiff.html +??????? //make the socket a dual mode socket ?missed space at the start of comment Now, comments for: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html #3: 276???? /* check for host:port or port */ 277???? colon = strrchr(address, ?:?); 278???? port = (colon == NULL ? address : colon + 1); 279???? /* ensure the port is valid (getaddrinfo allows port to be empty) */ 280???? if (getPortNumber(port) < 0) { #4: 298???????? hints.ai_family = allowOnlyIPv4 ? AF_INET : AF_INET6; 299???????? hints.ai_flags |= AI_PASSIVE | (allowOnlyIPv4 ? 0 : AI_V4MAPPED | AI_ALL); 300???????? 301???? } else { #5: Replace ?fills? with ?fills in? in: 341? * Parses address (IPv4 or IPv6), fills result by parsed address. 383? * Parses prefix length from buffer (integer value), fills result with corresponding net mask. 485? * Parses ?allow? argument (fills list of allowed peers (global _peers variable)). #6: 410???? // generate mask for prefix length 411???? memset(result, 0, sizeof(*result)); 412???? // prefixLen <= 128, so we won?t go over result?s size 413???? for (int i = 0; prefixLen > 0; i++, prefixLen -= :sunglasses: { #7: 623 socketTransport_startListening(jdwpTransportEnv* env, const char* address, 624??????????????????????????????? char** actualAddress) 625 { . . . . 1173 static int readBooleanSysProp(int *result, int trueValue, int falseValue, 1174???? JNIEnv* jniEnv, jclass sysClass, jmethodID getPropMethod, const char *propName) 1175 { ?Move ?{? to the end of 624 and 1174. (edited) #8: 1176???? jstring value; 1177???? jstring name = (*jniEnv)->NewStringUTF(jniEnv, propName); 1178???? if (name == NULL) { . . . . 1259???? } while (0); 1260???? if (jniEnv != NULL && (*jniEnv)->ExceptionCheck(jniEnv)) { Thanks! Serguei On 4/12/19 4:58 PM, Alex Menkov wrote: > updated webrev: > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/ > > changes (vs webrev.02) are non-functional (added/edited comments, code > reformatting, renaming convertIpv4ToIpv6 function to > convertIPv4ToIPv6, renaming env variable to jniEnv (env is already > used in one of the functions)). > > About pass/preferredAddressFamily conditions - there is no "logical > xor" in C/C++. Also I think that the current condition is clearer. > > --alex > > On 04/11/2019 17:18, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> Great debugging feature! >> While I'm still reading all the details, could you, please, fix some >> minor format issues? >> >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html >> >> >> + * If host is a literal IPv6 address, it may be in >> square brackets. Extra space before "square". >> >> >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html >> >> >> ??I'd suggest to unify comments before functions: >> ?? - start comment with a capital latter and ended with a dot >> ?? - use comment format like this: >> ??? /* >> ???? */ >> >> ??Examples of comments that need this change: >> >> 262 /* result must be release with dbgsysFreeAddrInfo */ => /* * >> Result must be release with dbgsysFreeAddrInfo. ?*/ >> >> 325 // input is sockaddr just because all clients have it => >> /* * Input is sockaddr just because all clients have it. */ >> >> 1129 /* reads boolean system value, >> 1130 * sets *result to trueValue if the ptoperty is "true", >> 1131 * to falseValue if the property if "false", >> 1132 * doesn't change *result if the property is not set or failed to >> read. >> 1133 */ => /* * Read boolean system value andset result to: * - >> trueValue if the property is "true" >> * - falseValue if the property is "false" *?? * Return JNI_OK if >> result is set, return JNI_ERR? otherwise. >> */ >> >> ? . . . >> >> 293 * use IPv6 socket (to accept IPv6 and mapped Ipv4),... >> 342 * (with AF_INET6 Ipv4 addresses are not parsed even with >> AI_V4MAPPED and AI_ALL flags) ...345 hints.ai_family = AF_UNSPEC; // >> IPv6 or mapped Ipv4 ... 360 } else { // Ipv4 address Replace Ipv4 >> with IPv4 for unification with IPv6 >> For unification replace: convertIpv4ToIpv6 => convertIPv4ToIPv6 >> >> 297 hints.ai_flags |= AI_PASSIVE >> 298 | (allowOnlyIPv4 ? 0 : AI_V4MAPPED | AI_ALL); Better to have just >> one line >> >> >> 1135 JNIEnv* env, . . . 1165 JNIEnv* jniEnv = NULL; >> >> A suggestion is to use the same name for JNIEnv*: >> >> ?? 1135 JNIEnv* jni, . . . >> ?? 1165 JNIEnv* jni = NULL; >> >> >> ?? Reformat: >> >> 608 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 609 || >> (pass == 1 && ai->ai_family != preferredAddressFamily)) and >> >> 828 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 829 || >> (pass == 1 && ai->ai_family != preferredAddressFamily)) => if ((pass >> == 0 && ai->ai_family == preferredAddressFamily) || (pass == 1 && >> ai->ai_family != preferredAddressFamily)) >> >> ?? Even better, replace it with logical XOR: >> >> ???????????? if ((pass == 0 ^^ ai->ai_family == preferredAddressFamily) >> >> >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/ListeningConnector/startListening/startlis001.java.frames.html >> >> >> 102 /* Check that listening address returned by >> ListeningConnector.startListening() >> 103 * matches the address which was set via connector's arguments. >> 104 * Empty host address causes listening for local connections only >> (loopback interface) >> 105 * */ Dot is missed at the end. Replace "* */" with "*/". >> >> >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/jdk/com/sun/jdi/JdwpAllowTest.java.frames.html >> >> >> 162 // generate allow address by changing random bit in the local >> address >> 163 // and calculate 2 masks (prefix length) - one is matches >> original local address >> 164 // and another doesn't. Replace with /* style of comment. >> >> >> 249 positiveTest("PositiveMaskTest(" + test.localAddress + ")", >> test.allowAddress + "/" + test.prefixLengthGood); >> 250 positiveTest("NegativeMaskTest(" + test.localAddress + ")", >> test.allowAddress + "/" + test.prefixLengthBad); >> >> ?? A suggestion to move second argument to additional line: >> >> positiveTest("PositiveMaskTest(" + test.localAddress + ")", >> test.allowAddress + "/" + test.prefixLengthGood); >> positiveTest("NegativeMaskTest(" + test.localAddress + ")", >> test.allowAddress + "/" + test.prefixLengthBad); >> >> Thanks, >> Serguei >> >> >> On 4/2/19 4:14 PM, Alex Menkov wrote: >>> Updated webrev: >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/ >>> >>> - added support for addresses enclosed in square brackets; >>> - updated SocketTransportService.java to handle empty hostname the >>> same way as JDWP agent (listen/attach to loopback address); >>> Has to update >>> nsk/jdi/ListeningConnector/startListening/startlis001.java >>> (all other com/sun/jdi, nsk/jdi, nsk/jdwp, nsk/jdb test are passed). >>> >>> --alex >>> >>> On 04/01/2019 11:21, Alex Menkov wrote: >>>> Hi Chris, >>>> >>>> On 04/01/2019 04:50, Chris Hegarty wrote: >>>>> Alex, >>>>> >>>>> On 29/03/2019 22:07, Alex Menkov wrote: >>>>>> (added net-dev as suggested by Alan) >>>>>> >>>>>> Net gurus, please assist in reviewing socket-related code. >>>>>> >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ >>>>> >>>>> Specifically on SocketTransportService.java. What Arthur has >>>>> proposed is better ( changing to lastIndexOf alone is not >>>>> sufficient ). Or is your assumption that the IPv6 literal is >>>>> not enclosed in square brackets? >>>> >>>> I didn't know about enclosing IPv6 in square brackets, but looks >>>> like that's standard way to alleviate conflict between IPv6 address >>>> and colon as port separator. >>>> Will update the fix to handle them in both JDI connectors >>>> (SocketTransportService.java) and debugger agent (socketTransport.c) >>>> >>>>> >>>>> If keeping Arthur's `static class HostPort` please make the >>>>> fields final. >>>>> >>>>> ?>> Would it make sense to support the preference properties? >>>>> ?>>??? java.net.preferIPv4Stack >>>>> ?>>??? java.net.preferIPv6Addresses >>>>> >>>>> I'm not sure about this, especially given the property name >>>>> prefixes. I need to think a little more on it. >>>> >>>> In the initial version of the fix I didn't check the properties. >>>> The rationale here is backward compatibility - is address is empty, >>>> old debuggers tries to connect to IPv4 address only. >>>> But handling this properties we will better handle clients with >>>> properties set (as jdb or any other debugger which uses JDI >>>> connectors are affected by the properties). >>>> >>>> BTW fix provided by Arthur implements listening on localhost >>>> differently - it creates several sockets and binds them to both >>>> IPv4 and IPv6 addresses. But the problem here (and that's the >>>> reason I decide to not implement it this way) - how to handle the >>>> case when we successfully bind on one address (for example IPv4), >>>> but fail to bind on other (for example the port is busy for IPv6 >>>> stack). Arthur's version just fail in the case (i.e. the whole Java >>>> process terminates) and I don't think this is good behavior. >>>> >>>> >>>>> >>>>> There is quite a bit of new native code, is it possible to >>>>> rewrite any of this in Java, e.g. reading of the system >>>>> properties ( if that is to remain )? >>>> >>>> socketTransport.c is a debugger agent which is completely native. >>>> >>>> --alex >>>> >>>>> >>>>> -Chris. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Mon Apr 15 23:37:47 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 15 Apr 2019 16:37:47 -0700 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: While looking through some of the tests, I noticed that there is jdk.test.lib.NetworkConfiguration which has methods like "isIPv6Available". It seems much more comprehensive than what is in the proposed IPSupport. Although the methods are not static and require a `NetworkConfiguration nc = NetworkConfiguration.probe()`. Regarding trusting the IPSupport, I have checked to make sure that it's reasonable. It correctly throws a jtreg.SkippedException when preferIPv4Stack is true and IPv4 is not available, and leaves the other cases alone. This was tested with an LD_PRELOAD that returns an error on `socket()`, `setsockopt()`, and `getifaddrs()` on IPv4 addresses. Under a typical dual stack configuration it doesn't throw anything. This was a good idea, thanks Daniel. On Thu, Apr 11, 2019 at 11:45 AM Roger Riggs wrote: > Hi, > > If you need different @requires for different runs, there is a jtreg > feature to define separate tests using separate comment blocks. > > /*... > *@requires x > *@run ... > */ > > /* > * @requires !x > * @run ... > */ > > $.02, Roger > > > On 04/11/2019 12:01 PM, Chris Hegarty wrote: > > Arthur, Daniel, > > > >> On 10 Apr 2019, at 21:39, Arthur Eubanks >> > wrote: > >> > >> Here's a prototype webrev to see if the approach is okay with you. If > >> it looks good, I'll continue with the remaining tests I can find. > >> (should I start a new thread for the webrev?) > >> > >> http://cr.openjdk.java.net/~aeubanks/8220673/webrev.00 > >> > > > > This is not too bad. Given the typical usage, one `@run` followed by a > > second `@run + system property`, then we kinda cannot easily use an > > `@requires`. Also, it would be good to dump/print the IP configuration, > > as determined by IPSupport, if the test will not be run. This will be > > useful when digging into test output jtr files. > > > > An alternative, rather than simple returning ( when the configuration > > does not support the test ) maybe throw jtreg.SkippedException [1], to > > indicate that the test has been skipped (rather than run successfully). > > An example of this is LevelTransitionTest [2]. > > > > That said, I would still like to experiment a little with @requires for > > the cases where there is a single @run tag. > > > > > >> On 11 Apr 2019, at 14:05, Daniel Fuchs >> > wrote: > >> ... > >> We're putting a lot of trust in the implementation of this > >> IPSupport utility. > > > > Yes, this is a good point. What?s nice about this is that there is just > > one body of code that provides the functionality ( and it is all in > > Java, not native). I'm interested to see how this performs in Arthur's > > experiments, and I need to do a little more testing myself. It is > > important that a test leaves enough breadcrumbs behind if it decides > > that it will not run. > > > > -Chris > > > > [1] > > > https://openjdk.java.net/jtreg/faq.html#what-if-a-test-does-not-apply-in-a-given-situation > > [2] > > > https://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/compiler/tiered/LevelTransitionTest.java > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Apr 16 08:54:49 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 16 Apr 2019 09:54:49 +0100 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> Message-ID: <28EBA60A-E235-4204-BC77-7E39F880ECBD@oracle.com> Arthur, > On 16 Apr 2019, at 00:37, Arthur Eubanks wrote: > > While looking through some of the tests, I noticed that there is jdk.test.lib.NetworkConfiguration which has methods like "isIPv6Available". It seems much more comprehensive than what is in the proposed IPSupport. Although the methods are not static and require a `NetworkConfiguration nc = NetworkConfiguration.probe()`. As the original author of NetworkConfiguration, its intent is to provide a consistent set of network interfaces that may be used by tests. For example, filtering out of certain problematic platform specific tunnelling interfaces, etc, can be done in a single place ( rather than scatter throughout individual tests ). Currently NetworkConfiguration knows nothing of the IP related system properties. What IPSupport provides is somewhat orthogonal to that of NetworkConfiguration. Of course their functionality could be merged into a single class, but I find the separation of concerns, network interface versus socket functionality, makes the code more readable and understandable. > Regarding trusting the IPSupport, I have checked to make sure that it's reasonable. It correctly throws a jtreg.SkippedException when preferIPv4Stack is true and IPv4 is not available, and leaves the other cases alone. This was tested with an LD_PRELOAD that returns an error on `socket()`, `setsockopt()`, and `getifaddrs()` on IPv4 addresses. Under a typical dual stack configuration it doesn't throw anything. This was a good idea, thanks Daniel. Indeed. And thanks for verify its behaviour. Let?s move ahead with this and see how far it can get us. -Chris. From aeubanks at google.com Tue Apr 16 21:14:33 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Tue, 16 Apr 2019 14:14:33 -0700 Subject: [ipv6] Regarding 8220673: Add test library support for determining platform IP support In-Reply-To: <28EBA60A-E235-4204-BC77-7E39F880ECBD@oracle.com> References: <34640ecf-3396-786d-f2dd-e87a174b3c4f@oracle.com> <64950EF2-A595-45A4-8960-4DD9089D8BF4@oracle.com> <28EBA60A-E235-4204-BC77-7E39F880ECBD@oracle.com> Message-ID: > > > Regarding trusting the IPSupport, I have checked to make sure that it's > reasonable. It correctly throws a jtreg.SkippedException when > preferIPv4Stack is true and IPv4 is not available, and leaves the other > cases alone. This was tested with an LD_PRELOAD that returns an error on > `socket()`, `setsockopt()`, and `getifaddrs()` on IPv4 addresses. Under a > typical dual stack configuration it doesn't throw anything. This was a good > idea, thanks Daniel. > > Indeed. And thanks for verify its behaviour. > I lied, it worked on our JDK which has some IPv6 patches, not the upstream JDK. I think to properly test this, first I need to upstream a tiny portion of the patches we have to the JDK. I'll file a bug and send out a patch. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Tue Apr 16 21:34:45 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Tue, 16 Apr 2019 14:34:45 -0700 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) Message-ID: Hi, Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: Some of the networking code tries to support dual socket support. However, it doesn't work with IPv6 only systems. setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with errno EAFNOSUPPORT, and then the networking code bails. It should not bail when it sees that trying to set IPV6_V6ONLY fails. webrev: http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.00/index.html One thing to note is that similar code exists in the Windows part of the networking code (e.g. java.base/windows/native/libnet/PlainSocketImpl.c), but I don't know how to test Windows code so I'm only touching the Unix-specific part. Running this through the submit repo. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmarks at google.com Wed Apr 17 00:26:11 2019 From: pmarks at google.com (Paul Marks) Date: Tue, 16 Apr 2019 17:26:11 -0700 Subject: [ipv6] Issue Tracking In-Reply-To: References: Message-ID: On Mon, Apr 1, 2019 at 9:38 AM Martin Buchholz wrote: > Sorry for never having had the time to finish > https://bugs.openjdk.java.net/browse/JDK-8179037 > Improve IPv6 support > I think https://bugs.openjdk.java.net/browse/JDK-8170568 (linked from 8179037) is the biggest problem here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at oracle.com Wed Apr 17 03:09:02 2019 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 17 Apr 2019 11:09:02 +0800 Subject: How to deal with multi-step WWW-Authenticate with the new HttpClient Message-ID: Hi, All, I am trying the new HttpClient to deal with a multi-step Negotiate authentication with Windows IIS, and here is my code. HttpClient hc = HttpClient.newBuilder().build(); var req = HttpRequest.newBuilder().uri(new URI(args[0])); while (true) { var resp = hc.send(req.build(), HttpResponse.BodyHandlers.ofString()); System.out.println("--------------------"); System.out.println(resp.statusCode()); String auth = resp.headers().allValues("WWW-Authenticate") .stream() .filter(s -> s.startsWith("Negotiate")) .findFirst() .orElseThrow() .substring(9) .trim(); System.out.println("incoming " + auth); byte[] in = auth.isEmpty() ? new byte[0] :Base64.getDecoder().decode(auth); byte[] out = calculate_token_from_incoming(in); if (out == null) break; // if status is still >400. No way to continue String sent = Base64.getEncoder().encodeToString(out); System.out.println("--------------------"); System.out.println("outgoing " + sent); req.header("Authorization", "Negotiate " + sent); } This works when there is only one request and one reply, but fails when there is more. Is this the correct way? Is there some keep-connnection thing I need to care about? Thanks, Max p.s. Or maybe there is something wrong with IIS. This is a corner case. From Alan.Bateman at oracle.com Wed Apr 17 06:54:52 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Apr 2019 07:54:52 +0100 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: On 16/04/2019 22:34, Arthur Eubanks wrote: > Hi, > > Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: > Some of the networking code tries to support dual socket support. > However, it doesn't work with IPv6 only systems. > > setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with > errno EAFNOSUPPORT, and then the networking code bails. It should not > bail when it sees that trying to set IPV6_V6ONLY fails. > > webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.00/index.html > > One thing to note is that similar code exists in the Windows part of > the networking code (e.g. > java.base/windows/native/libnet/PlainSocketImpl.c), but I don't know > how to test Windows code so I'm only touching the Unix-specific part. > This code is compiled on other Unix-like ports too so I think we should at least try to see if they also return EAFNOSUPPORT when configured for IPv6-only. -Alan From daniel.fuchs at oracle.com Wed Apr 17 11:13:42 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 17 Apr 2019 12:13:42 +0100 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: Hi Arthur, On 17/04/2019 07:54, Alan Bateman wrote: > On 16/04/2019 22:34, Arthur Eubanks wrote: >> Hi, >> >> Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: >> Some of the networking code tries to support dual socket support. >> However, it doesn't work with IPv6 only systems. >> >> setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with >> errno EAFNOSUPPORT, and then the networking code bails. It should not >> bail when it sees that trying to set IPV6_V6ONLY fails. >> >> webrev: >> http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.00/index.html >> >> One thing to note is that similar code exists in the Windows part of >> the networking code (e.g. >> java.base/windows/native/libnet/PlainSocketImpl.c), but I don't know >> how to test Windows code so I'm only touching the Unix-specific part. >> > This code is compiled on other Unix-like ports too so I think we should > at least try to see if they also return EAFNOSUPPORT when configured for > IPv6-only. Also should we have an ipv4_available() utility similar to ipv6_available() and check that before making the decision not to bail on EAFNOSUPPORT? Or would that be a chicken and egg problem? best regards, -- daniel > > -Alan From weijun.wang at oracle.com Wed Apr 17 13:24:54 2019 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 17 Apr 2019 21:24:54 +0800 Subject: How to deal with multi-step WWW-Authenticate with the new HttpClient In-Reply-To: References: Message-ID: Sorry, after adding "-Djdk.httpclient.HttpClient.log=headers,errors,channel" I found out it's my problem. > On Apr 17, 2019, at 11:09 AM, Weijun Wang wrote: > > Hi, All, > > I am trying the new HttpClient to deal with a multi-step Negotiate authentication with Windows IIS, and here is my code. > > HttpClient hc = HttpClient.newBuilder().build(); > var req = HttpRequest.newBuilder().uri(new URI(args[0])); > while (true) { > var resp = hc.send(req.build(), HttpResponse.BodyHandlers.ofString()); > System.out.println("--------------------"); > System.out.println(resp.statusCode()); > String auth = resp.headers().allValues("WWW-Authenticate") > .stream() > .filter(s -> s.startsWith("Negotiate")) > .findFirst() > .orElseThrow() > .substring(9) > .trim(); > System.out.println("incoming " + auth); > byte[] in = auth.isEmpty() ? new byte[0] :Base64.getDecoder().decode(auth); > byte[] out = calculate_token_from_incoming(in); > if (out == null) break; // if status is still >400. No way to continue > String sent = Base64.getEncoder().encodeToString(out); > System.out.println("--------------------"); > System.out.println("outgoing " + sent); > req.header("Authorization", "Negotiate " + sent); I am using the wrong method here, should be "setHeader". Thanks, Max > } > > This works when there is only one request and one reply, but fails when there is more. > > Is this the correct way? Is there some keep-connnection thing I need to care about? > > Thanks, > Max > > p.s. Or maybe there is something wrong with IIS. This is a corner case. > From chris.hegarty at oracle.com Wed Apr 17 14:01:20 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Apr 2019 15:01:20 +0100 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: Arthur, On 16/04/2019 22:34, Arthur Eubanks wrote: > Hi, > > Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: > Some of the networking code tries to support dual socket support. > However, it doesn't work with IPv6 only systems. > > setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with > errno EAFNOSUPPORT, and then the networking code bails. It should not > bail when it sees that trying to set IPV6_V6ONLY fails. I assume that you are on Linux, right? If so, have you compiled the Kernel without IPv4 support? How have you configured your system without IPv4 support? I'm curious as I would like to try something similar, as well as on other platforms, macOS, etc. -Chris. From chris.hegarty at oracle.com Wed Apr 17 14:58:13 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Apr 2019 15:58:13 +0100 Subject: How to deal with multi-step WWW-Authenticate with the new HttpClient In-Reply-To: References: Message-ID: <7369d9aa-a63a-e43c-1739-ccbfd387190c@oracle.com> Thank you for following up with the resolution, Max. -Chris. On 17/04/2019 14:24, Weijun Wang wrote: > Sorry, after adding "-Djdk.httpclient.HttpClient.log=headers,errors,channel" I found out it's my problem. > >> On Apr 17, 2019, at 11:09 AM, Weijun Wang wrote: >> >> Hi, All, >> >> I am trying the new HttpClient to deal with a multi-step Negotiate authentication with Windows IIS, and here is my code. >> >> HttpClient hc = HttpClient.newBuilder().build(); >> var req = HttpRequest.newBuilder().uri(new URI(args[0])); >> while (true) { >> var resp = hc.send(req.build(), HttpResponse.BodyHandlers.ofString()); >> System.out.println("--------------------"); >> System.out.println(resp.statusCode()); >> String auth = resp.headers().allValues("WWW-Authenticate") >> .stream() >> .filter(s -> s.startsWith("Negotiate")) >> .findFirst() >> .orElseThrow() >> .substring(9) >> .trim(); >> System.out.println("incoming " + auth); >> byte[] in = auth.isEmpty() ? new byte[0] :Base64.getDecoder().decode(auth); >> byte[] out = calculate_token_from_incoming(in); >> if (out == null) break; // if status is still >400. No way to continue >> String sent = Base64.getEncoder().encodeToString(out); >> System.out.println("--------------------"); >> System.out.println("outgoing " + sent); >> req.header("Authorization", "Negotiate " + sent); > > I am using the wrong method here, should be "setHeader". > > Thanks, > Max > >> } >> >> This works when there is only one request and one reply, but fails when there is more. >> >> Is this the correct way? Is there some keep-connnection thing I need to care about? >> >> Thanks, >> Max >> >> p.s. Or maybe there is something wrong with IIS. This is a corner case. >> > From daniel.fuchs at oracle.com Wed Apr 17 15:13:10 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 17 Apr 2019 16:13:10 +0100 Subject: RFR: 8222527: HttpClient doesn't send HOST header when tunelling HTTP/1.1 through http proxy Message-ID: <84606e2a-d3a5-4933-f867-d19d11f90147@oracle.com> Hi, Please find below a fix for: Issue: 8222527: HttpClient doesn't send HOST header when tunnelling HTTP/1.1 through http proxy https://bugs.openjdk.java.net/browse/JDK-8222527 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.00 When sending the internal CONNECT request to the proxy the stack strips out headers that are destined to the target host. This fix will ensure that the host header is not stripped in order to keep conformant with RFC 7230 section 5.4 and RFC 7231 section 4.3.6. best regards, -- daniel From chris.hegarty at oracle.com Wed Apr 17 15:16:12 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Apr 2019 16:16:12 +0100 Subject: [ipv6] Issue Tracking In-Reply-To: References: Message-ID: <4d208ade-6764-0353-5db3-ff5528953c06@oracle.com> Paul, On 17/04/2019 01:26, Paul Marks wrote: > On Mon, Apr 1, 2019 at 9:38 AM Martin Buchholz > wrote: > > Sorry for never having had the time to finish > https://bugs.openjdk.java.net/browse/JDK-8179037 > Improve IPv6 support > > > I think https://bugs.openjdk.java.net/browse/JDK-8170568?(linked > from?8179037) is the biggest problem here. I don't disagree. Unfortunately not much progress has been made in this area, yet. I remember discussing this with some folk while at the 80th IETF meeting in Prague, in 2011. Specifically at the time, the discussion was around prototyping Happy Eyeballs [1] at the java.net.Socket level. I am hopefully that this renewed effort, that is ongoing here, will eventually address this long outstanding deficiency. I'll make sure that all the JIRA issues are linked up appropriately, so that this issue, 8170568, is not overlooked. -Chris. [1] https://tools.ietf.org/html/rfc6555 From michael.x.mcmahon at oracle.com Wed Apr 17 16:06:45 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 17 Apr 2019 17:06:45 +0100 Subject: RFR: 8222527: HttpClient doesn't send HOST header when tunelling HTTP/1.1 through http proxy In-Reply-To: <84606e2a-d3a5-4933-f867-d19d11f90147@oracle.com> References: <84606e2a-d3a5-4933-f867-d19d11f90147@oracle.com> Message-ID: <5CB74F15.8090106@oracle.com> Hi Daniel, Looks fine. Though is the HOST_ALLOWED predicate actually needed in Utils.java? Can you not do the negation inline directly in the HOST_RESTRICTED predicate? - Michael. On 17/04/2019, 16:13, Daniel Fuchs wrote: > Hi, > > Please find below a fix for: > > Issue: > 8222527: HttpClient doesn't send HOST header when tunnelling HTTP/1.1 > through http proxy > https://bugs.openjdk.java.net/browse/JDK-8222527 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.00 > > When sending the internal CONNECT request to the proxy > the stack strips out headers that are destined to the > target host. This fix will ensure that the host header > is not stripped in order to keep conformant with RFC 7230 > section 5.4 and RFC 7231 section 4.3.6. > > best regards, > > -- daniel From daniel.fuchs at oracle.com Wed Apr 17 17:00:51 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 17 Apr 2019 18:00:51 +0100 Subject: RFR: 8222527: HttpClient doesn't send HOST header when tunelling HTTP/1.1 through http proxy In-Reply-To: <5CB74F15.8090106@oracle.com> References: <84606e2a-d3a5-4933-f867-d19d11f90147@oracle.com> <5CB74F15.8090106@oracle.com> Message-ID: <4d98da35-f5c4-b2e1-96a8-fd794538d377@oracle.com> Hi Michael, On 17/04/2019 17:06, Michael McMahon wrote: > Looks fine. Though is the HOST_ALLOWED predicate actually needed in > Utils.java? > Can you not do the negation inline directly in the HOST_RESTRICTED > predicate? Good point! http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.01 best regards, -- daniel > > - Michael. > > On 17/04/2019, 16:13, Daniel Fuchs wrote: >> Hi, >> >> Please find below a fix for: >> >> Issue: >> 8222527: HttpClient doesn't send HOST header when tunnelling HTTP/1.1 >> ???????? through http proxy >> https://bugs.openjdk.java.net/browse/JDK-8222527 >> >> webrev: >> http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.00 From aeubanks at google.com Wed Apr 17 17:57:45 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Wed, 17 Apr 2019 10:57:45 -0700 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: It turns out these patches only apply to a modified kernel which has patches to turn off IPv4. This specific patch doesn't make sense to submit if the kernel patches are not upstream. Maybe the entire call to setsockopt(IPV6_V6ONLY, 0) could be skipped if IPv4 is disabled. I'll look into Daniel's suggestion of a ipv4_available() utility. On Wed, Apr 17, 2019 at 7:01 AM Chris Hegarty wrote: > Arthur, > > On 16/04/2019 22:34, Arthur Eubanks wrote: > > Hi, > > > > Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: > > Some of the networking code tries to support dual socket support. > > However, it doesn't work with IPv6 only systems. > > > > setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with > > errno EAFNOSUPPORT, and then the networking code bails. It should not > > bail when it sees that trying to set IPV6_V6ONLY fails. > > I assume that you are on Linux, right? If so, have you compiled the > Kernel without IPv4 support? How have you configured your system without > IPv4 support? > > I'm curious as I would like to try something similar, as well as on > other platforms, macOS, etc. > > -Chris. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Thu Apr 18 15:03:59 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 18 Apr 2019 16:03:59 +0100 Subject: RFR: 8222527: HttpClient doesn't send HOST header when tunelling HTTP/1.1 through http proxy In-Reply-To: <4d98da35-f5c4-b2e1-96a8-fd794538d377@oracle.com> References: <84606e2a-d3a5-4933-f867-d19d11f90147@oracle.com> <5CB74F15.8090106@oracle.com> <4d98da35-f5c4-b2e1-96a8-fd794538d377@oracle.com> Message-ID: <5CB891DF.9000303@oracle.com> Thanks Daniel. All good. - Michael. On 17/04/2019, 18:00, Daniel Fuchs wrote: > Hi Michael, > > On 17/04/2019 17:06, Michael McMahon wrote: >> Looks fine. Though is the HOST_ALLOWED predicate actually needed in >> Utils.java? >> Can you not do the negation inline directly in the HOST_RESTRICTED >> predicate? > > Good point! > > http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.01 > > best regards, > > -- daniel > >> >> - Michael. >> >> On 17/04/2019, 16:13, Daniel Fuchs wrote: >>> Hi, >>> >>> Please find below a fix for: >>> >>> Issue: >>> 8222527: HttpClient doesn't send HOST header when tunnelling HTTP/1.1 >>> through http proxy >>> https://bugs.openjdk.java.net/browse/JDK-8222527 >>> >>> webrev: >>> http://cr.openjdk.java.net/~dfuchs/webrev_8222527/webrev.00 > From macanaoire at hotmail.com Fri Apr 19 10:44:25 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Fri, 19 Apr 2019 10:44:25 +0000 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: , Message-ID: a couple of points and observations * What is the platform's semantics when both java.net.preferIPv4Stack and java.net.preferIPv6Address are set simultaneously. * Should the EAFNOSUPPORT check be more pervasive within the native code? * IPv4 and IPv6 available doesn't mean that the stacks have been appropriately configured at the OS network level * It would seems worthwhile making access to the platforms perspective of stack duality directly available at the API level * It would seem worthwhile making the NetworkConfiguration abstraction available as util class in the API. * Is it possible at the OS level to dynamically turn on and off IPv6 support and IPv4 support? Thus a more dynamic test on on OS config is needed, rather than the onload config setting. * It may be useful to define the set of use case scenarios for the different possible IPv4 IPv6 combinations and descibe the expected behaviour 1. What is the platform's semantics when both java.net.preferIPv4Stack and java.net.preferIPv6Address are set simultaneously. The java.net.preferIPv4Stack precludes IPv6 usage, [ ? However, in the case an application would rather use IPv4 only sockets, then this property can be set to true. ]. While java.net.preferIPv6Addresses, does not exclude IPv4, [ ? This property can be set to true to change that preference and use IPv6 addresses over IPv4 ones where possible]. what should be the effect if an applications sets these properties during the course of its execution? afaik currently they are only accessed during startup and initialization and set the operational behaviour immutably thereafter. Thus, in IPv6_available is set on load. IPv6_available = IPv6_supported() & (!preferIPv4Stack); This doesn't take into account that an application might change it operational semantics dynamically ? 2. Should the EAFNOSUPPORT check be more pervasive within the native code? Is it not the case now, with the additional IPv6 support considerations, that the socket system call returns need to be augmented with an errno == EAFNOSUPPORT, also. Also as per the socket man pages description, the other errors are somewhat spurious and not indicative of no support, with exception of EPROTONOSUPPORT. So the generalised < 0 check should be refined with the additional check for errno == EAFNOSUPPORT . The other errno could considered exceptions and thrown as such. OR add EAFNOSUPPORT to handleSocketError ? so in IPv6_supported() if (fd < 0 && errno == EAFNOSUPPORT) { return JNI_FALSE; } else { // some approriate handling the others errors // are they indicative non availability or a transient // OS config error // could be run out of file descriptions! handleSocketError(...); } OR treat the other socket errors as spurious and proceed optimistically if (fd < 0 && errno == EAFNOSUPPORT) { return JNI_FALSE; } The java.net.preferIPv4Stack precludes IPv6 usage, so that is determined on load Thus in the case of Java_sun_nio_ch_Net_socket0 if the socket creation fails, should it be necessary to determine the semantics of the failure rather than just exiting ? maybe the OS ran out of file descriptors ! which domain was the socket creation using ... it could be IPv6 was supported at startup and then got reconfigured and is not supported when the socket call is being made so then an attempted should be made for IPv4 AF_INET or IPv4 was configured and now IPv6 instead (some puppet or chef jobby decided to reconfigure the host network configuration) if (fd < 0 && errno != EAFNOSUPPORT) { // handle the error } else { // check the preferences and attempt a socket in AF_INET domain } 3. IPv4 and IPv6 available doesn't mean that the stacks have been appropriately configured at the OS network level 4. It would seem worthwhile making the NetworkConfiguration abstraction available as util class in the API. Also worth noting that availability of either IPv6 and IPv4 doesn't guarantee that they have been configured in the operating system at the network interface level. As such, while sockets may be created their proper functioning may be impaired. Thus, the usefulness of the utility class NetworkConfiguration for determining how various interfaces are configured. As a convenience at the application level it may be worth considering promoting the NetworkConfiguration from the test environment to the java.net API as a utility class. Similarly IPSupport abstract provides some convient functionality. An alternative implementation to this abstraction could be to make available the IPv6_available IPv4_available via IPv6supported and IPv4supported methods as part of an IPSupport API utility class. 5. Is it possible at the OS level to dynamically turn on and off IPv6 support and IPv4 support? Thus a more dynamic test on on OS config is needed, rather than the onload config setting. the global IPv6_available is set once on load, afaik - which may change during the execution of an application (happens a lot in the test environments) - as mentioned above the return from the socket call is augmented with errno check So there are a range of possible scenarios, which may be affected by dynamic configuration of a host while an application is executing. regards Mark ________________________________ From: net-dev on behalf of Arthur Eubanks Sent: Wednesday 17 April 2019 17:57 To: Chris Hegarty Cc: OpenJDK Network Dev list Subject: Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) It turns out these patches only apply to a modified kernel which has patches to turn off IPv4. This specific patch doesn't make sense to submit if the kernel patches are not upstream. Maybe the entire call to setsockopt(IPV6_V6ONLY, 0) could be skipped if IPv4 is disabled. I'll look into Daniel's suggestion of a ipv4_available() utility. On Wed, Apr 17, 2019 at 7:01 AM Chris Hegarty > wrote: Arthur, On 16/04/2019 22:34, Arthur Eubanks wrote: > Hi, > > Copied from the bug https://bugs.openjdk.java.net/browse/JDK-8222562: > Some of the networking code tries to support dual socket support. > However, it doesn't work with IPv6 only systems. > > setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, 0) returns a failure with > errno EAFNOSUPPORT, and then the networking code bails. It should not > bail when it sees that trying to set IPV6_V6ONLY fails. I assume that you are on Linux, right? If so, have you compiled the Kernel without IPv4 support? How have you configured your system without IPv4 support? I'm curious as I would like to try something similar, as well as on other platforms, macOS, etc. -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Sun Apr 21 19:23:34 2019 From: martinrb at google.com (Martin Buchholz) Date: Sun, 21 Apr 2019 12:23:34 -0700 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: On Wed, Apr 17, 2019 at 10:58 AM Arthur Eubanks wrote: > It turns out these patches only apply to a modified kernel which has > patches to turn off IPv4. This specific patch doesn't make sense to submit > if the kernel patches are not upstream. > I disagree a little. Google may be decades ahead, but IPV4 being unsupported is a very reasonable system configuration option that will eventually be part of every OS. Most of the time nonstandard kernels can be simulated for testing via LD_PRELOAD interposing in front of libc, since we never talk to the kernel directly. And even if some changes can only be tested with Google's linux kernel mods, that is still OK IMO. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Mon Apr 22 16:29:55 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 22 Apr 2019 09:29:55 -0700 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: > > It turns out these patches only apply to a modified kernel which has >> patches to turn off IPv4. This specific patch doesn't make sense to submit >> if the kernel patches are not upstream. >> > > I disagree a little. > Google may be decades ahead, but IPV4 being unsupported is a very > reasonable system configuration option that will eventually be part of > every OS. > Most of the time nonstandard kernels can be simulated for testing via > LD_PRELOAD interposing in front of libc, since we never talk to the kernel > directly. > And even if some changes can only be tested with Google's linux kernel > mods, that is still OK IMO. > When I said "this specific patch" I meant specifically checking for EAFNOSUPPORT. Some other form of this, such as not trying to set IPV6_V6ONLY to 0 when IPv4 is detected to not be working, may still make sense (and that's what I'm looking into). And yes that can be tested with LD_PRELOAD. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Mon Apr 22 18:19:37 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Mon, 22 Apr 2019 11:19:37 -0700 Subject: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: Here's a new webrev that entirely skips the setting of IPV6_V6ONLY option to 0 if IPv4 isn't enabled. http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.01/index.html Right now the new ipv4_available() helper function is only used to check if we should set IPV6_V6ONLY to 0, but in the future it could be used for other things as well. I'm not sure if the checks in ipv6_available() apply to IPv4 as well, so I didn't bother copying those over to ipv4_available(). Plus as to not potentially break existing systems with IPv4 support, false positives for ipv4_available() would be preferable to false negatives. Will run through submit repo, mostly to check that this doesn't break the Windows build. On Mon, Apr 22, 2019 at 9:29 AM Arthur Eubanks wrote: > It turns out these patches only apply to a modified kernel which has >>> patches to turn off IPv4. This specific patch doesn't make sense to submit >>> if the kernel patches are not upstream. >>> >> >> I disagree a little. >> Google may be decades ahead, but IPV4 being unsupported is a very >> reasonable system configuration option that will eventually be part of >> every OS. >> Most of the time nonstandard kernels can be simulated for testing via >> LD_PRELOAD interposing in front of libc, since we never talk to the kernel >> directly. >> And even if some changes can only be tested with Google's linux kernel >> mods, that is still OK IMO. >> > When I said "this specific patch" I meant specifically checking > for EAFNOSUPPORT. Some other form of this, such as not trying to set > IPV6_V6ONLY to 0 when IPv4 is detected to not be working, may still make > sense (and that's what I'm looking into). And yes that can be tested with > LD_PRELOAD. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Apr 23 10:20:10 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Apr 2019 11:20:10 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: Message-ID: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> Arthur, On 22/04/2019 19:19, Arthur Eubanks wrote: > Here's a new webrev that entirely skips the setting of IPV6_V6ONLY > option to 0 if IPv4 isn't enabled. > > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.01/index.html This looks reasonable to me. -Chris. From chris.hegarty at oracle.com Tue Apr 23 10:56:53 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Apr 2019 11:56:53 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> Message-ID: <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> Arthur, On 23/04/2019 11:20, Chris Hegarty wrote: > Arthur, > > On 22/04/2019 19:19, Arthur Eubanks wrote: >> Here's a new webrev that entirely skips the setting of IPV6_V6ONLY >> option to 0 if IPv4 isn't enabled. >> >> http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.01/index.html > > This looks reasonable to me. The *approach* looks reasonable. Specifically on the changes... I believe that `DONT_ENABLE_IPV4`is not strictly required. Maybe just drop it? Is it an initial attempt to support an IPv6-only JDK build? If so, then maybe we separate out that requirement. -Chris. From pavel.rappo at oracle.com Tue Apr 23 12:22:13 2019 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 23 Apr 2019 13:22:13 +0100 Subject: [ipv6] On updating certificates in tests Message-ID: Arthur, Some time ago Chris mentioned [1] that there was at least one test failing due to missing SANs in its certificates: test/jdk/sun/net/www/protocol/https/HttpsURLConnection/IPAddressIPIdentities.java You then replied that your were planning to address that issue in a future change. I thought it might be helpful to you to get step-by-step instructions on how to change that test security setup, so you won't waste too much of your time. Bear in mind that this is just one way to achieve that. This should work reasonably well for not more than a handful of such tests. If you find more tests relying on SAN in certificates, please get back on the list and we'll try figure out a better way of doing this together. I would not go there just yet, because I have a heavy suspicion it will require programmatic access to keytool and its security friends. It might cause restructuring the test significantly. So let's see the affected area first. Here we go: 1. Create a directory and cd into it (so everything is clean and tidy) mkdir my-certificates-for-tests && cd my-certificates-for-tests 2. Generate a self-signed CA certificate keytool -genkeypair \ -keyalg RSA \ -startdate 2019/01/01 \ -validity 13000 \ -keysize 1024 \ -dname "C=XX, ST=CA-State, L=CA-City, O=CA-Org" \ -ext bc=ca:true \ -storetype PKCS12 \ -alias root \ -keystore temp.jks \ -storepass 123456 3. Generate a self-signed (for now) certificate for the server keytool -genkeypair \ -keyalg RSA \ -keysize 1024 \ -dname "C=YY, ST=Server-State, L=Server-City, O=Server, OU=Server-Unit, CN=Server-Name" \ -alias server \ -keystore temp.jks \ -storepass 123456 4. Sign that server certificate using the previously generated CA?s certificate keytool -certreq \ -alias server \ -keystore temp.jks \ -storepass 123456 \ | keytool -gencert \ -rfc \ -startdate 2019/01/02 \ -validity 12000 \ -ext san:critical=ip:127.0.0.1,ip:0:0:0:0:0:0:0:1 \ -alias root \ -keystore temp.jks \ -storepass 123456 \ | keytool -importcert \ -alias server \ -keystore temp.jks \ -storepass 123456 5. Repeat the process for the client certificate keytool -genkeypair \ -keyalg RSA \ -keysize 1024 \ -dname "C=ZZ, ST=Client-State, L=Client-City, O=Client, OU=Client-Unit, CN=Client-Name" \ -alias client \ -keystore temp.jks \ -storepass 123456 then keytool -certreq \ -alias client \ -keystore temp.jks \ -storepass 123456 \ | keytool -gencert \ -rfc \ -startdate 2019/01/02 \ -validity 12000 \ -ext san:critical=ip:127.0.0.1,ip:0:0:0:0:0:0:0:1 \ -alias root \ -keystore temp.jks \ -storepass 123456 \ | keytool -importcert \ -alias client \ -keystore temp.jks \ -storepass 123456 Now, so as to update the test source, you will need to print the certificates and the keys. As far as I know, keytool cannot really do this (however, there's an API to which we might fall back later). For now you could simply use the openssl tool. 6. Print the certificates keytool -list \ -rfc \ -alias root \ -keystore temp.jks \ -storepass 123456 \ | openssl x509 -inform pem -text keytool -list \ -rfc \ -alias server \ -keystore temp.jks \ -storepass 123456 \ | openssl x509 -inform pem -text keytool -list \ -rfc \ -alias client \ -keystore temp.jks \ -storepass 123456 \ | openssl x509 -inform pem -text 7. Export the keys from the keystore openssl pkcs12 -in temp.jks -nodes -nocerts -out keys.pem -passin pass:123456 8. Manually split the resulting `keys.pem` file into 2: server.pem and client.pem, discarding the root key. 9. Print the keys openssl rsa -in server.pem -text openssl rsa -in client.pem -text Now you should have all the data required to update the test. -Pavel -------------------------------------------------------------------------------- [1] https://mail.openjdk.java.net/pipermail/net-dev/2019-March/012311.html From aeubanks at google.com Tue Apr 23 15:54:52 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Tue, 23 Apr 2019 08:54:52 -0700 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> Message-ID: > > I believe that `DONT_ENABLE_IPV4`is not strictly required. Maybe just > drop it? Is it an initial attempt to support an IPv6-only JDK build? > If so, then maybe we separate out that requirement. > It was in the IPv6 code right below so I just copied it in case we wanted an IPv6-only build at some point. But yes that can be in a separate change. Removed. New webrev: http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Apr 23 20:27:53 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Apr 2019 21:27:53 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> Message-ID: On 23/04/2019 16:54, Arthur Eubanks wrote: > > I believe that `DONT_ENABLE_IPV4`is not strictly required. Maybe just > drop it?? Is it an initial attempt to support an IPv6-only JDK build? > If so, then maybe we separate out that requirement. > > It was in the IPv6 code right below so I just copied it in case we > wanted an IPv6-only build at some point. But yes that can be in a > separate change. Removed. > New webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html The changes in this webrev look okay. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Tue Apr 23 23:35:44 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 23 Apr 2019 16:35:44 -0700 Subject: RFR: JDK-8184770: JDWP support for IPv6 In-Reply-To: <6dedcf93-6afe-8ea9-f0dd-f8da8825910a@oracle.com> References: <114d0c7e-9f88-7fab-d535-0fce9ef57945@oracle.com> <5C9CDDE6.90608@oracle.com> <29f35cad-3bcc-813c-dd41-b501b9440fba@oracle.com> <7a180db1-b1c0-c924-7f3b-725e51311f76@oracle.com> <1a836ba1-f8b9-7c0e-a8fb-1e6da25b5046@oracle.com> <6dedcf93-6afe-8ea9-f0dd-f8da8825910a@oracle.com> Message-ID: Hi Serguei, updated webrev: http://cr.openjdk.java.net/~amenkov/IPv6/webrev.04/ Fixed everything except #7 (for consistency - in the file if function arguments spread on several lines, opening curly bracket is placed to separate line to separate function declaration from the function body). --alex On 04/12/2019 18:52, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > Thank you for the update! > > One more round of minor formatting change requests for better > readability. :) > > #1: > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html > > +??????????????? port = Integer.decode(hostPort.substring(splitIndex+1)); > . . . > +??????????? } else if (hostPort.charAt(0) == ?[? && > hostPort.charAt(splitIndex-1) == ?]?) { > > ?Need spaces around ?+? and ?-? signs. > > #2: > > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdwp.agent/windows/native/libdt_socket/socket_md.c.udiff.html > > +??????? //make the socket a dual mode socket > > ?missed space at the start of comment > > > Now, comments for: > http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html > > #3: > > 276???? /* check for host:port or port */ > 277???? colon = strrchr(address, ?:?); > 278???? port = (colon == NULL ? address : colon + 1); > > 279???? /* ensure the port is valid (getaddrinfo allows port to be empty) */ > 280???? if (getPortNumber(port) < 0) { > > #4: > > 298???????? hints.ai_family = allowOnlyIPv4 ? AF_INET : AF_INET6; > 299???????? hints.ai_flags |= AI_PASSIVE | (allowOnlyIPv4 ? 0 : > AI_V4MAPPED | AI_ALL); > 300???????? > 301???? } else { > > #5: Replace ?fills? with ?fills in? in: > > 341? * Parses address (IPv4 or IPv6), fills result by parsed address. > 383? * Parses prefix length from buffer (integer value), fills result > with corresponding net mask. > 485? * Parses ?allow? argument (fills list of allowed peers (global > _peers variable)). > > #6: > > 410???? // generate mask for prefix length > 411???? memset(result, 0, sizeof(*result)); > > 412???? // prefixLen <= 128, so we won?t go over result?s size > 413???? for (int i = 0; prefixLen > 0; i++, prefixLen -= :sunglasses: { > > #7: > > 623 socketTransport_startListening(jdwpTransportEnv* env, const char* > address, > 624??????????????????????????????? char** actualAddress) > 625 { > . . . . > 1173 static int readBooleanSysProp(int *result, int trueValue, int > falseValue, > 1174???? JNIEnv* jniEnv, jclass sysClass, jmethodID getPropMethod, const > char *propName) > 1175 { > > ?Move ?{? to the end of 624 and 1174. (edited) > > #8: > > 1176???? jstring value; > 1177???? jstring name = (*jniEnv)->NewStringUTF(jniEnv, propName); > > 1178???? if (name == NULL) { > . . . . > 1259???? } while (0); > > 1260???? if (jniEnv != NULL && (*jniEnv)->ExceptionCheck(jniEnv)) { > > > Thanks! > Serguei > > > On 4/12/19 4:58 PM, Alex Menkov wrote: >> updated webrev: >> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.03/ >> >> changes (vs webrev.02) are non-functional (added/edited comments, code >> reformatting, renaming convertIpv4ToIpv6 function to >> convertIPv4ToIPv6, renaming env variable to jniEnv (env is already >> used in one of the functions)). >> >> About pass/preferredAddressFamily conditions - there is no "logical >> xor" in C/C++. Also I think that the current condition is clearer. >> >> --alex >> >> On 04/11/2019 17:18, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> Great debugging feature! >>> While I'm still reading all the details, could you, please, fix some >>> minor format issues? >>> >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java.udiff.html >>> >>> >>> + * If host is a literal IPv6 address, it may be in >>> square brackets. Extra space before "square". >>> >>> >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c.frames.html >>> >>> >>> ??I'd suggest to unify comments before functions: >>> ?? - start comment with a capital latter and ended with a dot >>> ?? - use comment format like this: >>> ??? /* >>> ???? */ >>> >>> ??Examples of comments that need this change: >>> >>> 262 /* result must be release with dbgsysFreeAddrInfo */ => /* * >>> Result must be release with dbgsysFreeAddrInfo. ?*/ >>> >>> 325 // input is sockaddr just because all clients have it => >>> /* * Input is sockaddr just because all clients have it. */ >>> >>> 1129 /* reads boolean system value, >>> 1130 * sets *result to trueValue if the ptoperty is "true", >>> 1131 * to falseValue if the property if "false", >>> 1132 * doesn't change *result if the property is not set or failed to >>> read. >>> 1133 */ => /* * Read boolean system value andset result to: * - >>> trueValue if the property is "true" >>> * - falseValue if the property is "false" *?? * Return JNI_OK if >>> result is set, return JNI_ERR? otherwise. >>> */ >>> >>> ? . . . >>> >>> 293 * use IPv6 socket (to accept IPv6 and mapped Ipv4),... >>> 342 * (with AF_INET6 Ipv4 addresses are not parsed even with >>> AI_V4MAPPED and AI_ALL flags) ...345 hints.ai_family = AF_UNSPEC; // >>> IPv6 or mapped Ipv4 ... 360 } else { // Ipv4 address Replace Ipv4 >>> with IPv4 for unification with IPv6 >>> For unification replace: convertIpv4ToIpv6 => convertIPv4ToIPv6 >>> >>> 297 hints.ai_flags |= AI_PASSIVE >>> 298 | (allowOnlyIPv4 ? 0 : AI_V4MAPPED | AI_ALL); Better to have just >>> one line >>> >>> >>> 1135 JNIEnv* env, . . . 1165 JNIEnv* jniEnv = NULL; >>> >>> A suggestion is to use the same name for JNIEnv*: >>> >>> ?? 1135 JNIEnv* jni, . . . >>> ?? 1165 JNIEnv* jni = NULL; >>> >>> >>> ?? Reformat: >>> >>> 608 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 609 || >>> (pass == 1 && ai->ai_family != preferredAddressFamily)) and >>> >>> 828 if ((pass == 0 && ai->ai_family == preferredAddressFamily) 829 || >>> (pass == 1 && ai->ai_family != preferredAddressFamily)) => if ((pass >>> == 0 && ai->ai_family == preferredAddressFamily) || (pass == 1 && >>> ai->ai_family != preferredAddressFamily)) >>> >>> ?? Even better, replace it with logical XOR: >>> >>> ???????????? if ((pass == 0 ^^ ai->ai_family == preferredAddressFamily) >>> >>> >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/ListeningConnector/startListening/startlis001.java.frames.html >>> >>> >>> 102 /* Check that listening address returned by >>> ListeningConnector.startListening() >>> 103 * matches the address which was set via connector's arguments. >>> 104 * Empty host address causes listening for local connections only >>> (loopback interface) >>> 105 * */ Dot is missed at the end. Replace "* */" with "*/". >>> >>> >>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/test/jdk/com/sun/jdi/JdwpAllowTest.java.frames.html >>> >>> >>> 162 // generate allow address by changing random bit in the local >>> address >>> 163 // and calculate 2 masks (prefix length) - one is matches >>> original local address >>> 164 // and another doesn't. Replace with /* style of comment. >>> >>> >>> 249 positiveTest("PositiveMaskTest(" + test.localAddress + ")", >>> test.allowAddress + "/" + test.prefixLengthGood); >>> 250 positiveTest("NegativeMaskTest(" + test.localAddress + ")", >>> test.allowAddress + "/" + test.prefixLengthBad); >>> >>> ?? A suggestion to move second argument to additional line: >>> >>> positiveTest("PositiveMaskTest(" + test.localAddress + ")", >>> test.allowAddress + "/" + test.prefixLengthGood); >>> positiveTest("NegativeMaskTest(" + test.localAddress + ")", >>> test.allowAddress + "/" + test.prefixLengthBad); >>> >>> Thanks, >>> Serguei >>> >>> >>> On 4/2/19 4:14 PM, Alex Menkov wrote: >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.02/ >>>> >>>> - added support for addresses enclosed in square brackets; >>>> - updated SocketTransportService.java to handle empty hostname the >>>> same way as JDWP agent (listen/attach to loopback address); >>>> Has to update >>>> nsk/jdi/ListeningConnector/startListening/startlis001.java >>>> (all other com/sun/jdi, nsk/jdi, nsk/jdwp, nsk/jdb test are passed). >>>> >>>> --alex >>>> >>>> On 04/01/2019 11:21, Alex Menkov wrote: >>>>> Hi Chris, >>>>> >>>>> On 04/01/2019 04:50, Chris Hegarty wrote: >>>>>> Alex, >>>>>> >>>>>> On 29/03/2019 22:07, Alex Menkov wrote: >>>>>>> (added net-dev as suggested by Alan) >>>>>>> >>>>>>> Net gurus, please assist in reviewing socket-related code. >>>>>>> >>>>>>> New webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/IPv6/webrev.01/ >>>>>> >>>>>> Specifically on SocketTransportService.java. What Arthur has >>>>>> proposed is better ( changing to lastIndexOf alone is not >>>>>> sufficient ). Or is your assumption that the IPv6 literal is >>>>>> not enclosed in square brackets? >>>>> >>>>> I didn't know about enclosing IPv6 in square brackets, but looks >>>>> like that's standard way to alleviate conflict between IPv6 address >>>>> and colon as port separator. >>>>> Will update the fix to handle them in both JDI connectors >>>>> (SocketTransportService.java) and debugger agent (socketTransport.c) >>>>> >>>>>> >>>>>> If keeping Arthur's `static class HostPort` please make the >>>>>> fields final. >>>>>> >>>>>> ?>> Would it make sense to support the preference properties? >>>>>> ?>>??? java.net.preferIPv4Stack >>>>>> ?>>??? java.net.preferIPv6Addresses >>>>>> >>>>>> I'm not sure about this, especially given the property name >>>>>> prefixes. I need to think a little more on it. >>>>> >>>>> In the initial version of the fix I didn't check the properties. >>>>> The rationale here is backward compatibility - is address is empty, >>>>> old debuggers tries to connect to IPv4 address only. >>>>> But handling this properties we will better handle clients with >>>>> properties set (as jdb or any other debugger which uses JDI >>>>> connectors are affected by the properties). >>>>> >>>>> BTW fix provided by Arthur implements listening on localhost >>>>> differently - it creates several sockets and binds them to both >>>>> IPv4 and IPv6 addresses. But the problem here (and that's the >>>>> reason I decide to not implement it this way) - how to handle the >>>>> case when we successfully bind on one address (for example IPv4), >>>>> but fail to bind on other (for example the port is busy for IPv6 >>>>> stack). Arthur's version just fail in the case (i.e. the whole Java >>>>> process terminates) and I don't think this is good behavior. >>>>> >>>>> >>>>>> >>>>>> There is quite a bit of new native code, is it possible to >>>>>> rewrite any of this in Java, e.g. reading of the system >>>>>> properties ( if that is to remain )? >>>>> >>>>> socketTransport.c is a debugger agent which is completely native. >>>>> >>>>> --alex >>>>> >>>>>> >>>>>> -Chris. >>> > From aeubanks at google.com Wed Apr 24 03:56:18 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Tue, 23 Apr 2019 20:56:18 -0700 Subject: [ipv6] On updating certificates in tests In-Reply-To: References: Message-ID: Thanks, I'll try it out. This seems very complicated for updating a cert though. On Tue, Apr 23, 2019, 5:22 AM Pavel Rappo wrote: > Arthur, > > Some time ago Chris mentioned [1] that there was at least one test failing > due > to missing SANs in its certificates: > > > test/jdk/sun/net/www/protocol/https/HttpsURLConnection/IPAddressIPIdentities.java > > You then replied that your were planning to address that issue in a future > change. > > I thought it might be helpful to you to get step-by-step instructions on > how > to change that test security setup, so you won't waste too much of your > time. > > Bear in mind that this is just one way to achieve that. This should work > reasonably well for not more than a handful of such tests. If you find more > tests relying on SAN in certificates, please get back on the list and > we'll try > figure out a better way of doing this together. I would not go there just > yet, > because I have a heavy suspicion it will require programmatic access to > keytool > and its security friends. It might cause restructuring the test > significantly. > So let's see the affected area first. > > Here we go: > > 1. Create a directory and cd into it (so everything is clean and tidy) > > mkdir my-certificates-for-tests && cd my-certificates-for-tests > > 2. Generate a self-signed CA certificate > > keytool -genkeypair \ > -keyalg RSA \ > -startdate 2019/01/01 \ > -validity 13000 \ > -keysize 1024 \ > -dname "C=XX, ST=CA-State, L=CA-City, O=CA-Org" \ > -ext bc=ca:true \ > -storetype PKCS12 \ > -alias root \ > -keystore temp.jks \ > -storepass 123456 > > 3. Generate a self-signed (for now) certificate for the server > > keytool -genkeypair \ > -keyalg RSA \ > -keysize 1024 \ > -dname "C=YY, ST=Server-State, L=Server-City, O=Server, > OU=Server-Unit, CN=Server-Name" \ > -alias server \ > -keystore temp.jks \ > -storepass 123456 > > 4. Sign that server certificate using the previously generated CA?s > certificate > > keytool -certreq \ > -alias server \ > -keystore temp.jks \ > -storepass 123456 \ > | keytool -gencert \ > -rfc \ > -startdate 2019/01/02 \ > -validity 12000 \ > -ext san:critical=ip:127.0.0.1,ip:0:0:0:0:0:0:0:1 \ > -alias root \ > -keystore temp.jks \ > -storepass 123456 \ > | keytool -importcert \ > -alias server \ > -keystore temp.jks \ > -storepass 123456 > > 5. Repeat the process for the client certificate > > keytool -genkeypair \ > -keyalg RSA \ > -keysize 1024 \ > -dname "C=ZZ, ST=Client-State, L=Client-City, O=Client, > OU=Client-Unit, CN=Client-Name" \ > -alias client \ > -keystore temp.jks \ > -storepass 123456 > > then > > keytool -certreq \ > -alias client \ > -keystore temp.jks \ > -storepass 123456 \ > | keytool -gencert \ > -rfc \ > -startdate 2019/01/02 \ > -validity 12000 \ > -ext san:critical=ip:127.0.0.1,ip:0:0:0:0:0:0:0:1 \ > -alias root \ > -keystore temp.jks \ > -storepass 123456 \ > | keytool -importcert \ > -alias client \ > -keystore temp.jks \ > -storepass 123456 > > Now, so as to update the test source, you will need to print the > certificates > and the keys. As far as I know, keytool cannot really do this (however, > there's > an API to which we might fall back later). For now you could simply use the > openssl tool. > > 6. Print the certificates > > keytool -list \ > -rfc \ > -alias root \ > -keystore temp.jks \ > -storepass 123456 \ > | openssl x509 -inform pem -text > > keytool -list \ > -rfc \ > -alias server \ > -keystore temp.jks \ > -storepass 123456 \ > | openssl x509 -inform pem -text > > keytool -list \ > -rfc \ > -alias client \ > -keystore temp.jks \ > -storepass 123456 \ > | openssl x509 -inform pem -text > > 7. Export the keys from the keystore > > openssl pkcs12 -in temp.jks -nodes -nocerts -out keys.pem -passin > pass:123456 > > 8. Manually split the resulting `keys.pem` file into 2: server.pem and > client.pem, discarding the root key. > > 9. Print the keys > > openssl rsa -in server.pem -text > > openssl rsa -in client.pem -text > > Now you should have all the data required to update the test. > > -Pavel > > > -------------------------------------------------------------------------------- > [1] https://mail.openjdk.java.net/pipermail/net-dev/2019-March/012311.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Apr 24 08:06:29 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 24 Apr 2019 09:06:29 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> Message-ID: Arthur, > On 23 Apr 2019, at 16:54, Arthur Eubanks wrote: > > ... > New webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html Looks good to me. -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Wed Apr 24 10:40:59 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 24 Apr 2019 11:40:59 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> Message-ID: <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Hi Arthur, The jdk.changeset file in your webrev looks completely wrong. http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/jdk.changeset I assume it's just garbage and we should ignore it. Can you confirm? WRT to the individual files changes listed at http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html In unix/native/libnet/net_util_md.c ipv4_supported(): I was wondering if we should be more selective about what kind of errors we should consider meaning that IPv4 is not available. But given that this method executes in OnLoad and that there is a precedent with ipv6_supported() I believe that what you have is fine. The rest looks very reasonable to me. Could you try to fix the jdk.changeset in the webrev so that we could try to import and test your proposed changes? best regards, -- daniel On 23/04/2019 16:54, Arthur Eubanks wrote: > I believe that `DONT_ENABLE_IPV4`is not strictly required. Maybe just > drop it?? Is it an initial attempt to support an IPv6-only JDK build? > If so, then maybe we separate out that requirement. > > It was in the IPv6 code right below so I just copied it in case we > wanted an IPv6-only build at some point. But yes that can be in a > separate change. Removed. > New webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html From aeubanks at google.com Wed Apr 24 15:51:28 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Wed, 24 Apr 2019 08:51:28 -0700 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Message-ID: > > The jdk.changeset file in your webrev looks completely wrong. > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/jdk.changeset > I assume it's just garbage and we should ignore it. > Can you confirm? > Yes sorry, that's just garbage. Updated webrev: http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.03/index.html (no change to the patch itself) I have already run this through the submit repo and it doesn't break anything. -------------- next part -------------- An HTML attachment was scrubbed... URL: From macanaoire at hotmail.com Wed Apr 24 16:03:13 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Wed, 24 Apr 2019 16:03:13 +0000 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> , <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Message-ID: an observation on IPv4_supported and IPV6_supported for your consideration both, IPv4_support and IPv6_support use socket creation in the appropriate AF domain as a a verification of support, but the v6 version also checks that there is a set of address binding or ipv6 address configuration by accessing /proc/net/if_inet6 afaik there is no equivalent /proc/net/if_inet or if_inet4, thus the ipv4 version more liberal in its deliberation. as such, it is possible that for IPv4 the support is available but not yet configured, but that has no impact on availability. For IPv6 that the support is enabled but not yet configured and that will impact the availability evaluation. Implicit in this is the /proc/net/if_inet6 considered absolutely necessary ? this has at times been the cause of spurious intermittent test failures if my memory is correct. Another consideration is that if an application starts when IPv6 has yet to be configured properly, it will most likely lead to the application requiring a restart after the IPv6 to access IPv6 functionality regards Mark ________________________________ From: net-dev on behalf of Daniel Fuchs Sent: Wednesday 24 April 2019 10:40 To: Arthur Eubanks; Chris Hegarty Cc: OpenJDK Network Dev list Subject: Re: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) Hi Arthur, The jdk.changeset file in your webrev looks completely wrong. http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/jdk.changeset I assume it's just garbage and we should ignore it. Can you confirm? WRT to the individual files changes listed at http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html In unix/native/libnet/net_util_md.c ipv4_supported(): I was wondering if we should be more selective about what kind of errors we should consider meaning that IPv4 is not available. But given that this method executes in OnLoad and that there is a precedent with ipv6_supported() I believe that what you have is fine. The rest looks very reasonable to me. Could you try to fix the jdk.changeset in the webrev so that we could try to import and test your proposed changes? best regards, -- daniel On 23/04/2019 16:54, Arthur Eubanks wrote: > I believe that `DONT_ENABLE_IPV4`is not strictly required. Maybe just > drop it? Is it an initial attempt to support an IPv6-only JDK build? > If so, then maybe we separate out that requirement. > > It was in the IPv6 code right below so I just copied it in case we > wanted an IPv6-only build at some point. But yes that can be in a > separate change. Removed. > New webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.02/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeubanks at google.com Wed Apr 24 16:22:04 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Wed, 24 Apr 2019 09:22:04 -0700 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Message-ID: On Wed, Apr 24, 2019 at 9:03 AM mark sheppard wrote: > an observation on IPv4_supported and IPV6_supported for your consideration > > > > both, IPv4_support and IPv6_support use socket creation in the > appropriate AF domain as a > a verification of support, but the v6 version also checks that there is a > set of > address binding or ipv6 address configuration by accessing > /proc/net/if_inet6 > > afaik there is no equivalent /proc/net/if_inet or if_inet4, thus the ipv4 > version more > liberal in its deliberation. > > as such, it is possible that for IPv4 the support is available but not yet > configured, > but that has no impact on availability. > For IPv6 that the support is enabled but not yet configured and that will > impact the > availability evaluation. > > Implicit in this is the /proc/net/if_inet6 considered absolutely > necessary ? > > this has at times been the cause of spurious intermittent test failures if > my memory is correct. > > Another consideration is that if an application starts when IPv6 has yet > to be configured properly, > it will most likely lead to the application requiring a restart after the > IPv6 to access IPv6 functionality > > > > > regards > Mark > IIUC, you're saying that the IPv6_supported() may return false when it should return true in some cases? That's not directly related to this change, since this change doesn't touch IPv6_supported(). A repro of intermittent test failures or discussion about this in the past would be good. Maybe you could file a bug or propose a patch? -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Thu Apr 25 11:11:42 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 25 Apr 2019 12:11:42 +0100 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Message-ID: Thanks Arthur! I think this is fine. I have done some testing and observed no regression. I believe you should also add the noreg-hard label to 8222562 - since it requires a special set-up to verify that the issue has been fixed. As far as I'm concerned you're good to go with this one! best regards, -- daniel On 24/04/2019 16:51, Arthur Eubanks wrote: > Yes sorry, that's just garbage. > Updated webrev: > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.03/index.html > (no change to the patch itself) > > I have already run this through the submit repo and it doesn't break > anything. From aeubanks at google.com Thu Apr 25 20:18:15 2019 From: aeubanks at google.com (Arthur Eubanks) Date: Thu, 25 Apr 2019 13:18:15 -0700 Subject: [ipv6] Re: [RFR]: 8222562: IPv6 only systems fail on setsockopt(IPV6_V6ONLY, 0) In-Reply-To: References: <86c4e5fb-6f1f-5975-bc14-8d3d043931d3@oracle.com> <84f30f71-060a-92c3-39de-506c755ddc3c@oracle.com> <236ff699-cf6c-a1dc-17c4-793939162614@oracle.com> Message-ID: On Thu, Apr 25, 2019 at 4:11 AM Daniel Fuchs wrote: > Thanks Arthur! > > I think this is fine. I have done some testing and observed no > regression. I believe you should also add the noreg-hard > label to 8222562 - since it requires a special set-up to > verify that the issue has been fixed. > Done. > > As far as I'm concerned you're good to go with this one! > Thanks for the reviews! > > best regards, > > -- daniel > > On 24/04/2019 16:51, Arthur Eubanks wrote: > > Yes sorry, that's just garbage. > > Updated webrev: > > http://cr.openjdk.java.net/~aeubanks/ipv6setsockopt/webrev.03/index.html > > (no change to the patch itself) > > > > I have already run this through the submit repo and it doesn't break > > anything. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Fri Apr 26 13:33:30 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 26 Apr 2019 14:33:30 +0100 Subject: RFR: 8129315: java/net/Socket/LingerTest.java and java/net/Socket/ShutdownBoth.java timeout intermittently Message-ID: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> Hi, Please find below a test stabilization fix for: 8129315: java/net/Socket/LingerTest.java and java/net/Socket/ShutdownBoth.java timeout intermittently http://cr.openjdk.java.net/~dfuchs/webrev_8129315/webrev.00/index.html While testing I stumbled on some other tests failing intermittently for the fame reasons and went ahead and fixed some of them too. Tests fixed in this batch: test/jdk/java/net/Socket/LingerTest.java test/jdk/java/net/Socket/ShutdownBoth.java test/jdk/java/net/Socks/SocksIPv6Test.java test/jdk/sun/net/www/http/HttpURLConnection/PostOnDelete.java The crux of the issue is that using the wildcard address to bind test servers makes them susceptible of receiving traffic from other processes/tests on the same machine, and make them also susceptible of not receiving traffic that was intended for them, if some other process has managed to bind on a specific address with the same port number. I have started a longer background investigation task to identify and fix the other tests that may present the same symptoms, so more RFRs like this one will probably come in the future. best regards, -- daniel From chris.hegarty at oracle.com Fri Apr 26 14:56:24 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 26 Apr 2019 15:56:24 +0100 Subject: RFR: 8129315: java/net/Socket/LingerTest.java and java/net/Socket/ShutdownBoth.java timeout intermittently In-Reply-To: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> References: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> Message-ID: Daniel, On 26/04/2019 14:33, Daniel Fuchs wrote: > Hi, > > Please find below a test stabilization fix for: > > 8129315: java/net/Socket/LingerTest.java and > ???????? java/net/Socket/ShutdownBoth.java timeout intermittently > http://cr.openjdk.java.net/~dfuchs/webrev_8129315/webrev.00/index.html This looks good. > While testing I stumbled on some other tests failing > intermittently for the fame reasons and went ahead and > fixed some of them too. Tests fixed in this batch: > > test/jdk/java/net/Socket/LingerTest.java > test/jdk/java/net/Socket/ShutdownBoth.java > test/jdk/java/net/Socks/SocksIPv6Test.java > test/jdk/sun/net/www/http/HttpURLConnection/PostOnDelete.java > > The crux of the issue is that using the wildcard address to > bind test servers makes them susceptible of receiving traffic from > other processes/tests on the same machine, and make them also > susceptible of not receiving traffic that was intended for them, > if some other process has managed to bind on a specific address > with the same port number. > > I have started a longer background investigation task to identify > and fix the other tests that may present the same symptoms, so > more RFRs like this one will probably come in the future. Good strategy, small(ish) incremental improvements are easier to review and improve test stability in a more timely manner. -Chris. From Alan.Bateman at oracle.com Sat Apr 27 14:29:46 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 27 Apr 2019 15:29:46 +0100 Subject: RFR: 8129315: java/net/Socket/LingerTest.java and java/net/Socket/ShutdownBoth.java timeout intermittently In-Reply-To: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> References: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> Message-ID: <5108bf69-3746-9cf6-7bde-97418b461af8@oracle.com> On 26/04/2019 14:33, Daniel Fuchs wrote: > Hi, > > Please find below a test stabilization fix for: > > 8129315: java/net/Socket/LingerTest.java and > ???????? java/net/Socket/ShutdownBoth.java timeout intermittently > http://cr.openjdk.java.net/~dfuchs/webrev_8129315/webrev.00/index.html > One minor comment is that the 3-arg constructor requires you to specify a value for the backlog (or <= 0 to use an implementation default). An alternative that I used in other tests is to use the no-arg constructor and then bind to the loopback. It just avoids need to think about the backlog, otherwise the same. -Alan From daniel.fuchs at oracle.com Mon Apr 29 08:43:02 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 29 Apr 2019 09:43:02 +0100 Subject: RFR: 8129315: java/net/Socket/LingerTest.java and java/net/Socket/ShutdownBoth.java timeout intermittently In-Reply-To: <5108bf69-3746-9cf6-7bde-97418b461af8@oracle.com> References: <04f3743e-37e9-b945-2876-ce7b486d91df@oracle.com> <5108bf69-3746-9cf6-7bde-97418b461af8@oracle.com> Message-ID: <2caad085-fe54-518b-7c64-7f672ee9dcea@oracle.com> Thanks Alan! I have already pushed those changes - but I'll retain this trick for the next batch. FWIW I saw that the default value for the backlog in the impl was 50, so that's what I'd been using. But not having to specify it at all is indeed a better solution. best regards, -- daniel On 27/04/2019 15:29, Alan Bateman wrote: > One minor comment is that the 3-arg constructor requires you to specify > a value for the backlog (or <= 0 to use an implementation default). An > alternative that I used in other tests is to use the no-arg constructor > and then bind to the loopback. It just avoids need to think about the > backlog, otherwise the same. From michael.x.mcmahon at oracle.com Mon Apr 29 09:52:14 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 10:52:14 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations Message-ID: <5CC6C94E.8010300@oracle.com> Hi, This is another change which is part of the general cleanup of SocketImpls. The change removes support for pre JDK 1.4 socketimpls which do not implement the timed connect method. These SocketImpls have not been compilable since 1.4 and limited runtime support has been provided since then, which is now being removed. Webrev ------- http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ CSR ---- https://bugs.openjdk.java.net/browse/JDK-8222546 Thanks, Michael. From chris.hegarty at oracle.com Mon Apr 29 10:26:31 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 29 Apr 2019 11:26:31 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC6C94E.8010300@oracle.com> References: <5CC6C94E.8010300@oracle.com> Message-ID: <752df375-5cf6-7179-37e2-6fd89c8a0b37@oracle.com> Michael, On 29/04/2019 10:52, Michael McMahon wrote: > Hi, > > This is another change which is part of the general cleanup of SocketImpls. > The change removes support for pre JDK 1.4 socketimpls which do not > implement > the timed connect method. These SocketImpls have not been compilable > since 1.4 and limited runtime support has been provided since then, > which is now being removed. > > Webrev > ------- > http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ Mostly looks good. A few specific comments: Socket.java Please remove these comments, as they are no longer applicable: 1675 // Before 1.3 Sockets were always connected during creation 1692 // Before 1.3 Sockets were always bound during creation ServerSocket.java Please remove: 75 * Are we using an older SocketImpl? 76 */ 77 private boolean oldImpl = false; And there is still one reference to it: 347 if (!oldImpl && isBound()) SocketImpl.java Please remove: Defaults to false, unless setIsServer() called 79 * from ServerSocket -Chris. From daniel.fuchs at oracle.com Mon Apr 29 10:34:21 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 29 Apr 2019 11:34:21 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC6C94E.8010300@oracle.com> References: <5CC6C94E.8010300@oracle.com> Message-ID: Hi Michael, I'm too new to networking libs to actually review this change. However I have eyeballed it and it looks like a very nice simplification and cleanup! I didn't see anything that looked wrong. Two thing though: java/net/Socket.java: (and at multiple other places in this file) 1615 // Before 1.3 Sockets were always connected during creation I believe that with your change this comment is now obsolete. Maybe it should be altered / removed? AbsctractSocketImpl/PlainSocketImpl: It's a bit surprising that createSocket has now acquired an isServer boolean, as it makes it look as if a value different than that given to the constructor could be passed in. Aren't the PlainSocketImpl subclasses all able to access this `isServer` field? I understand that the unix impl needs to pass the value of that field to the native, but maybe it could simply have a one arg java createSocket method that calls the underlying two args native impl? best regards, -- daniel On 29/04/2019 10:52, Michael McMahon wrote: > Hi, > > This is another change which is part of the general cleanup of SocketImpls. > The change removes support for pre JDK 1.4 socketimpls which do not > implement > the timed connect method. These SocketImpls have not been compilable > since 1.4 and limited runtime support has been provided since then, > which is now being removed. > > Webrev > ------- > http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ > > CSR > ---- > https://bugs.openjdk.java.net/browse/JDK-8222546 > > > Thanks, > Michael. From michael.x.mcmahon at oracle.com Mon Apr 29 10:57:32 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 11:57:32 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <752df375-5cf6-7179-37e2-6fd89c8a0b37@oracle.com> References: <5CC6C94E.8010300@oracle.com> <752df375-5cf6-7179-37e2-6fd89c8a0b37@oracle.com> Message-ID: <5CC6D89C.4070306@oracle.com> Thanks Chris. Comments noted. - Michael On 29/04/2019, 11:26, Chris Hegarty wrote: > Michael, > > On 29/04/2019 10:52, Michael McMahon wrote: >> Hi, >> >> This is another change which is part of the general cleanup of >> SocketImpls. >> The change removes support for pre JDK 1.4 socketimpls which do not >> implement >> the timed connect method. These SocketImpls have not been compilable >> since 1.4 and limited runtime support has been provided since then, >> which is now being removed. >> >> Webrev >> ------- >> http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ > > Mostly looks good. A few specific comments: > > Socket.java > Please remove these comments, as they are no longer applicable: > > 1675 // Before 1.3 Sockets were always connected during creation > 1692 // Before 1.3 Sockets were always bound during creation > > ServerSocket.java > Please remove: > 75 * Are we using an older SocketImpl? > 76 */ > 77 private boolean oldImpl = false; > And there is still one reference to it: > 347 if (!oldImpl && isBound()) > > SocketImpl.java > Please remove: > Defaults to false, unless setIsServer() called > 79 * from ServerSocket > > -Chris. From michael.x.mcmahon at oracle.com Mon Apr 29 11:06:21 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 12:06:21 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: References: <5CC6C94E.8010300@oracle.com> Message-ID: <5CC6DAAD.50302@oracle.com> Hi Daniel, On 29/04/2019, 11:34, Daniel Fuchs wrote: > Hi Michael, > > I'm too new to networking libs to actually review > this change. However I have eyeballed it and it looks > like a very nice simplification and cleanup! > I didn't see anything that looked wrong. > > Two thing though: > > java/net/Socket.java: (and at multiple other places in this file) > > 1615 // Before 1.3 Sockets were always connected during creation > > I believe that with your change this comment is now obsolete. > Maybe it should be altered / removed? > > AbsctractSocketImpl/PlainSocketImpl: > > It's a bit surprising that createSocket has now acquired > an isServer boolean, as it makes it look as if a value > different than that given to the constructor could be > passed in. Aren't the PlainSocketImpl subclasses all able > to access this `isServer` field? > > I understand that the unix impl needs to pass the value > of that field to the native, but maybe it could simply > have a one arg java createSocket method that calls > the underlying two args native impl? > I think that's a fair point. Actually, the 'isServer' field was added after other changes had already been made. But, particularly since the field/parameter is not used on windows then it could be refactored such that it doesn't appear in Windows and the abstract socketCreate() method would not need it either. Thanks, Michael. From Alan.Bateman at oracle.com Mon Apr 29 11:17:30 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 29 Apr 2019 12:17:30 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC6C94E.8010300@oracle.com> References: <5CC6C94E.8010300@oracle.com> Message-ID: <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> On 29/04/2019 10:52, Michael McMahon wrote: > Hi, > > This is another change which is part of the general cleanup of > SocketImpls. > The change removes support for pre JDK 1.4 socketimpls which do not > implement > the timed connect method. These SocketImpls have not been compilable > since 1.4 and limited runtime support has been provided since then, > which is now being removed. > > Webrev > ------- > http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ > This is a good cleanup. Changing SIS.close and SOS.close to caller super.close raises a number of questions. These close should never be called Socket.getInputStream and getOutputStream don't leak these streams to user code (they used to but now in JDK 13). My concern is that if they were ever to be called then it will be calling the FIS/FOS close methods which brings along a several questions on it interacts with the cleaner mechanism used by those classes. I don't think AbstractPlainSocketImpl.isBound needs to be volatile as it is guarded by the synchronization on the impl (the doConnect and bind methods are synchronized). The Windows version of PlainSocketImpl shouldn't need the constructor to be public (I see the Unix version is package-private). Should we use the opportunity to add something to the javadoc for the SocketImpl constructor so it's not empty? It can be as simple as "Initialize a new instance of this class" as used in other places where there isn't anything to say. -Alan. From daniel.fuchs at oracle.com Mon Apr 29 12:00:08 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 29 Apr 2019 13:00:08 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> Message-ID: <68d9d438-9bd8-9b5d-7760-2ccddfdba29c@oracle.com> Hi Alan, On 29/04/2019 12:17, Alan Bateman wrote: > I don't think AbstractPlainSocketImpl.isBound needs to be volatile as it > is guarded by the synchronization on the impl (the doConnect and bind > methods are synchronized). I see that it is set outside of any synchronized block in AbstractPlainSocketImpl::bind 433 protected synchronized void bind(InetAddress address, int lport) 434 throws IOException 435 { 436 synchronized (fdLock) { 437 if (!closePending && !isBound) { 438 NetHooks.beforeTcpBind(fd, address, lport); 439 } 440 } 441 socketBind(address, lport); 442 isBound = true; 443 } which for me justifies that it should be volatile. best regards, -- daniel From chris.hegarty at oracle.com Mon Apr 29 12:08:40 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 29 Apr 2019 13:08:40 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> Message-ID: On 29/04/2019 12:17, Alan Bateman wrote: > ... > Changing SIS.close and SOS.close to caller super.close raises a number > of questions. These close should never be called Socket.getInputStream > and getOutputStream don't leak these streams to user code (they used to > but now in JDK 13). My concern is that if they were ever to be called > then it will be calling the FIS/FOS close methods which brings along a > several questions on it interacts with the cleaner mechanism used by > those classes. Since 8220493, these socket in/out stream close methods are effectively no longer in charge of closing the socket ( since that will happen from the outer stream wrapper in Socket ). I wonder if they should simply call impl.close()? -Chris. From Alan.Bateman at oracle.com Mon Apr 29 12:24:57 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 29 Apr 2019 13:24:57 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <68d9d438-9bd8-9b5d-7760-2ccddfdba29c@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> <68d9d438-9bd8-9b5d-7760-2ccddfdba29c@oracle.com> Message-ID: <628c1f54-80c7-6aae-e92c-9e38bb7bee52@oracle.com> On 29/04/2019 13:00, Daniel Fuchs wrote: > Hi Alan, > > On 29/04/2019 12:17, Alan Bateman wrote: >> I don't think AbstractPlainSocketImpl.isBound needs to be volatile as >> it is guarded by the synchronization on the impl (the doConnect and >> bind methods are synchronized). > > I see that it is set outside of any synchronized block > in AbstractPlainSocketImpl::bind > > ?433???? protected synchronized void bind(InetAddress address, int lport) > ?434???????? throws IOException > ?435???? { > ?436??????? synchronized (fdLock) { > ?437???????????? if (!closePending && !isBound) { > ?438???????????????? NetHooks.beforeTcpBind(fd, address, lport); > ?439???????????? } > ?440???????? } > ?441???????? socketBind(address, lport); > ?442???????? isBound = true; > > > > ?443???? } > > which for me justifies that it should be volatile. I think you are might be mixing up the lock on the vs. fdLock. From what I can tell, isBound is only accessed by doConnect and bind and they are both synchronized methods. -Alan From daniel.fuchs at oracle.com Mon Apr 29 13:10:00 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 29 Apr 2019 14:10:00 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <628c1f54-80c7-6aae-e92c-9e38bb7bee52@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> <68d9d438-9bd8-9b5d-7760-2ccddfdba29c@oracle.com> <628c1f54-80c7-6aae-e92c-9e38bb7bee52@oracle.com> Message-ID: On 29/04/2019 13:24, Alan Bateman wrote: >> ?433???? protected synchronized void bind(InetAddress address, int lport) [...] >> which for me justifies that it should be volatile. > I think you are might be mixing up the lock on the vs. fdLock. From what > I can tell, isBound is only accessed by doConnect and bind and they are > both synchronized methods Darn. My mistake. I see it now :-( -- daniel From michael.x.mcmahon at oracle.com Mon Apr 29 15:54:25 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 16:54:25 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> Message-ID: <5CC71E31.8050400@oracle.com> On 29/04/2019, 13:08, Chris Hegarty wrote: > > On 29/04/2019 12:17, Alan Bateman wrote: >> ... >> Changing SIS.close and SOS.close to caller super.close raises a >> number of questions. These close should never be called >> Socket.getInputStream and getOutputStream don't leak these streams to >> user code (they used to but now in JDK 13). My concern is that if >> they were ever to be called then it will be calling the FIS/FOS close >> methods which brings along a several questions on it interacts with >> the cleaner mechanism used by those classes. > > Since 8220493, these socket in/out stream close methods are effectively > no longer in charge of closing the socket ( since that will happen from > the outer stream wrapper in Socket ). I wonder if they should simply > call impl.close()? > On that point. Since it might not be obvious that SIS.close and SOS.close are no longer used, it might be more useful to just replace the close implementations with a comment and an assertion which underlines that fact. - Michael From david.lloyd at redhat.com Mon Apr 29 16:35:22 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Mon, 29 Apr 2019 11:35:22 -0500 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC71E31.8050400@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> <5CC71E31.8050400@oracle.com> Message-ID: On Mon, Apr 29, 2019 at 10:54 AM Michael McMahon wrote: > > > > On 29/04/2019, 13:08, Chris Hegarty wrote: > > > > On 29/04/2019 12:17, Alan Bateman wrote: > >> ... > >> Changing SIS.close and SOS.close to caller super.close raises a > >> number of questions. These close should never be called > >> Socket.getInputStream and getOutputStream don't leak these streams to > >> user code (they used to but now in JDK 13). My concern is that if > >> they were ever to be called then it will be calling the FIS/FOS close > >> methods which brings along a several questions on it interacts with > >> the cleaner mechanism used by those classes. > > > > Since 8220493, these socket in/out stream close methods are effectively > > no longer in charge of closing the socket ( since that will happen from > > the outer stream wrapper in Socket ). I wonder if they should simply > > call impl.close()? > > > > On that point. Since it might not be obvious that SIS.close and > SOS.close are no longer used, > it might be more useful to just replace the close implementations with a > comment and an assertion > which underlines that fact. As someone who does a lot of socket programming (albeit mostly non-blocking), I would be *shocked* if closing a socket input or output stream didn't translate into shutting down input or output (respectively). Not close per se but socket shutdown. -- - DML From michael.x.mcmahon at oracle.com Mon Apr 29 16:47:49 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 17:47:49 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> <5CC71E31.8050400@oracle.com> Message-ID: <5CC72AB5.5090001@oracle.com> On 29/04/2019, 17:35, David Lloyd wrote: > On Mon, Apr 29, 2019 at 10:54 AM Michael McMahon > wrote: >> >> >> On 29/04/2019, 13:08, Chris Hegarty wrote: >>> On 29/04/2019 12:17, Alan Bateman wrote: >>>> ... >>>> Changing SIS.close and SOS.close to caller super.close raises a >>>> number of questions. These close should never be called >>>> Socket.getInputStream and getOutputStream don't leak these streams to >>>> user code (they used to but now in JDK 13). My concern is that if >>>> they were ever to be called then it will be calling the FIS/FOS close >>>> methods which brings along a several questions on it interacts with >>>> the cleaner mechanism used by those classes. >>> Since 8220493, these socket in/out stream close methods are effectively >>> no longer in charge of closing the socket ( since that will happen from >>> the outer stream wrapper in Socket ). I wonder if they should simply >>> call impl.close()? >>> >> On that point. Since it might not be obvious that SIS.close and >> SOS.close are no longer used, >> it might be more useful to just replace the close implementations with a >> comment and an assertion >> which underlines that fact. > As someone who does a lot of socket programming (albeit mostly > non-blocking), I would be *shocked* if closing a socket input or > output stream didn't translate into shutting down input or output > (respectively). Not close per se but socket shutdown. It still ends up as a close of the socket's file descriptor at the OS level one way or the other. Closing a socket's InputStream or OutputStream never resulted in a shutdown() call to the OS. If you want socket shutdown then you need to call shutdownInput() or shutdownOutput API on j.n.Socket directly. - Michael. From michael.x.mcmahon at oracle.com Mon Apr 29 17:07:59 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 29 Apr 2019 18:07:59 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> Message-ID: <5CC72F6F.20308@oracle.com> On 29/04/2019, 12:17, Alan Bateman wrote: > On 29/04/2019 10:52, Michael McMahon wrote: >> Hi, >> >> This is another change which is part of the general cleanup of >> SocketImpls. >> The change removes support for pre JDK 1.4 socketimpls which do not >> implement >> the timed connect method. These SocketImpls have not been compilable >> since 1.4 and limited runtime support has been provided since then, >> which is now being removed. >> >> Webrev >> ------- >> http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ >> > This is a good cleanup. > > Changing SIS.close and SOS.close to caller super.close raises a number > of questions. These close should never be called Socket.getInputStream > and getOutputStream don't leak these streams to user code (they used > to but now in JDK 13). My concern is that if they were ever to be > called then it will be calling the FIS/FOS close methods which brings > along a several questions on it interacts with the cleaner mechanism > used by those classes. > > I don't think AbstractPlainSocketImpl.isBound needs to be volatile as > it is guarded by the synchronization on the impl (the doConnect and > bind methods are synchronized). > ok > The Windows version of PlainSocketImpl shouldn't need the constructor > to be public (I see the Unix version is package-private). > ok > Should we use the opportunity to add something to the javadoc for the > SocketImpl constructor so it's not empty? It can be as simple as > "Initialize a new instance of this class" as used in other places > where there isn't anything to say. > The empty comment block looks odd I admit. It was needed to make it build and I left it empty so that the generated apidoc would not change. However, I guess I can add the above suggestion to the CSR Thanks, Michael From Alan.Bateman at oracle.com Mon Apr 29 17:15:57 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 29 Apr 2019 18:15:57 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC72AB5.5090001@oracle.com> References: <5CC6C94E.8010300@oracle.com> <866ea538-b04e-2714-da9a-fa48b9c70b34@oracle.com> <5CC71E31.8050400@oracle.com> <5CC72AB5.5090001@oracle.com> Message-ID: <920e6a0b-9eb3-580b-2dd4-1ad6fb6e4fb0@oracle.com> On 29/04/2019 17:47, Michael McMahon wrote: > : > > It still ends up as a close of the socket's file descriptor at the OS > level > one way or the other. Closing a socket's InputStream or OutputStream > never resulted in a shutdown() call to the OS. If you want socket > shutdown > then you need to call shutdownInput() or shutdownOutput API on > j.n.Socket directly. Right, I suspect David may have mis-read the discussion as it may not be obvious that Socket::getInputStream and Socket::getOutputStream return streams that implement close to close the Socket. One detail on shutdown is that the close may shutdown the stream for writing as part of the close. This is normal and long standing behavior for cases where SO_LINGER has not been set. -Alan From michael.x.mcmahon at oracle.com Tue Apr 30 09:53:35 2019 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 30 Apr 2019 10:53:35 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC6C94E.8010300@oracle.com> References: <5CC6C94E.8010300@oracle.com> Message-ID: <5CC81B1F.1060100@oracle.com> Thanks for all the comments. A new webrev is at: http://cr.openjdk.java.net/~michaelm/8216978/webrev.2/index.html The CSR now also includes the minor API doc update suggested for the no-arg SocketImpl constructor. Thanks, Michael On 29/04/2019, 10:52, Michael McMahon wrote: > Hi, > > This is another change which is part of the general cleanup of > SocketImpls. > The change removes support for pre JDK 1.4 socketimpls which do not > implement > the timed connect method. These SocketImpls have not been compilable > since 1.4 and limited runtime support has been provided since then, > which is now being removed. > > Webrev > ------- > http://cr.openjdk.java.net/~michaelm/8216978/webrev.1/ > > CSR > ---- > https://bugs.openjdk.java.net/browse/JDK-8222546 > > > Thanks, > Michael. From Alan.Bateman at oracle.com Tue Apr 30 10:34:48 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Apr 2019 11:34:48 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC81B1F.1060100@oracle.com> References: <5CC6C94E.8010300@oracle.com> <5CC81B1F.1060100@oracle.com> Message-ID: <3c6e5283-433c-f331-9aa1-a9f7cde1ed7b@oracle.com> On 30/04/2019 10:53, Michael McMahon wrote: > Thanks for all the comments. A new webrev is at: > > http://cr.openjdk.java.net/~michaelm/8216978/webrev.2/index.html > > The CSR now also includes the minor API doc update suggested for the > no-arg SocketImpl constructor. This looks okay to me. -Alan From chris.hegarty at oracle.com Tue Apr 30 11:12:47 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 30 Apr 2019 12:12:47 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC81B1F.1060100@oracle.com> References: <5CC6C94E.8010300@oracle.com> <5CC81B1F.1060100@oracle.com> Message-ID: <91342BB0-529B-47AA-BF7C-5A3D4696C7A5@oracle.com> > On 30 Apr 2019, at 10:53, Michael McMahon wrote: > > Thanks for all the comments. A new webrev is at: > > http://cr.openjdk.java.net/~michaelm/8216978/webrev.2/index.html Looks good. -Chris. From daniel.fuchs at oracle.com Tue Apr 30 11:16:19 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 30 Apr 2019 12:16:19 +0100 Subject: RFR: 8223145: [teststabilization] Replace wildcard address with loopback or local host in test - part 1 Message-ID: <19405da6-e153-ce73-76e8-82742a296654@oracle.com> Hi, Please find below a patch for: 8223145: [teststabilization] Replace wildcard address with loopback or local host in test - part 1 https://bugs.openjdk.java.net/browse/JDK-8223145 http://cr.openjdk.java.net/~dfuchs/webrev_8223145/webrev.00/ This is the first in a series of patches that will try to address intermittent failures which are sometime observed when tests use the wildcard address to bind their test servers. These changes are IpV6-only friendly. best regards, -- daniel From daniel.fuchs at oracle.com Tue Apr 30 11:17:21 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 30 Apr 2019 12:17:21 +0100 Subject: RFR 8216978: Drop support for pre JDK 1.4 SocketImpl implementations In-Reply-To: <5CC81B1F.1060100@oracle.com> References: <5CC6C94E.8010300@oracle.com> <5CC81B1F.1060100@oracle.com> Message-ID: <2cec4775-684f-4a4b-bf3d-cdff42b4eccc@oracle.com> On 30/04/2019 10:53, Michael McMahon wrote: > Thanks for all the comments. A new webrev is at: > > http://cr.openjdk.java.net/~michaelm/8216978/webrev.2/index.html Looks good Michael! cheers, -- daniel From sean.coffey at oracle.com Tue Apr 30 17:19:31 2019 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 30 Apr 2019 18:19:31 +0100 Subject: RFR: 8217364: Custom URLStreamHandler for jrt or file protocol can override default handler Message-ID: <1d37762f-7f17-6094-6193-885af8d72b72@oracle.com> Looking to correct an issue that arose during the JDK-8213942 fix. https://bugs.openjdk.java.net/browse/JDK-8217364 https://cr.openjdk.java.net/~coffeys/webrev.8217364/webrev/ regards, Sean. From macanaoire at hotmail.com Tue Apr 30 22:58:40 2019 From: macanaoire at hotmail.com (mark sheppard) Date: Tue, 30 Apr 2019 22:58:40 +0000 Subject: RFR: 8223145: [teststabilization] Replace wildcard address with loopback or local host in test - part 1 In-Reply-To: <19405da6-e153-ce73-76e8-82742a296654@oracle.com> References: <19405da6-e153-ce73-76e8-82742a296654@oracle.com> Message-ID: ? Hi Daniel,? ? interesting set of changes -? But could it be the case that, for some tests, you might change the operational semantics of a test, when? this applying this change. For example, in the case of GetLocalAddress.? The original is to use a wild card for the server, and a directed address for the ? client connect. Thus the server is listening INADDR_ANY or the IPv6 equivalent, accepts? a connection, then a check is made to see that socket created is bound to the same address? as that specified in the clients connect request.? ? If the server socket is bound to a specific address then the socket created by the accept? will (always) have the same address as that of initial server socket (listener).? ? As the tests are run with 3 different java.net property settings, it is worth asking if the tests behaves as expected? for the case when neither java.net property is set.? AFAIK, on windows it shows some ambiguity and difference to the default setting described in? [1]. The default run returns IPV4 addresses, creates AF_INTET sockets and binds to IPv4 addresses.? This is possibly at odds with the default property settings are described in the docs.? ? java.net.preferIPv4Stack (default: false)? If IPv6 is available on the operating system the underlying native socket will be, by default,? an IPv6 socket which lets applications connect to, and accept connections from, both IPv4 and IPv6 hosts. ? This implies that the first test run, i.e. default property settings, should be an AF_INET6 socket as in new ServerSocket? should be bound to the IPv6 unspecified address.? ? While? java.net.preferIPv6Addresses (default: false)? When dealing with a host which has both IPv4 and IPv6 addresses, ? and if IPv6 is available on the operating system, the default ? behavior is to prefer using IPv4 addresses over IPv6 ones.? ? This implies that the InetAddress.getLocalHost() should in the default case? return an IPv4 address.? ? The client socket connect with IPv4 address to IPv6 wild card resulting in an IPv4 mapped? binding in the socket created in the accept. This would mean the first run should fail.? ? In any case a test using a server socket listening on a wildcard (INADDR_ANY) address has possibly? slightly different semantics than a server socket listening on a specific host address.? ? regards Mark ? ? ? [1] https://docs.oracle.com/javase/8/docs/api/java/net/doc-files/net-properties.html ________________________________ From: net-dev on behalf of Daniel Fuchs Sent: Tuesday 30 April 2019 11:16 To: OpenJDK Network Dev list Subject: RFR: 8223145: [teststabilization] Replace wildcard address with loopback or local host in test - part 1 Hi, Please find below a patch for: 8223145: [teststabilization] Replace wildcard address with loopback or local host in test - part 1 https://bugs.openjdk.java.net/browse/JDK-8223145 http://cr.openjdk.java.net/~dfuchs/webrev_8223145/webrev.00/ This is the first in a series of patches that will try to address intermittent failures which are sometime observed when tests use the wildcard address to bind their test servers. These changes are IpV6-only friendly. best regards, -- daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: