From wenboz at google.com Thu Oct 1 01:02:50 2015 From: wenboz at google.com (Wenbo Zhu) Date: Wed, 30 Sep 2015 18:02:50 -0700 Subject: WebSocket client API In-Reply-To: References: Message-ID: Thanks for sharing this, and my initial feedback as following: === Optionally, keep the API simpler and match the JS API (W3C) 1. remove sendAsync .. send() should be async ** 2. remove suspend/resume 3. do not involve HTTPClient and support explicit handshake methods 4. remove createAsync ... and create() should be async, with an onOpen callback ** leave flow-control completely to applications ... === Alternatively or additionally: support "standard" blocking I/O, e.g. WebSocketInputChannel WebSocket.getInputChannel() WebSocketOutputChannel WebSocket.getOutputChannel) interface WebSocketChannel extends InterruptableChannel === Style issues 1. keep Incoming and Chunks two separate APIs? 2. is OutGoing necessary? Is it better to have 4 overloaded send() methods? === Nice to have: 1. expose ping-pong messages On Mon, Aug 31, 2015 at 7:30 AM, Pavel Rappo wrote: > Hi, > > I would appreciate if you help to review a WebSocket client API proposed > for > JDK 9. This work is a part of JEP-110 [1] and has started its public path > with > HTTP client review in March this year [2]. > > Proposed WebSocket API is relatively small and focuses on convenient > exchange of > data in a fully asynchronous fashion. API consists of 6 types located in > the > java.net package [3]: > > 1. WebSocket > 2. WebSocket.Builder > 3. WebSocket.Incoming > 4. WebSocket.Incoming.Chunks > 5. WebSocket.Outgoing > 6. WebSocketException > > Starting point is a class description for java.net.WebSocket. Along with > in-javadoc examples, several API test samples are provided in the webrev > [4] and > named test/java/net/WebSocket/Example%.java. They are only for > informational > purposes and won't be included in the final version of the API. > > I would appreciate any feedback on this API. Thanks. > > > ------------------------------------------------------------------------------- > [1] http://openjdk.java.net/jeps/110 > [2] http://mail.openjdk.java.net/pipermail/net-dev/2015-March/008932.html > [3] http://cr.openjdk.java.net/~prappo/8087113/javadoc.00/ > [4] http://cr.openjdk.java.net/~prappo/8087113/webrev.00/ > > -Pavel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bradford.wetmore at oracle.com Fri Oct 2 01:03:58 2015 From: bradford.wetmore at oracle.com (Bradford Wetmore) Date: Thu, 1 Oct 2015 18:03:58 -0700 Subject: TLS ALPN Proposal v6 In-Reply-To: <560AA924.1050706@redhat.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <560! 5AD08.9070806@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> Message-ID: <560DD7FE.2060806@oracle.com> You guys (David/Simone/Bernd/Jason) are more on the front lines in server development and how functional this API will be, so I'll trust your judgement here. If you are ok with: 1. potentially being blind during renegotiations in the existing TLSv1/v1.1/v1.2, and, 2. not having an SSLExplorer as part of the JDK (i.e. you parsing the ClientHellos ala SSLExplorer), 3. requiring all ALPN logic be in the application and none in the JDK, I'm willing to go with this approach. It doesn't seem optimal for what I would call casual users, but it does solve the ugly issues with the matches() API. (BTW, I did consider adding a ClientHelloCallback/ClientHello/ServerHelloCallback/ServerHello class that would handle callbacks, but that was getting complicated also.) For current renegotiations, the big use case is adding client authentication, so it seems likely that the same ciphersuite will be offered/chosen, so it's likely moot. For the existing URL code, do you think we need: 1. to provide a "https.alpn" System Property for the existing URLConnections? 2. a getApplicationProtocol() for HttpsURLConnection? Michael (net-dev) says H2 will not be backported into the URL mechanism and doesn't see a need for it yet, so I'm inclined to say no. More inline: On 9/29/2015 8:07 AM, David M. Lloyd wrote: > Hi Brad, thanks for replying; comments are inline: > On 09/28/2015 08:40 PM, Bradford Wetmore wrote: >> 1. Only the initial ClientHellos are parsable. >> =============================================== >> The biggest problem I have with an Explorer-based design is that only >> the initial ClientHello on a connection is passed in the clear. >> Subsequent negotiations on this connection will be completely missed, as >> the ClientHellos are now encrypted. >> >> This seems like a deal breaker for me. > > You are right, I cannot come up with a good solution for this, so that > might mean the idea is shot - *however* - I would point out that the > latest draft of TLS 1.3 [1] completely kills off the capability of the > client to renegotiate a connection, meaning that this will no longer be > possible anyway, and given it's a 1% kind of use case, that might be > enough to let it slide. Combine this with what I consider to be the > unlikelihood of this working with HTTP/2.0, and I would feel very safe > assuming that nobody will ever actually do this. Thanks for pointing this out, I thought PSK+tickets were a replacement for a renegotiation (Section 6.2.3), but it's apparently only for session resumption. BTW, the WG is up to a Sept 29, 2015 version (draft-09). [1] https://tlswg.github.io/tls13-spec/ > I would also note that, as you state later on, it would be possible to > combine this solution with any other solution (including the proposed > one) to cover both cases. And given that this is still (in my > estimation) a "99%" solution, in my opinion it is still a viable > candidate for adding this functionality to Java 8 as a first pass or > stopgap as I described in my emails, particularly if the method(s) to > establish/query the protocol names are a strict subset of the proposed > Java 9 API (given that we cannot really overhaul the Java SE 8 API at > this point). > >> [...] >> 2. "SSLExplorer" or something similar is needed. >> ================================================= >> This approach depends on "examining SSLClientHello"s, but there isn't a >> class for this other than some sample code from a previous attempt. I >> am assuming that this approach would make such an external API a >> necessity? Being able to parse possible ClientHello formats is not a >> straightforward/easy job. This will add a fair amount of complexity, >> and likely not an easy job in the remaining few weeks. It could be >> added later for JDK 10 but that means apps would likely need to roll >> their own for 9. > > And 8, yes, you definitely would need to roll your own, though Xuelei > Fan already has a nice example up on his blog that was built for SNI > (but uses the same principle). If you are referring to: http://simsmi.blogspot.com/2014/01/jep-114-tls-sni-extension-virtual.html This is just describing the general approach for the sample SSLExplorer/SSLCapabilities code in the JSSE Reference Guide. The actual code can be found here: http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#CodeExamples My hope is to expand it to include parsing the ciphersuites and ALPN extensions. I've moved/added some helper functions from ApplicationProtocol to StandardConstants. > If it were me, I wouldn't even bother > adding it even in JDK 10, since (a) it applies only to the server side > and (b) there are a plethora of third-party server-side network I/O and > security libraries which are natural candidates to host this type of logic. Ok. >> 3. "no_application_protocol" >> ============================= >> If the server doesn't support the protocols that the client advertises, >> the "no_application_protocol" must be thrown. We could add a >> "no_application_protocol" protocol String that would flag such a >> condition internally. > > Sure, though if you use the same method on both the client and server to > specify the matched protocol, then the method necessarily accepts an > array, in which case a null/unset could mean "no ALPN response" and an > empty array could mean "no acceptable protocols". But yeah I agree > otherwise. I meant if this value is set, then when the ALPN extensions are selected, it will send the alert to the peer and return an Exception to the local code. The latest webrev is at: http://cr.openjdk.java.net/~wetmore/8051498/webrev.16 This will be the final version as far as major architectural design goes, but minor things like parameter types/wording can still be tweaked. Major changes: 1. ApplicationProtocols is gone. The H2 black list and comparator were moved to StandardConstants. 2. StandardConstants. Strings for "h2" and "http/1.1" are back. And now that you are parsing the raw network bytes, I added a convenience mapping between the two byte ciphersuite IANA-assigned value and the Java Standard Name. 3. SSLParameter (set/get) are moved to SSLSocket/SSLEngine. Even though these could go into SSLParameters, this change makes backporting much easier. The helper code simply has to reflectively look for the four methods in the implementation classes, and call if they are there. Otherwise, there would have to be reflection both in the user code (above) and implementation (to see if the passed SSLParameters had the new methods via a subclass). Thanks, Brad From xuelei.fan at oracle.com Fri Oct 2 02:35:50 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 2 Oct 2015 10:35:50 +0800 Subject: TLS ALPN Proposal v6 In-Reply-To: <560DD7FE.2060806@oracle.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <560! 5AD08.9070806@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> Message-ID: <560DED86.5070702@oracle.com> On 10/2/2015 9:03 AM, Bradford Wetmore wrote: > Major changes: > > 1. ApplicationProtocols is gone. The H2 black list and comparator were > moved to StandardConstants. > > 2. StandardConstants. Strings for "h2" and "http/1.1" are back. And > now that you are parsing the raw network bytes, I added a convenience > mapping between the two byte ciphersuite IANA-assigned value and the > Java Standard Name. > There is no SSLExplorer in OpenJDK. I think, maybe, the map is not belong to OpenJDK either. I think, the constants for HTTP2 is also belong to application protocol (HTTP2) layer. Application (HTTP2) implementation would take care of them. Maybe, they are not a part of JSSE framework either. I would like to have "h2" and "http/1.1" defined as Standard Algorithms Docs as we usually did for other standard constants. > 3. SSLParameter (set/get) are moved to SSLSocket/SSLEngine. Even > though these could go into SSLParameters, this change makes backporting > much easier. The helper code simply has to reflectively look for the > four methods in the implementation classes, and call if they are there. > > Otherwise, there would have to be reflection both in the user code > (above) and implementation (to see if the passed SSLParameters had the > new methods via a subclass). But, looking forward, per JSSE framework, SSLParameters should be the central place to define SSL/TLS configuration parameters. We'd better follow the conventions so that application developers won't get confused about where SSL/TLS parameters should be configured. Maybe, we cannot add public APIs for backporting. I think backporting is another history, and would better not impact too much of the design for JDK 9 and future releases. Hope it helps! Xuelei From bradford.wetmore at oracle.com Sat Oct 3 00:19:00 2015 From: bradford.wetmore at oracle.com (Bradford Wetmore) Date: Fri, 2 Oct 2015 17:19:00 -0700 Subject: TLS ALPN Proposal v7 In-Reply-To: <560DED86.5070702@oracle.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <560! 5AD08.9070806@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> <560DED86.5070702@oracle.com> Message-ID: <560F1EF4.4060405@oracle.com> On 10/1/2015 7:35 PM, Xuelei Fan wrote: > On 10/2/2015 9:03 AM, Bradford Wetmore wrote: >> Major changes: >> >> 1. ApplicationProtocols is gone. The H2 black list and comparator were >> moved to StandardConstants. >> >> 2. StandardConstants. Strings for "h2" and "http/1.1" are back. And >> now that you are parsing the raw network bytes, I added a convenience >> mapping between the two byte ciphersuite IANA-assigned value and the >> Java Standard Name. >> > There is no SSLExplorer in OpenJDK. I think, maybe, the map is not > belong to OpenJDK either. > > I think, the constants for HTTP2 is also belong to application protocol > (HTTP2) layer. Application (HTTP2) implementation would take care of > them. Maybe, they are not a part of JSSE framework either. Ok, I've removed all of the H2 constants from StandardConstants. I've mentioned to the the H2/network team that this is something they will need to handle/include (e.g. Blacklist/Comparator) in their code, and they might want to consider APIs similar to what I had. I personally found it very useful to have a proper mapping to get the SSL_ vs. TLS_ prefix correct in the blacklist. > I would like to have "h2" and "http/1.1" defined as Standard Algorithms > Docs as we usually did for other standard constants. Of course, they will be added as Standard Algorithm names. >> 3. SSLParameter (set/get) are moved to SSLSocket/SSLEngine. Even >> though these could go into SSLParameters, this change makes backporting >> much easier. The helper code simply has to reflectively look for the >> four methods in the implementation classes, and call if they are there. >> >> Otherwise, there would have to be reflection both in the user code >> (above) and implementation (to see if the passed SSLParameters had the >> new methods via a subclass). > But, looking forward, per JSSE framework, SSLParameters should be the > central place to define SSL/TLS configuration parameters. We'd better > follow the conventions so that application developers won't get confused > about where SSL/TLS parameters should be configured. I went back and forth on this many, many times yesterday. > Maybe, we cannot add public APIs for backporting. I figured we'd have to use reflection on user-derived classes to see if the methods were there, but apparently implementation-specifc APIs can be added in an update release, just not Java APIs. So if we really can do that, then we can add a jdk.SSLParameters/ org.openjdk.jsse.SSLParameters extends SSLParameters, and in the implementation, look for instanceof. > I think backporting is > another history, and would better not impact too much of the design for > JDK 9 and future releases. Ok, so get/setApplicationProtocols() is back in SSLParameters. Thanks for the comments everyone. I'm submitting the following to the CCC (internal review board): http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ Changes: 1. No H2 Blacklist/Comparator 2. set/getApplicationProtocols() back to SSLParameters. Thanks, Brad From xuelei.fan at oracle.com Sat Oct 3 00:48:46 2015 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Sat, 3 Oct 2015 08:48:46 +0800 Subject: TLS ALPN Proposal v7 In-Reply-To: <560F1EF4.4060405@oracle.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <560! 5AD08.9070806@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> <560DED86.5070702@oracle.com> <560F1EF4.4060405@oracle.com> Message-ID: <560F25EE.9080005@oracle.com> Thanks! Xuelei On 10/3/2015 8:19 AM, Bradford Wetmore wrote: > > > On 10/1/2015 7:35 PM, Xuelei Fan wrote: >> On 10/2/2015 9:03 AM, Bradford Wetmore wrote: >>> Major changes: >>> >>> 1. ApplicationProtocols is gone. The H2 black list and comparator were >>> moved to StandardConstants. >>> >>> 2. StandardConstants. Strings for "h2" and "http/1.1" are back. And >>> now that you are parsing the raw network bytes, I added a convenience >>> mapping between the two byte ciphersuite IANA-assigned value and the >>> Java Standard Name. >>> >> There is no SSLExplorer in OpenJDK. I think, maybe, the map is not >> belong to OpenJDK either. >> >> I think, the constants for HTTP2 is also belong to application protocol >> (HTTP2) layer. Application (HTTP2) implementation would take care of >> them. Maybe, they are not a part of JSSE framework either. > > Ok, I've removed all of the H2 constants from StandardConstants. I've > mentioned to the the H2/network team that this is something they will > need to handle/include (e.g. Blacklist/Comparator) in their code, and > they might want to consider APIs similar to what I had. > > I personally found it very useful to have a proper mapping to get the > SSL_ vs. TLS_ prefix correct in the blacklist. > >> I would like to have "h2" and "http/1.1" defined as Standard Algorithms >> Docs as we usually did for other standard constants. > > Of course, they will be added as Standard Algorithm names. > >>> 3. SSLParameter (set/get) are moved to SSLSocket/SSLEngine. Even >>> though these could go into SSLParameters, this change makes backporting >>> much easier. The helper code simply has to reflectively look for the >>> four methods in the implementation classes, and call if they are there. >>> >>> Otherwise, there would have to be reflection both in the user code >>> (above) and implementation (to see if the passed SSLParameters had the >>> new methods via a subclass). >> But, looking forward, per JSSE framework, SSLParameters should be the >> central place to define SSL/TLS configuration parameters. We'd better >> follow the conventions so that application developers won't get confused >> about where SSL/TLS parameters should be configured. > > I went back and forth on this many, many times yesterday. > >> Maybe, we cannot add public APIs for backporting. > > I figured we'd have to use reflection on user-derived classes to see if > the methods were there, but apparently implementation-specifc APIs can > be added in an update release, just not Java APIs. So if we really can > do that, then we can add a jdk.SSLParameters/ > org.openjdk.jsse.SSLParameters extends SSLParameters, and in the > implementation, look for instanceof. > >> I think backporting is >> another history, and would better not impact too much of the design for >> JDK 9 and future releases. > > Ok, so get/setApplicationProtocols() is back in SSLParameters. > > Thanks for the comments everyone. I'm submitting the following to the > CCC (internal review board): > > http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ > > Changes: > > 1. No H2 Blacklist/Comparator > > 2. set/getApplicationProtocols() back to SSLParameters. > > Thanks, > > Brad > From pavel.rappo at oracle.com Tue Oct 6 10:27:28 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 6 Oct 2015 11:27:28 +0100 Subject: WebSocket client API In-Reply-To: References: Message-ID: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> Hi, Here's an update on the WebSocket API. This iteration tries to address all issues have been discussed so far: webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.01/ javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.01/ Main differences from the previous version: * Extension support has been postponed and remains an open issue [1] * WebSocket.Builder now also accepts HttpRequest.Builder (for providing custom opening handshake headers) * Outgoing is gone, instead a user can send incomplete Binary and Text messages with ByteBuffers and CharSequences directly * Incoming is gone, instead WebSocket.Builder provides a handler assigning method per event (message or error) * Async methods take a custom context object and a potentially reusable completion handler (NIO2 style) * The API is now j.u.c.Flow-friendly ------------------------------------------------------------------------------- [1] https://bugs.openjdk.java.net/browse/JDK-8138949; That doesn't mean the default implementation won't support 'permessage-deflate'. -Pavel > On 31 Aug 2015, at 15:30, Pavel Rappo wrote: > > Hi, > > I would appreciate if you help to review a WebSocket client API proposed for > JDK 9. This work is a part of JEP-110 [1] and has started its public path with > HTTP client review in March this year [2]. > > Proposed WebSocket API is relatively small and focuses on convenient exchange of > data in a fully asynchronous fashion. API consists of 6 types located in the > java.net package [3]: > > 1. WebSocket > 2. WebSocket.Builder > 3. WebSocket.Incoming > 4. WebSocket.Incoming.Chunks > 5. WebSocket.Outgoing > 6. WebSocketException > > Starting point is a class description for java.net.WebSocket. Along with > in-javadoc examples, several API test samples are provided in the webrev [4] and > named test/java/net/WebSocket/Example%.java. They are only for informational > purposes and won't be included in the final version of the API. > > I would appreciate any feedback on this API. Thanks. > > ------------------------------------------------------------------------------- > [1] http://openjdk.java.net/jeps/110 > [2] http://mail.openjdk.java.net/pipermail/net-dev/2015-March/008932.html > [3] http://cr.openjdk.java.net/~prappo/8087113/javadoc.00/ > [4] http://cr.openjdk.java.net/~prappo/8087113/webrev.00/ > > -Pavel From sebastian.sickelmann at gmx.de Tue Oct 6 11:13:55 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 6 Oct 2015 13:13:55 +0200 Subject: RFR: JDK-8022748 (new URI(u.toString()).equals(u), does not hold with paths containing colons Message-ID: <5613ACF3.10802@gmx.de> Hi, i investigated the problem described in JDK-8022748[1] i found that the parser needed to be rescued for confusion while handling relative URIs. A URI created through the relativize-method is schemaless and so it need to handle the special-case (a colon in the path-element). While there is also another way to handle it (encode the colon as %3A) i think we should not choose to encode everything. First it would introduce another style of special-case handling of a colon in the path (see the method maybeAddLeadingDot which is used while normalizing) and when resolving it back we would need to decode it back or leave it the encoded way which is not suggest by RFC2396 section "1.5 URI Transcribability". Also in Section 5 of RFC2396 it is suggested to rescue a colon in the path-element in a relative URI through prepending a "./" just like in maybeAddLeadingDot. I am not sure if it is worth to refactor the split(),join(),normalize() methods to provide a reusable colon-detection and -handling. So, please find my webrev with a simpler solution at: http://cr.openjdk.java.net/~sebastian/8022748/webrev.00/ -- Sebastian [1] https://bugs.openjdk.java.net/browse/JDK-8022748 -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Tue Oct 6 14:42:03 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 6 Oct 2015 09:42:03 -0500 Subject: TLS ALPN Proposal v7 In-Reply-To: <560F1EF4.4060405@oracle.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <560! 5AD08.9070806@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> <560DED86.5070702@oracle.com> <560F1EF4.4060405@oracle.com> Message-ID: <5613DDBB.9020707@redhat.com> I didn't reply on this last week, but this looks workable to me. Thanks! On 10/02/2015 07:19 PM, Bradford Wetmore wrote: > > > On 10/1/2015 7:35 PM, Xuelei Fan wrote: >> On 10/2/2015 9:03 AM, Bradford Wetmore wrote: >>> Major changes: >>> >>> 1. ApplicationProtocols is gone. The H2 black list and comparator were >>> moved to StandardConstants. >>> >>> 2. StandardConstants. Strings for "h2" and "http/1.1" are back. And >>> now that you are parsing the raw network bytes, I added a convenience >>> mapping between the two byte ciphersuite IANA-assigned value and the >>> Java Standard Name. >>> >> There is no SSLExplorer in OpenJDK. I think, maybe, the map is not >> belong to OpenJDK either. >> >> I think, the constants for HTTP2 is also belong to application protocol >> (HTTP2) layer. Application (HTTP2) implementation would take care of >> them. Maybe, they are not a part of JSSE framework either. > > Ok, I've removed all of the H2 constants from StandardConstants. I've > mentioned to the the H2/network team that this is something they will > need to handle/include (e.g. Blacklist/Comparator) in their code, and > they might want to consider APIs similar to what I had. > > I personally found it very useful to have a proper mapping to get the > SSL_ vs. TLS_ prefix correct in the blacklist. > >> I would like to have "h2" and "http/1.1" defined as Standard Algorithms >> Docs as we usually did for other standard constants. > > Of course, they will be added as Standard Algorithm names. > >>> 3. SSLParameter (set/get) are moved to SSLSocket/SSLEngine. Even >>> though these could go into SSLParameters, this change makes backporting >>> much easier. The helper code simply has to reflectively look for the >>> four methods in the implementation classes, and call if they are there. >>> >>> Otherwise, there would have to be reflection both in the user code >>> (above) and implementation (to see if the passed SSLParameters had the >>> new methods via a subclass). >> But, looking forward, per JSSE framework, SSLParameters should be the >> central place to define SSL/TLS configuration parameters. We'd better >> follow the conventions so that application developers won't get confused >> about where SSL/TLS parameters should be configured. > > I went back and forth on this many, many times yesterday. > >> Maybe, we cannot add public APIs for backporting. > > I figured we'd have to use reflection on user-derived classes to see if > the methods were there, but apparently implementation-specifc APIs can > be added in an update release, just not Java APIs. So if we really can > do that, then we can add a jdk.SSLParameters/ > org.openjdk.jsse.SSLParameters extends SSLParameters, and in the > implementation, look for instanceof. > >> I think backporting is >> another history, and would better not impact too much of the design for >> JDK 9 and future releases. > > Ok, so get/setApplicationProtocols() is back in SSLParameters. > > Thanks for the comments everyone. I'm submitting the following to the > CCC (internal review board): > > http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ > > Changes: > > 1. No H2 Blacklist/Comparator > > 2. set/getApplicationProtocols() back to SSLParameters. > > Thanks, > > Brad > -- - DML From paul.sandoz at oracle.com Tue Oct 6 14:45:57 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 6 Oct 2015 16:45:57 +0200 Subject: WebSocket client API In-Reply-To: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> Message-ID: <3A437E60-C4BA-446E-BF15-825061B30727@oracle.com> Hi, Overall i think this API seems to be compressing down nicely to a small number of classes/interfaces, but i still think there is some room for further simplifications. WebSocket I don?t see the need for a context parameter. Context can be obtained via capturing. That reduces the requirement for a specific but generic BiHandler, since a BiConsumer could be used instead. A null completion handler could be allowed for those not interested in completion events. My preferred alternative to the callback handler approach is to return a Completable that completes with the WebSocket instance or exceptionally. That to me better fits with the notion of completion of a specific event (just like that when async building a connected WebSocket), that allows one to chain off events or even block if necessary. It also reduces the need for a specific but generic functional interface. I suspect the common case would be to ignore the CF that is returned. WebSocket.Builder The protocol of receiving events is spread out over multiple behavioural parameters. Instead we could have a functional interface, WebSocketMessageListener say, with defaults for all but the most common event, which i guess is the receiving of text messages. It?s easy to reintroduce the current behaviour as a separate layer if so desired e.g. a simple message event builder, but it?s really awkward to do it the other way around, which is a design smell to me. The builder can then be reformulated as: Builder(String uri) // A URI is the only required argument using(HttpClient ) // Overrides any previous call using(HttpRequest.Builder ) // Overrides any previous call subprotocols(String , String ...) listener(WebSocketMessageListener ) // Overrides any previous call WebSocket build( ) CF buildAsync() The end result is WebSocket.BiHandler/Handler go away to be replaced with one specific web socket message listener interface whose documentation describes the receiving protocol and the threading/concurrency callback behaviour, and overall there is a simplification in the set of methods and their signatures. Paul. > On 6 Oct 2015, at 12:27, Pavel Rappo wrote: > > Hi, > > Here's an update on the WebSocket API. This iteration tries to address all > issues have been discussed so far: > > webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.01/ > javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.01/ > > Main differences from the previous version: > > * Extension support has been postponed and remains an open issue [1] > * WebSocket.Builder now also accepts HttpRequest.Builder (for providing > custom opening handshake headers) > * Outgoing is gone, instead a user can send incomplete Binary and Text > messages with ByteBuffers and CharSequences directly > * Incoming is gone, instead WebSocket.Builder provides a handler > assigning method per event (message or error) > * Async methods take a custom context object and a potentially reusable > completion handler (NIO2 style) > * The API is now j.u.c.Flow-friendly > > ------------------------------------------------------------------------------- > [1] https://bugs.openjdk.java.net/browse/JDK-8138949; > That doesn't mean the default implementation won't support > 'permessage-deflate'. > > -Pavel > >> On 31 Aug 2015, at 15:30, Pavel Rappo wrote: >> >> Hi, >> >> I would appreciate if you help to review a WebSocket client API proposed for >> JDK 9. This work is a part of JEP-110 [1] and has started its public path with >> HTTP client review in March this year [2]. >> >> Proposed WebSocket API is relatively small and focuses on convenient exchange of >> data in a fully asynchronous fashion. API consists of 6 types located in the >> java.net package [3]: >> >> 1. WebSocket >> 2. WebSocket.Builder >> 3. WebSocket.Incoming >> 4. WebSocket.Incoming.Chunks >> 5. WebSocket.Outgoing >> 6. WebSocketException >> >> Starting point is a class description for java.net.WebSocket. Along with >> in-javadoc examples, several API test samples are provided in the webrev [4] and >> named test/java/net/WebSocket/Example%.java. They are only for informational >> purposes and won't be included in the final version of the API. >> >> I would appreciate any feedback on this API. Thanks. >> >> ------------------------------------------------------------------------------- >> [1] http://openjdk.java.net/jeps/110 >> [2] http://mail.openjdk.java.net/pipermail/net-dev/2015-March/008932.html >> [3] http://cr.openjdk.java.net/~prappo/8087113/javadoc.00/ >> [4] http://cr.openjdk.java.net/~prappo/8087113/webrev.00/ >> >> -Pavel > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From artem.smotrakov at oracle.com Wed Oct 7 11:19:41 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Wed, 7 Oct 2015 14:19:41 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed Message-ID: <5614FFCD.5070903@oracle.com> Hello, Please review this for 9. According to [1], an HTTP client should try to use another HTTP authentication scheme if negotiate process failed for some reason, and a user didn't specify SPNEGO or Kerberos in "http.auth.preference" system property. But no fallback happens if, for example: - an HTTP server supports both Negotiate (via Kerberos) and Basic authentication schemes - first, a user provides correct Kerberos credentials, and a connection is successfully established with Negotiate scheme - then, a user provides wrong Kerberos credentials, but correct Basic credentials This fix updates HttpURLConnection to try another authentication scheme negotiate process failed, and SPNEGO and Kerberos schemes are not preferred. The fix may be shorter, for example: if ( serverAuthentication != null || inNegotiate && !"negotiate".equals(AuthenticationHeader.authPref)) { , but I thought that some logging might be helpful. Also added a test which checks this and a couple of other scenarios work fine. Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ [1] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.coffey at oracle.com Wed Oct 7 14:28:57 2015 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 7 Oct 2015 15:28:57 +0100 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <5614FFCD.5070903@oracle.com> References: <5614FFCD.5070903@oracle.com> Message-ID: <56152C29.7050504@oracle.com> Thanks for handling Artem. I'll leave the main review to someone more knowledgeable with http authentication schemes but can I suggest that your print the AuthenticationHeader.authPref string out with the "Negotiate process failed, fallback" logger message. It's a useful variable to capture. Regards, Sean. On 07/10/2015 12:19, Artem Smotrakov wrote: > Hello, > > Please review this for 9. > > According to [1], an HTTP client should try to use another HTTP > authentication scheme if negotiate process failed for some reason, and > a user didn't specify SPNEGO or Kerberos in "http.auth.preference" > system property. But no fallback happens if, for example: > - an HTTP server supports both Negotiate (via Kerberos) and Basic > authentication schemes > - first, a user provides correct Kerberos credentials, and a > connection is successfully established with Negotiate scheme > - then, a user provides wrong Kerberos credentials, but correct Basic > credentials > > This fix updates HttpURLConnection to try another authentication > scheme negotiate process failed, and SPNEGO and Kerberos schemes are > not preferred. The fix may be shorter, for example: > > if ( serverAuthentication != null || inNegotiate && > !"negotiate".equals(AuthenticationHeader.authPref)) { > > , but I thought that some logging might be helpful. > > Also added a test which checks this and a couple of other scenarios > work fine. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 > Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ > > [1] > https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html > > Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at oracle.com Wed Oct 7 14:39:01 2015 From: weijun.wang at oracle.com (Wang Weijun) Date: Wed, 7 Oct 2015 22:39:01 +0800 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <5614FFCD.5070903@oracle.com> References: <5614FFCD.5070903@oracle.com> Message-ID: <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> I will look into this. Busy on something else at the moment. Do you think this would happen in reality? There weren't a lot of fallback in HTTP auth, IMO, is that because in most cases username and password are the same for all schemes, and if one fails, we believe the pair is wrong and there is no need to try another. Negotiate was picked as a special case because configuration could go wrong even if the username and password are correct, and we provide a fallback. For example, what about fallback from Digest to Basic? Could the digest credentials be provided correctly at first and wrong later? What would happen? I haven't read the HttpURLConnection class for a long time and I could be wrong. Thanks Max > On Oct 7, 2015, at 7:19 PM, Artem Smotrakov wrote: > > Hello, > > Please review this for 9. > > According to [1], an HTTP client should try to use another HTTP authentication scheme if negotiate process failed for some reason, and a user didn't specify SPNEGO or Kerberos in "http.auth.preference" system property. But no fallback happens if, for example: > - an HTTP server supports both Negotiate (via Kerberos) and Basic authentication schemes > - first, a user provides correct Kerberos credentials, and a connection is successfully established with Negotiate scheme > - then, a user provides wrong Kerberos credentials, but correct Basic credentials > > This fix updates HttpURLConnection to try another authentication scheme negotiate process failed, and SPNEGO and Kerberos schemes are not preferred. The fix may be shorter, for example: > > if ( serverAuthentication != null || inNegotiate && !"negotiate".equals(AuthenticationHeader.authPref)) { > > , but I thought that some logging might be helpful. > > Also added a test which checks this and a couple of other scenarios work fine. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 > Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ > > [1] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html > > Artem From artem.smotrakov at oracle.com Wed Oct 7 15:25:22 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Wed, 7 Oct 2015 18:25:22 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <56152C29.7050504@oracle.com> References: <5614FFCD.5070903@oracle.com> <56152C29.7050504@oracle.com> Message-ID: <56153962.7060408@oracle.com> Hi Sean, Sure, it may be useful to print it out. I will update the webrev. Artem On 10/07/2015 05:28 PM, Se?n Coffey wrote: > Thanks for handling Artem. I'll leave the main review to someone more > knowledgeable with http authentication schemes but can I suggest that > your print the AuthenticationHeader.authPref string out with the > "Negotiate process failed, fallback" logger message. It's a useful > variable to capture. > Regards, > Sean. > On 07/10/2015 12:19, Artem Smotrakov wrote: >> Hello, >> >> Please review this for 9. >> >> According to [1], an HTTP client should try to use another HTTP >> authentication scheme if negotiate process failed for some reason, >> and a user didn't specify SPNEGO or Kerberos in >> "http.auth.preference" system property. But no fallback happens if, >> for example: >> - an HTTP server supports both Negotiate (via Kerberos) and Basic >> authentication schemes >> - first, a user provides correct Kerberos credentials, and a >> connection is successfully established with Negotiate scheme >> - then, a user provides wrong Kerberos credentials, but correct Basic >> credentials >> >> This fix updates HttpURLConnection to try another authentication >> scheme negotiate process failed, and SPNEGO and Kerberos schemes are >> not preferred. The fix may be shorter, for example: >> >> if ( serverAuthentication != null || inNegotiate && >> !"negotiate".equals(AuthenticationHeader.authPref)) { >> >> , but I thought that some logging might be helpful. >> >> Also added a test which checks this and a couple of other scenarios >> work fine. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 >> Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ >> >> [1] >> https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html >> >> Artem > -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.smotrakov at oracle.com Wed Oct 7 15:51:11 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Wed, 7 Oct 2015 18:51:11 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> Message-ID: <56153F6F.1080801@oracle.com> Hi Max, HttpURLConnection obtains credentials for HTTP authentication from Authenticator [1] implementation. Only one authenticator can be set in JVM instance. It can have built-in credentials, or do some interactions with user to get them. Theoretically, it can provide different credentials depending on a user/application/etc. I don't know how it is used in real application, but it seems to be a possible situation. When I was looking into this, I found a tech note [2] which says the following about fallback ... Fallback If the server has provided more than one authentication schemes (including Negotiate), according to the processing order mentioned in the last section, Java will try to challenge the Negotiate scheme. However, if the protocol cannot be established successfully (e.g. The kerberos configuration is not correct, or the server's hostname is not recorded in the KDC principal DB, or the username and password provided by Authenticator is wrong), then the 2nd strongest scheme will be automatically used. Attention : If http.auth.preference is set to SPNEGO or Kerberos, then we assume you only want to try the Negotiate scheme even if it fails. we won't fallback to any other scheme and your program will result in throwing an IOException saying it receives a 401 or 407 error from the HTTP response. ... As far as I understand, the current version of HttpURLConnection doesn't seem to follow this. That's why I think it needs to be fixed. Otherwise, the tech note [2] should be updated. It doesn't look like a serious issue for me (that's why it is P3, or maybe it should be P4). Furthermore, it looks like nobody has had such a problem before because I didn't fine any bug about that at https://bugs.openjdk.java.net According to [2], Digest -> Basic fallback should not happen. HttpURLConnection is quite smart, and if I understand correctly, we have only "http.auth.preference" and Authenticator.setDefault() to control HTTP authentication process. Maybe we can make it more configurable. [1] http://docs.oracle.com/javase/8/docs/api/java/net/Authenticator.html [2] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html Artem On 10/07/2015 05:39 PM, Wang Weijun wrote: > I will look into this. Busy on something else at the moment. > > Do you think this would happen in reality? There weren't a lot of fallback in HTTP auth, IMO, is that because in most cases username and password are the same for all schemes, and if one fails, we believe the pair is wrong and there is no need to try another. Negotiate was picked as a special case because configuration could go wrong even if the username and password are correct, and we provide a fallback. > > For example, what about fallback from Digest to Basic? Could the digest credentials be provided correctly at first and wrong later? What would happen? > > I haven't read the HttpURLConnection class for a long time and I could be wrong. > > Thanks > Max > >> On Oct 7, 2015, at 7:19 PM, Artem Smotrakov wrote: >> >> Hello, >> >> Please review this for 9. >> >> According to [1], an HTTP client should try to use another HTTP authentication scheme if negotiate process failed for some reason, and a user didn't specify SPNEGO or Kerberos in "http.auth.preference" system property. But no fallback happens if, for example: >> - an HTTP server supports both Negotiate (via Kerberos) and Basic authentication schemes >> - first, a user provides correct Kerberos credentials, and a connection is successfully established with Negotiate scheme >> - then, a user provides wrong Kerberos credentials, but correct Basic credentials >> >> This fix updates HttpURLConnection to try another authentication scheme negotiate process failed, and SPNEGO and Kerberos schemes are not preferred. The fix may be shorter, for example: >> >> if ( serverAuthentication != null || inNegotiate && !"negotiate".equals(AuthenticationHeader.authPref)) { >> >> , but I thought that some logging might be helpful. >> >> Also added a test which checks this and a couple of other scenarios work fine. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 >> Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ >> >> [1] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html >> >> Artem From artem.smotrakov at oracle.com Wed Oct 7 15:58:33 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Wed, 7 Oct 2015 18:58:33 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <56153F6F.1080801@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> <56153F6F.1080801@oracle.com> Message-ID: <56154129.4020606@oracle.com> Please see updated webrev http://cr.openjdk.java.net/~asmotrak/8138953/webrev.01/ Artem On 10/07/2015 06:51 PM, Artem Smotrakov wrote: > Hi Max, > > HttpURLConnection obtains credentials for HTTP authentication from > Authenticator [1] implementation. Only one authenticator can be set in > JVM instance. It can have built-in credentials, or do some > interactions with user to get them. Theoretically, it can provide > different credentials depending on a user/application/etc. I don't > know how it is used in real application, but it seems to be a possible > situation. When I was looking into this, I found a tech note [2] which > says the following about fallback > > ... > Fallback > If the server has provided more than one authentication schemes > (including Negotiate), according to the processing order mentioned in > the last section, Java will try to challenge the Negotiate scheme. > However, if the protocol cannot be established successfully (e.g. The > kerberos configuration is not correct, or the server's hostname is not > recorded in the KDC principal DB, or the username and password > provided by Authenticator is wrong), then the 2nd strongest scheme > will be automatically used. Attention : If http.auth.preference is set > to SPNEGO or Kerberos, then we assume you only want to try the > Negotiate scheme even if it fails. we won't fallback to any other > scheme and your program will result in throwing an IOException saying > it receives a 401 or 407 error from the HTTP response. > ... > > As far as I understand, the current version of HttpURLConnection > doesn't seem to follow this. That's why I think it needs to be fixed. > Otherwise, the tech note [2] should be updated. > > It doesn't look like a serious issue for me (that's why it is P3, or > maybe it should be P4). Furthermore, it looks like nobody has had such > a problem before because I didn't fine any bug about that at > https://bugs.openjdk.java.net > > According to [2], Digest -> Basic fallback should not happen. > HttpURLConnection is quite smart, and if I understand correctly, we > have only "http.auth.preference" and Authenticator.setDefault() to > control HTTP authentication process. Maybe we can make it more > configurable. > > [1] http://docs.oracle.com/javase/8/docs/api/java/net/Authenticator.html > [2] > https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html > > Artem > > On 10/07/2015 05:39 PM, Wang Weijun wrote: >> I will look into this. Busy on something else at the moment. >> >> Do you think this would happen in reality? There weren't a lot of >> fallback in HTTP auth, IMO, is that because in most cases username >> and password are the same for all schemes, and if one fails, we >> believe the pair is wrong and there is no need to try another. >> Negotiate was picked as a special case because configuration could go >> wrong even if the username and password are correct, and we provide a >> fallback. >> >> For example, what about fallback from Digest to Basic? Could the >> digest credentials be provided correctly at first and wrong later? >> What would happen? >> >> I haven't read the HttpURLConnection class for a long time and I >> could be wrong. >> >> Thanks >> Max >> >>> On Oct 7, 2015, at 7:19 PM, Artem Smotrakov >>> wrote: >>> >>> Hello, >>> >>> Please review this for 9. >>> >>> According to [1], an HTTP client should try to use another HTTP >>> authentication scheme if negotiate process failed for some reason, >>> and a user didn't specify SPNEGO or Kerberos in >>> "http.auth.preference" system property. But no fallback happens if, >>> for example: >>> - an HTTP server supports both Negotiate (via Kerberos) and Basic >>> authentication schemes >>> - first, a user provides correct Kerberos credentials, and a >>> connection is successfully established with Negotiate scheme >>> - then, a user provides wrong Kerberos credentials, but correct >>> Basic credentials >>> >>> This fix updates HttpURLConnection to try another authentication >>> scheme negotiate process failed, and SPNEGO and Kerberos schemes are >>> not preferred. The fix may be shorter, for example: >>> >>> if ( serverAuthentication != null || inNegotiate && >>> !"negotiate".equals(AuthenticationHeader.authPref)) { >>> >>> , but I thought that some logging might be helpful. >>> >>> Also added a test which checks this and a couple of other scenarios >>> work fine. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138953 >>> Webrev: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.00/ >>> >>> [1] >>> https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html >>> >>> Artem > From sebastian.sickelmann at gmx.de Wed Oct 7 19:59:15 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Wed, 7 Oct 2015 21:59:15 +0200 Subject: RFR: 5108778 Too many instances of java.lang.Boolean created in Java application(net) Message-ID: <56157993.5010000@gmx.de> Hello, Actually I am searching through the JBS for low hanging fruits. Right now i am looking through the openjdk-sources and try to evaluate if i can make something about JDK-5108778. Please find my webrevs for the jdk(net) repos at: http://cr.openjdk.java.net/~sebastian/5108778/net/webrev.00/ The changes are javadoc only. For some general discussion on regression-tests for this please find the thread in discuss[0][1] and for the general suggestion to make more wrapper-type-constructors deprecated find [2] at core-libs-dev. [0] http://mail.openjdk.java.net/pipermail/discuss/2015-September/003804.html [1] http://mail.openjdk.java.net/pipermail/discuss/2015-October/003805.html [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-October/035642.html -- Sebastian From weijun.wang at oracle.com Thu Oct 8 08:41:32 2015 From: weijun.wang at oracle.com (Wang Weijun) Date: Thu, 8 Oct 2015 16:41:32 +0800 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <56153F6F.1080801@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> <56153F6F.1080801@oracle.com> Message-ID: <803BC362-2AEF-4469-B0A9-807C652C8742@oracle.com> > On Oct 7, 2015, at 11:51 PM, Artem Smotrakov wrote: > > Hi Max, > > HttpURLConnection obtains credentials for HTTP authentication from Authenticator [1] implementation. Only one authenticator can be set in JVM instance. It can have built-in credentials, or do some interactions with user to get them. Theoretically, it can provide different credentials depending on a user/application/etc. I don't know how it is used in real application, but it seems to be a possible situation. When I was looking into this, I found a tech note [2] which says the following about fallback > > ... > Fallback > If the server has provided more than one authentication schemes (including Negotiate), according to the processing order mentioned in the last section, Java will try to challenge the Negotiate scheme. However, if the protocol cannot be established successfully (e.g. The kerberos configuration is not correct, or the server's hostname is not recorded in the KDC principal DB, or the username and password provided by Authenticator is wrong), then the 2nd strongest scheme will be automatically used. Attention : If http.auth.preference is set to SPNEGO or Kerberos, then we assume you only want to try the Negotiate scheme even if it fails. we won't fallback to any other scheme and your program will result in throwing an IOException saying it receives a 401 or 407 error from the HTTP response. > ... This was written by me. As I said in the previous mail, this is the only case where I think a fallback is worth doing. > > As far as I understand, the current version of HttpURLConnection doesn't seem to follow this. That's why I think it needs to be fixed. Otherwise, the tech note [2] should be updated. Not exactly. The tech note is mostly about config error and not about wrong credentials (see the e.g. line). It is quite rare someone provides wrong credentials for one scheme and correct ones for another. If it really could happen, we should consider a more generalized fallback and not only from Negotiate. > > It doesn't look like a serious issue for me (that's why it is P3, or maybe it should be P4). Furthermore, it looks like nobody has had such a problem before because I didn't fine any bug about that at https://bugs.openjdk.java.net > > According to [2], Digest -> Basic fallback should not happen. Yes, according to the words. But we are talking about what the most correct behavior is. Thanks Max > HttpURLConnection is quite smart, and if I understand correctly, we have only "http.auth.preference" and Authenticator.setDefault() to control HTTP authentication process. Maybe we can make it more configurable. > > [1] http://docs.oracle.com/javase/8/docs/api/java/net/Authenticator.html > [2] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html > > Artem From simone.bordet at gmail.com Thu Oct 8 16:55:35 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Thu, 8 Oct 2015 18:55:35 +0200 Subject: TLS ALPN Proposal v7 In-Reply-To: <560F1EF4.4060405@oracle.com> References: <55ED0A6A.3050806@oracle.com> <56051836.70901@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> <560DED86.5070702@oracle.com> <560F1EF4.4060405@oracle.com> Message-ID: Bradford, On Sat, Oct 3, 2015 at 2:19 AM, Bradford Wetmore wrote: > Thanks for the comments everyone. I'm submitting the following to the CCC > (internal review board): > > http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ > > Changes: > > 1. No H2 Blacklist/Comparator > > 2. set/getApplicationProtocols() back to SSLParameters. Have you implemented this solution already ? Also for clients ? Do you have feedback on actually implementing ALPN in this way ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From bradford.wetmore at oracle.com Thu Oct 8 17:32:25 2015 From: bradford.wetmore at oracle.com (Bradford Wetmore) Date: Thu, 8 Oct 2015 10:32:25 -0700 Subject: TLS ALPN Proposal v7 In-Reply-To: References: <55ED0A6A.3050806@oracle.com> <56053AFA.4000100@redhat.com> <5605421F.8010105@redhat.com> <560557F6.6090107@oracle.com> <56055EBC.9040202@redhat.com> <560566DB.20805@redhat.com> <56057B25.4070103@redhat.com> <560582F4.9000906@redhat.com> <7CC07DE5-778C-428E-9E02-19E7A883CB23@redhat.com> <5609EBF6.7080306@oracle.com> <560AA924.1050706@redhat.com> <560DD7FE.2060806@oracle.com> <560DED86.5070702@oracle.com> <560F1EF4.4060405@oracle.com> Message-ID: <5616A8A9.9010809@oracle.com> > On Sat, Oct 3, 2015 at 2:19 AM, Bradford Wetmore > wrote: >> Thanks for the comments everyone. I'm submitting the following to the CCC >> (internal review board): >> >> http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ >> >> Changes: >> >> 1. No H2 Blacklist/Comparator >> >> 2. set/getApplicationProtocols() back to SSLParameters. > > Have you implemented this solution already ? It is underway. The guts was already done based on previous API versions. The new API is less involved, so it should be simpler to do as it's just cutting out the existing ciphersuite/ALPN selection stuff. > Also for clients ? Client/server are both externally(API)/internally essentially the same. The big difference is that for clients you send all the values passed in, for servers you only consult the first which is the selected ALPN value. > Do you have feedback on actually implementing ALPN in this way ? Based on what I saw previously, it should be pretty straighforward. Brad From simone.bordet at gmail.com Thu Oct 8 19:51:09 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Thu, 8 Oct 2015 21:51:09 +0200 Subject: WebSocket client API In-Reply-To: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> Message-ID: Hi, On Tue, Oct 6, 2015 at 12:27 PM, Pavel Rappo wrote: > Hi, > > Here's an update on the WebSocket API. This iteration tries to address all > issues have been discussed so far: > > webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.01/ > javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.01/ I agree with Paul that the context does not provide much benefit and will simplify the API :) What it is still missing is the fact that there is no specification about the onXXX methods regarding the lifecycle of the parameters passed in. For example, this is going to surprise users (simple echo): onBinary((ws, bytes, last) -> { ws.sendBinary(bytes, last, null, (_, failure) -> {}); } It's not going to work because the send is async, and there is no specification about who owns the ByteBuffer "bytes". In my experience, it is bad to force applications to perform a copy (not to mention that the copy could be really expensive, as WebSocket frames could be large). This would also lead to applications being forced to block waiting for the send to complete, which defeats the goal of an async API (and may take a long time). A throw-away ByteBuffer is an alternative, but it becomes an allocation hotspot. There are well known, proven, and widely available solution for this particular problem; any works for me, but designing the API without is IMHO a mistake, especially considering that this is going to be a low-level API that manages frames and not messages. The *API* should provide a callback to notify when the ByteBuffer has been consumed. The current proposal suffers from the fact that you want to use only 1-2 classes (Handler and BiHandler) for too many use cases. Again I agree with Paul that this smells, and I would prefer a Listener interface. StatusCode makes little sense to me: it's a wrapper for int, and nothing more. I would prefer to see primitive int in the signatures. Method isClosed() does not convey enough information. WebSocket has built-in in the protocol half-closes, so it should be able to report this information, like Socket returns isInputShutDown(), isOutputShutDown() and isOpen(). A simple enum would do. I think it's enough for the builder to provide an async build() method only. There is no specification for connect and idle timeouts. Are these supposed to be configured in HttpClient ? Likewise for cookies and headers ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From pavel.rappo at oracle.com Fri Oct 9 13:18:51 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 9 Oct 2015 14:18:51 +0100 Subject: WebSocket client API In-Reply-To: References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> Message-ID: <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> > On 8 Oct 2015, at 20:51, Simone Bordet wrote: > > What it is still missing is the fact that there is no specification > about the onXXX methods regarding the lifecycle of the parameters > passed in. There is, actually. I have put it as a top-level javadoc, not as a javadoc to each single method. But that's an editorial problem, not a spec one. 1. Go to: http://cr.openjdk.java.net/~prappo/8087113/webrev.01/raw_files/new/src/java.httpclient/share/classes/java/net/httpclient/WebSocket.java 2. Look for occurrences of "reuse". There are exactly 2 of them :-) > For example, this is going to surprise users (simple echo): > > onBinary((ws, bytes, last) -> { > ws.sendBinary(bytes, last, null, (_, failure) -> {}); > } That would only surprise people who don't read javadoc. Or you want to say it's an inherently bad decision to allow people to reuse implementation's ByteBuffer? > It's not going to work because the send is async, and there is no > specification about who owns the ByteBuffer "bytes". > In my experience, it is bad to force applications to perform a copy > (not to mention that the copy could be really expensive, as WebSocket > frames could be large). I wonder what percentage of use cases this scenario corresponds to? Namely, a huge payload being received, transformed(?) and sent on the WebSocket. I ask this because I have to understand whether the added complexity worth the value provided by the functionality. > This would also lead to applications being forced to block waiting for > the send to complete How come? Send is not blocking. Unless one performs a rather odd thing: will block on handler completion before returning an onXXX call. > StatusCode makes little sense to me: it's a wrapper for int, and > nothing more. I would prefer to see primitive int in the signatures. Type safety, having documentation in a single place, maybe richer string representation (toString)? > Method isClosed() does not convey enough information. > WebSocket has built-in in the protocol half-closes, so it should be > able to report this information, like Socket returns > isInputShutDown(), isOutputShutDown() and isOpen(). > A simple enum would do. What's the use case other than debugging? > I think it's enough for the builder to provide an async build() method only. Makes sense. From chris.hegarty at oracle.com Fri Oct 9 13:52:02 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 9 Oct 2015 14:52:02 +0100 Subject: RFR [9] 8139179: URLStreamHandler* should link to URL ctor that specifies how factories/providers are located Message-ID: <5617C682.3030100@oracle.com> It was pointed out that the updated URL spec that describes how URL protocol handlers are located isn't prominent in the avadoc. In particular it was noted that it's not linked from URLStreamHandlerFactory or URLStreamHandlerProvider. Adding such links will make it clear how these classes tie together. diff --git a/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java b/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java --- a/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java +++ b/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java @@ -28,9 +28,9 @@ /** * This interface defines a factory for {@code URL} stream * protocol handlers. - *

- * It is used by the {@code URL} class to create a - * {@code URLStreamHandler} for a specific protocol. + * + *

A URL stream handler factory is used as specified in the + * {@linkplain java.net.URL#URL(String,String,int,String) URL constructor}. * * @author Arthur van Hoff * @see java.net.URL diff --git a/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java b/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java --- a/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java +++ b/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java @@ -41,6 +41,9 @@ * fully-qualified concrete URL stream handler provider class names, one per * line. * + *

URL stream handler providers are located at runtime, as specified in the + * {@linkplain java.net.URL#URL(String,String,int,String) URL constructor}. + * * @since 1.9 */ public abstract class URLStreamHandlerProvider -Chris. From Alan.Bateman at oracle.com Fri Oct 9 14:29:29 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 9 Oct 2015 15:29:29 +0100 Subject: RFR [9] 8139179: URLStreamHandler* should link to URL ctor that specifies how factories/providers are located In-Reply-To: <5617C682.3030100@oracle.com> References: <5617C682.3030100@oracle.com> Message-ID: <5617CF49.3060309@oracle.com> This looks okay to me. On 09/10/2015 14:52, Chris Hegarty wrote: > It was pointed out that the updated URL spec that describes how > URL protocol handlers are located isn't prominent in the > avadoc. In particular it was noted that it's not linked from > URLStreamHandlerFactory or URLStreamHandlerProvider. > > Adding such links will make it clear how these classes tie > together. > > diff --git > a/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java > b/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java > --- a/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java > +++ b/src/java.base/share/classes/java/net/URLStreamHandlerFactory.java > @@ -28,9 +28,9 @@ > /** > * This interface defines a factory for {@code URL} stream > * protocol handlers. > - *

> - * It is used by the {@code URL} class to create a > - * {@code URLStreamHandler} for a specific protocol. > + * > + *

A URL stream handler factory is used as specified in the > + * {@linkplain java.net.URL#URL(String,String,int,String) URL > constructor}. > * > * @author Arthur van Hoff > * @see java.net.URL > diff --git > a/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java > b/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java > --- > a/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java > +++ > b/src/java.base/share/classes/java/net/spi/URLStreamHandlerProvider.java > @@ -41,6 +41,9 @@ > * fully-qualified concrete URL stream handler provider class names, > one per > * line. > * > + *

URL stream handler providers are located at runtime, as > specified in the > + * {@linkplain java.net.URL#URL(String,String,int,String) URL > constructor}. > + * > * @since 1.9 > */ > public abstract class URLStreamHandlerProvider > > -Chris. From paul.sandoz at oracle.com Fri Oct 9 15:07:12 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 9 Oct 2015 17:07:12 +0200 Subject: WebSocket client API In-Reply-To: <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> Message-ID: <56B68859-39F2-41DB-978A-BCEB950DCCBF@oracle.com> Hi Pavel, Simone, One way to make progress is to get the basic shape of the API agreed on without flow/resource management features. That probably represented the simplest form. I think we are very close to that. Then lets iterate from that form and consider the additional features (back-pressure and resource management), see how they fit or push/pull the basic shape in various directions, and importantly see what additional demands are imposed on the developer. Also, I expect those features would need to be co-ordinated somewhat with the HTTP API to ensure a common pattern and possibly reuse of certain types. -- Regarding back pressure, i like what Michael/Pavel did with LongConsumer. Regarding resource management. This one is tricky. For HTTP Michael had a clever trick of piggy backing off the back pressure support, but i think that might be too clever as it is conflating two independent actions. I have a hunch that to do this effectively we may need some sort of resource supplier that is registered with the builder. That supplier defines the resource management strategy and requirements on a listener consuming resources (that is somewhat annoying because it couples the resource supplier with the listener). Paul. > On 9 Oct 2015, at 15:18, Pavel Rappo wrote: > > >> On 8 Oct 2015, at 20:51, Simone Bordet wrote: >> >> What it is still missing is the fact that there is no specification >> about the onXXX methods regarding the lifecycle of the parameters >> passed in. > > There is, actually. I have put it as a top-level javadoc, not as a javadoc to > each single method. But that's an editorial problem, not a spec one. > > 1. Go to: http://cr.openjdk.java.net/~prappo/8087113/webrev.01/raw_files/new/src/java.httpclient/share/classes/java/net/httpclient/WebSocket.java > 2. Look for occurrences of "reuse". There are exactly 2 of them :-) > >> For example, this is going to surprise users (simple echo): >> >> onBinary((ws, bytes, last) -> { >> ws.sendBinary(bytes, last, null, (_, failure) -> {}); >> } > > That would only surprise people who don't read javadoc. Or you > want to say it's an inherently bad decision to allow people to > reuse implementation's ByteBuffer? > >> It's not going to work because the send is async, and there is no >> specification about who owns the ByteBuffer "bytes". >> In my experience, it is bad to force applications to perform a copy >> (not to mention that the copy could be really expensive, as WebSocket >> frames could be large). > > I wonder what percentage of use cases this scenario corresponds to? Namely, a > huge payload being received, transformed(?) and sent on the WebSocket. I ask > this because I have to understand whether the added complexity worth the value > provided by the functionality. > >> This would also lead to applications being forced to block waiting for >> the send to complete > > How come? Send is not blocking. Unless one performs a rather odd thing: will > block on handler completion before returning an onXXX call. > >> StatusCode makes little sense to me: it's a wrapper for int, and >> nothing more. I would prefer to see primitive int in the signatures. > > Type safety, having documentation in a single place, maybe richer string > representation (toString)? > >> Method isClosed() does not convey enough information. >> WebSocket has built-in in the protocol half-closes, so it should be >> able to report this information, like Socket returns >> isInputShutDown(), isOutputShutDown() and isOpen(). >> A simple enum would do. > > What's the use case other than debugging? > >> I think it's enough for the builder to provide an async build() method only. > > Makes sense. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From joakim.erdfelt at gmail.com Fri Oct 9 18:21:32 2015 From: joakim.erdfelt at gmail.com (Joakim Erdfelt) Date: Fri, 9 Oct 2015 11:21:32 -0700 Subject: WebSocket client API In-Reply-To: <56B68859-39F2-41DB-978A-BCEB950DCCBF@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> <56B68859-39F2-41DB-978A-BCEB950DCCBF@oracle.com> Message-ID: On Fri, Oct 9, 2015 at 8:07 AM, Paul Sandoz wrote: > Hi Pavel, Simone, > > One way to make progress is to get the basic shape of the API agreed on > without flow/resource management features. That probably represented the > simplest form. I think we are very close to that. > > Then lets iterate from that form and consider the additional features > (back-pressure and resource management), see how they fit or push/pull the > basic shape in various directions, and importantly see what additional > demands are imposed on the developer. > The backpressure situation for a framing protocol like WebSocket is a different animal entirely vs more traditional streaming protocols like HTTP. Whatever was come up with back in the HttpURLConnection days is irrelevant today. However, what you came up with for HTTP2 in the new HttpClient could be a good candidate (if portable). Techniques like Flow (reactive-streams) simplifies and solves most of these issues in WebSocket. The backpressure should be based on the feedback at the network level, not the API, as there is almost never a 1::1 relationship of API call to network websocket frame result (except for extremely small, non-extension controlled, frames). Extensions and auto-fragmentation (a requirement for most of the WebSocket over HTTP2 proposals) mess with this in a large way. Then there is also the interleaving of DATA vs CONTROL frames that can mess with this backpressure behaviors as well. Punting on backpressure is going to be a big mistake. Every early implementation of WebSocket has suffered from this. There are many rewrites, API breaking rewrites, underway (in many implementations) to solve this. > Also, I expect those features would need to be co-ordinated somewhat with > the HTTP API to ensure a common pattern and possibly reuse of certain types. > WebSocket (currently) is an upgraded connection, once upgraded it is no longer HTTP or follows any HTTP semantics. Attempting to make it follow the HTTP connection behaviors is a mistake. Now, that being said, once the WebSocket over HTTP2 specs settle down, then this *might* be solved with the new HttpClient and its handling of HTTP2 specific streams (but this is a giant *IF* as there's 3 different proposals for how to accomplish this, some using HTTP2 protocol directly, some using tunneled behavior) However, the existing HTTP/1.1 upgraded to WebSocket connection will likely continue to be different behavior then what will come in the future. The API for the WebSocket client should attempt to find a good middle ground where the connection/framing behaviors are not hardcoded to assume HTTP/1.1 or HTTP/2. Even some of the assumptions in WebSocket currently (like no interleaved TEXT vs BINARY messages) might be invalid in this future WebSocket over HTTP2 world (one of the proposed solutions would utilize the built-in mux of HTTP2 to fix this), then what will you do? If you went with a Flow based solution then this nuance becomes irrelevant, as its up to the connection type to determine behavior. > > -- > > Regarding back pressure, i like what Michael/Pavel did with LongConsumer. > > Regarding resource management. This one is tricky. For HTTP Michael had a > clever trick of piggy backing off the back pressure support, but i think > that might be too clever as it is conflating two independent actions. I > have a hunch that to do this effectively we may need some sort of resource > supplier that is registered with the builder. That supplier defines the > resource management strategy and requirements on a listener consuming > resources (that is somewhat annoying because it couples the resource > supplier with the listener). > > Paul. > > I encourage you to implement now. Don't wait for us. Discover what we've been telling you quickly, the sooner the better, then you'll be on-board with what we have been telling you. We are quite confident of that. Don't repeat the mistakes of the past (bad backpressure, send queues in async, ignoring extensions, bad partial support, bad javadoc), don't reinvent the wheel (untested backpressure techniques), use what you have available to yourself (autobahn testsuite, Flow). - Joakim -------------- next part -------------- An HTML attachment was scrubbed... URL: From simone.bordet at gmail.com Fri Oct 9 20:12:50 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 9 Oct 2015 22:12:50 +0200 Subject: WebSocket client API In-Reply-To: <56B68859-39F2-41DB-978A-BCEB950DCCBF@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> <56B68859-39F2-41DB-978A-BCEB950DCCBF@oracle.com> Message-ID: Hi, On Fri, Oct 9, 2015 at 5:07 PM, Paul Sandoz wrote: > Regarding resource management. This one is tricky. For HTTP Michael had a clever trick of piggy backing off the back pressure support, but i think that might be too clever as it is conflating two independent actions. I have a hunch that to do this effectively we may need some sort of resource supplier that is registered with the builder. That supplier defines the resource management strategy and requirements on a listener consuming resources (that is somewhat annoying because it couples the resource supplier with the listener). > I have already proposed this: interface WebSocket void sendText(ByteBuffer text, boolean last, Consumer handler); void sendBinary(ByteBuffer binary, boolean last, Consumer handler); void sendPing(ByteBuffer payload, Consumer handler); void sendPong(ByteBuffer payload, Consumer handler); void close(int code, ByteBuffer reason, Consumer handler); String getSubProtocol(); void request(long); void abort(); CloseState getCloseState(); class Builder Builder listener(Listener l); Builder subProtocols(String... p); CompletableFuture build(); end interface Listener default void onOpen(WebSocket ws) { ws.request(1); } default void onClose(WebSocket ws, int code, ByteBuffer reason) { } default void onPing(WebSocket ws, ByteBuffer payload) { ws.request(1); } default void onPong(WebSocket ws, ByteBuffer payload) { ws.request(1); } default void onText(WebSocket ws, ByteBuffer text, boolean isLast, Consumer handler) { handler.accept(null); ws.request(1); } default void onBinary(WebSocket ws, ByteBuffer binary, boolean isLast, Consumer handler) { handler.accept(null); ws.request(1); } default void onError(WebSocket ws, Throwable failure) { } end enum CloseState { OPEN, REMOTE, LOCAL, CLOSED, ABORTED } end I hope it gets a second consideration. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From simone.bordet at gmail.com Mon Oct 12 22:07:27 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Tue, 13 Oct 2015 00:07:27 +0200 Subject: WebSocket client API In-Reply-To: <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <79C1BF82-F8CA-4ACB-94BB-0C483B93B8F4@oracle.com> Message-ID: Hi, On Fri, Oct 9, 2015 at 3:18 PM, Pavel Rappo wrote: >> On 8 Oct 2015, at 20:51, Simone Bordet wrote: >> For example, this is going to surprise users (simple echo): >> >> onBinary((ws, bytes, last) -> { >> ws.sendBinary(bytes, last, null, (_, failure) -> {}); >> } > > That would only surprise people who don't read javadoc. Or you > want to say it's an inherently bad decision to allow people to > reuse implementation's ByteBuffer? > >> It's not going to work because the send is async, and there is no >> specification about who owns the ByteBuffer "bytes". >> In my experience, it is bad to force applications to perform a copy >> (not to mention that the copy could be really expensive, as WebSocket >> frames could be large). > > I wonder what percentage of use cases this scenario corresponds to? Namely, a > huge payload being received, transformed(?) and sent on the WebSocket. I ask > this because I have to understand whether the added complexity worth the value > provided by the functionality. Allow me put some context around why I keep pushing for a completely async API. The current API proposal is very low level (and that's ok). The current API proposal deals with WebSocket frames, not messages. Take for example the exchange of JSON messages over WebSocket. The major JSON libraries available out there do not support asynchronous JSON parsing: they can parse a *whole* JSON (or block waiting for it), but they cannot parse chunks of it, asynchronously, threadlessly waiting for the chunks to arrive. This means that an application will have to recompose the whole JSON message from the frames it gets from the current API before passing it to the JSON parser. The current API proposal requires users to understand what is backpressure and that behind every invocation of onXXX() methods there must be a call to WebSocket.request(long) that triggers the read of more data by the implementation. And that WebSocket.request(long) must be invoked at the right time to produce proper backpressure, and this is even more important if the users use the current API proposal it with some other async libraries (that may do async socket, file, database, etc. I/O or other async processing). Given that the API is so low level, it would be a shame to not provide that couple of primitives, that currently are not present, that will make the API cover virtually all use cases and be very efficient. I specifically refer here to a callback for onXXX() methods and a better close status report. With such a low level API, it would then be trivial to lay on top a Flow-based API that deals with WebSocket messages (not frames) that most applications may find more palatable. Not a fan of Flow API ? Any other API can be based on top of a good, complete, low level API, even a blocking one. Yet, people that needs low-level, full control, highly efficient, little-or-no-allocating, non-copying, API will be able to have it (and build wonderful and efficient things on top of it). Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From pavel.rappo at oracle.com Tue Oct 13 09:40:28 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 13 Oct 2015 10:40:28 +0100 Subject: WebSocket client API In-Reply-To: References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> Message-ID: <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> Hi Simone, > On 8 Oct 2015, at 20:51, Simone Bordet wrote: > > The *API* should provide a callback to notify when the ByteBuffer has > been consumed. Here is a proposed mechanism for managing buffers used by Listener. 1. WebSocket.Builder gets 2 new methods (may not be an actual javadoc): /** * Specifies a function that provides {@code ByteBuffer}s for {@code * WebSocket} to receive Binary, Ping and Pong messages' payload to. * *

The function is called by {@code WebSocket} with a number of * bytes should remain in the required buffer. This serves as a hint * from the implementation to a user. * * @param provider the providing function * @return this builder */ public Builder byteBuffersForListener(IntFunction provider); /** * Specifies a function that provides {@code CharBuffer}s for {@code * WebSocket} to receive Text and Close messages' payload to. * *

The function is called by {@code WebSocket} with a number of * chars should remain in the required buffer. This serves as a hint * from the implementation to a user. * * @param provider the providing function * @return this builder */ public Builder charBuffersForListener(IntFunction provider); 2. If a user wants to use their own strategy of allocation/reuse they are fully in charge of this. For example: IntFunction provider = (r) -> { CharBuffer charBuffer = pool.getWithRemaining(r); if (charBuffer == null) charBuffer = CharBuffer.allocate(r); return charBuffer; }; ... builder.charBuffersForListener(provider) ... .buildAsync(); ... Later in the listener far, far away: @Override public void onText(CharBuffer payload, boolean isLast) { // ... ws.sendText(payload, isLast).thenRun(() -> pool.recycle(payload)); } Since the user constructs both the listener and the provider, they surely may now of each other, so the 'pool' can be easily captured by the listener. 3. On the other hand we could specify a set of predefined providers, and default behaviour like: * one off provider: constructs buffers on demand for one time use * reusing provider: always returns a buffer to the implementation at the end of the onXXX invocation In both cases above the user doesn't have to know about some additional recycle-handlers. Hence there's no need for onXXX methods to change their signatures to accommodate for it. What would you think about it? -Pavel From chris.hegarty at oracle.com Tue Oct 13 20:02:04 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 13 Oct 2015 21:02:04 +0100 Subject: WebSocket client API In-Reply-To: <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> Message-ID: <9B0A6E67-A078-406F-98B1-B4FADC7F8D56@oracle.com> Pavel, > On 13 Oct 2015, at 10:40, Pavel Rappo wrote: > > Hi Simone, > >> On 8 Oct 2015, at 20:51, Simone Bordet wrote: >> >> The *API* should provide a callback to notify when the ByteBuffer has >> been consumed. > > Here is a proposed mechanism for managing buffers used by Listener. I think that this is quite good. There is clearly a need for the receiving callbacks, onXXX methods, to allocate ( since they pass the payload as a ByteBuffer ), so exposing, through a small surface area, an API that gives better control over this allocation ( for the 0.1%that may want to do this ), without impacting on the 99.9% that couldn?t care less about it, seems reasonable. It is a nice side-effect that pinning / explicit release can be built on top of this, albeit with a small amount of work. -Chris. > 1. WebSocket.Builder gets 2 new methods (may not be an actual javadoc): > > /** > * Specifies a function that provides {@code ByteBuffer}s for {@code > * WebSocket} to receive Binary, Ping and Pong messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * bytes should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder byteBuffersForListener(IntFunction provider); > > /** > * Specifies a function that provides {@code CharBuffer}s for {@code > * WebSocket} to receive Text and Close messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * chars should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder charBuffersForListener(IntFunction provider); > > 2. If a user wants to use their own strategy of allocation/reuse they are fully > in charge of this. For example: > > IntFunction provider = (r) -> { > CharBuffer charBuffer = pool.getWithRemaining(r); > if (charBuffer == null) > charBuffer = CharBuffer.allocate(r); > return charBuffer; > }; > > ... > builder.charBuffersForListener(provider) ... .buildAsync(); > ... > > Later in the listener far, far away: > > @Override > public void onText(CharBuffer payload, boolean isLast) { > // ... > ws.sendText(payload, isLast).thenRun(() -> pool.recycle(payload)); > } > > Since the user constructs both the listener and the provider, they surely may now > of each other, so the 'pool' can be easily captured by the listener. > > 3. On the other hand we could specify a set of predefined providers, and default > behaviour like: > > * one off provider: constructs buffers on demand for one time use > * reusing provider: always returns a buffer to the implementation at the > end of the onXXX invocation > > In both cases above the user doesn't have to know about some additional > recycle-handlers. Hence there's no need for onXXX methods to change their > signatures to accommodate for it. > > What would you think about it? > > -Pavel > From simone.bordet at gmail.com Wed Oct 14 20:29:39 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 14 Oct 2015 22:29:39 +0200 Subject: WebSocket client API In-Reply-To: <9B0A6E67-A078-406F-98B1-B4FADC7F8D56@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> <9B0A6E67-A078-406F-98B1-B4FADC7F8D56@oracle.com> Message-ID: Hi, On Tue, Oct 13, 2015 at 10:02 PM, Chris Hegarty wrote: >> Here is a proposed mechanism for managing buffers used by Listener. > > I think that this is quite good. There is clearly a need for the receiving > callbacks, onXXX methods, to allocate ( since they pass the payload > as a ByteBuffer ), so exposing, through a small surface area, an API > that gives better control over this allocation ( for the 0.1%that may > want to do this ), without impacting on the 99.9% that couldn?t > care less about it, seems reasonable. It is a nice side-effect that > pinning / explicit release can be built on top of this, albeit with a > small amount of work. I am not sure I like it. This being a WebSocket API, I am not sure that exposing ByteBuffer pooling in the WebSocket API is a good idea. I agree that there is a need to configure this, but seems to me more an implementation detail or something you want to configure in a concrete implementation class, rather than in the API. IMHO there are other ways to solve the same problem than exposing a ByteBuffer pool API inside a WebSocket API. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From artem.smotrakov at oracle.com Thu Oct 15 14:49:32 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Thu, 15 Oct 2015 17:49:32 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <803BC362-2AEF-4469-B0A9-807C652C8742@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> <56153F6F.1080801@oracle.com> <803BC362-2AEF-4469-B0A9-807C652C8742@oracle.com> Message-ID: <561FBCFC.6000601@oracle.com> Hi Max, RFC 2617 [1] requires a user agent to use one of the challenges with the strongest auth scheme it understands (please see section 1.2): ... The user agent MUST choose to use one of the challenges with the strongest auth-scheme it understands and request credentials from the user based upon that challenge. ... This RFC doesn't mention fallbacks. But Negotiate/Kerberos authentication requires remote KDCs (AD controllers) which may be temporary down for some reason. In this case, it may be better to fallback to another scheme in something went wrong with Negotiate. Just to be able to authenticate users if something is wrong with infra. Taking into account the note about strongest auth scheme above, it might probably be better not to fallback to another schemes (like Digest -> Basic). It is possible to use "http.auth.preference" system property to specify preferred auth scheme. But setting this property means that the only specified scheme should be used (no fallback should happen at all). Maybe we can add a method to HttpURLConnection which can set a scheme in case of fallback. [1] http://www.ietf.org/rfc/rfc2617.txt Artem On 10/08/2015 11:41 AM, Wang Weijun wrote: >> On Oct 7, 2015, at 11:51 PM, Artem Smotrakov wrote: >> >> Hi Max, >> >> HttpURLConnection obtains credentials for HTTP authentication from Authenticator [1] implementation. Only one authenticator can be set in JVM instance. It can have built-in credentials, or do some interactions with user to get them. Theoretically, it can provide different credentials depending on a user/application/etc. I don't know how it is used in real application, but it seems to be a possible situation. When I was looking into this, I found a tech note [2] which says the following about fallback >> >> ... >> Fallback >> If the server has provided more than one authentication schemes (including Negotiate), according to the processing order mentioned in the last section, Java will try to challenge the Negotiate scheme. However, if the protocol cannot be established successfully (e.g. The kerberos configuration is not correct, or the server's hostname is not recorded in the KDC principal DB, or the username and password provided by Authenticator is wrong), then the 2nd strongest scheme will be automatically used. Attention : If http.auth.preference is set to SPNEGO or Kerberos, then we assume you only want to try the Negotiate scheme even if it fails. we won't fallback to any other scheme and your program will result in throwing an IOException saying it receives a 401 or 407 error from the HTTP response. >> ... > This was written by me. As I said in the previous mail, this is the only case where I think a fallback is worth doing. > >> As far as I understand, the current version of HttpURLConnection doesn't seem to follow this. That's why I think it needs to be fixed. Otherwise, the tech note [2] should be updated. > Not exactly. The tech note is mostly about config error and not about wrong credentials (see the e.g. line). It is quite rare someone provides wrong credentials for one scheme and correct ones for another. If it really could happen, we should consider a more generalized fallback and not only from Negotiate. > >> It doesn't look like a serious issue for me (that's why it is P3, or maybe it should be P4). Furthermore, it looks like nobody has had such a problem before because I didn't fine any bug about that at https://bugs.openjdk.java.net >> >> According to [2], Digest -> Basic fallback should not happen. > Yes, according to the words. But we are talking about what the most correct behavior is. > > Thanks > Max > >> HttpURLConnection is quite smart, and if I understand correctly, we have only "http.auth.preference" and Authenticator.setDefault() to control HTTP authentication process. Maybe we can make it more configurable. >> >> [1] http://docs.oracle.com/javase/8/docs/api/java/net/Authenticator.html >> [2] https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-auth.html >> >> Artem From peter.levart at gmail.com Thu Oct 15 15:24:26 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 15 Oct 2015 17:24:26 +0200 Subject: WebSocket client API In-Reply-To: <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> Message-ID: <561FC52A.9030701@gmail.com> Hi Pavel, There's a RFR being discussed on core-libs-dev: "8138696 : java.lang.ref.Cleaner - an easy to use alternative to finalization". Although it does not currently support it, I have been doing experiments with FinalReference(s) (a package-private subtype of Reference that is used internally to support finalization) that allowed me to code a simple pool of recyclable objects that works with the help of GC and reachability. The idea is very simple: ConcurrentLinkedDequeue pool = ...; Cleaner cleaner = ...; T getPooledInstance(Supplier factory) { T instance = pool.pollLast(); if (instance == null) { instance = factory.get(); } // register cleanup to recycle the instance cleaner.finalizableCleanup(instance, pool::addFirst); } .... finalize() method allows for cleanup to be performed at most once for each finalizable instance, but an API like Cleaner that explicitly registers a FinalReference with an instance would allow it to be recycled and re-registered multiple times. If you are interested to explore this possibility for recycling of ByteBuffer(s), I can try proposing this to the Cleaner API. Regards, Peter On 10/13/2015 11:40 AM, Pavel Rappo wrote: > Hi Simone, > >> On 8 Oct 2015, at 20:51, Simone Bordet wrote: >> >> The *API* should provide a callback to notify when the ByteBuffer has >> been consumed. > Here is a proposed mechanism for managing buffers used by Listener. > > 1. WebSocket.Builder gets 2 new methods (may not be an actual javadoc): > > /** > * Specifies a function that provides {@code ByteBuffer}s for {@code > * WebSocket} to receive Binary, Ping and Pong messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * bytes should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder byteBuffersForListener(IntFunction provider); > > /** > * Specifies a function that provides {@code CharBuffer}s for {@code > * WebSocket} to receive Text and Close messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * chars should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder charBuffersForListener(IntFunction provider); > > 2. If a user wants to use their own strategy of allocation/reuse they are fully > in charge of this. For example: > > IntFunction provider = (r) -> { > CharBuffer charBuffer = pool.getWithRemaining(r); > if (charBuffer == null) > charBuffer = CharBuffer.allocate(r); > return charBuffer; > }; > > ... > builder.charBuffersForListener(provider) ... .buildAsync(); > ... > > Later in the listener far, far away: > > @Override > public void onText(CharBuffer payload, boolean isLast) { > // ... > ws.sendText(payload, isLast).thenRun(() -> pool.recycle(payload)); > } > > Since the user constructs both the listener and the provider, they surely may now > of each other, so the 'pool' can be easily captured by the listener. > > 3. On the other hand we could specify a set of predefined providers, and default > behaviour like: > > * one off provider: constructs buffers on demand for one time use > * reusing provider: always returns a buffer to the implementation at the > end of the onXXX invocation > > In both cases above the user doesn't have to know about some additional > recycle-handlers. Hence there's no need for onXXX methods to change their > signatures to accommodate for it. > > What would you think about it? > > -Pavel > From weijun.wang at oracle.com Fri Oct 16 02:18:17 2015 From: weijun.wang at oracle.com (Wang Weijun) Date: Fri, 16 Oct 2015 10:18:17 +0800 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <561FBCFC.6000601@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> <56153F6F.1080801@oracle.com> <803BC362-2AEF-4469-B0A9-807C652C8742@oracle.com> <561FBCFC.6000601@oracle.com> Message-ID: <454C81B5-67AA-43AB-8348-53BD427CC872@oracle.com> Let's go back to the bug description: But no fallback happens if: 1. an HTTP server supports both Negotiate (via Kerberos) and Basic authentication schemes 2. first, a user provides correct Kerberos credentials, and a connection is successfully established with Negotiate scheme 3. then, a user provides wrong Kerberos credentials, but correct Basic credentials So, with #2, the HTTPP connection already succeeds. When will #3 happen? Visiting another page on the same server and see another 401? If this is a new connection, does HttpURLConnection still remember #2? Sorry for asking these. I have always been afraid of HttpURLConnection and although I've made some modifications to the code, I never dare say I fully understand it, at least not today. Thanks Max From pavel.rappo at oracle.com Fri Oct 16 10:35:15 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 16 Oct 2015 11:35:15 +0100 Subject: WebSocket client API In-Reply-To: References: Message-ID: Hi, Here's a second update on the WebSocket API: webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.02/ javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.02/ Main differences from the previous version: * WebSocket has become an abstract class * New methods in Builder: headers, connectTimeout * WebSocket.Builder no longer accepts HttpRequest.Builder; only HttpClient * One Listener instead of many onXXX handlers * Completion handlers with custom contexts are gone; send methods return CompletableFuture * onXXX methods in Listener return CompletableFuture as a means of asynchronous completion signalling * StatusCode is now ClosureCode -Pavel > On 31 Aug 2015, at 15:30, Pavel Rappo wrote: > > Hi, > > I would appreciate if you help to review a WebSocket client API proposed for > JDK 9. This work is a part of JEP-110 [1] and has started its public path with > HTTP client review in March this year [2]. > > Proposed WebSocket API is relatively small and focuses on convenient exchange of > data in a fully asynchronous fashion. API consists of 6 types located in the > java.net package [3]: > > 1. WebSocket > 2. WebSocket.Builder > 3. WebSocket.Incoming > 4. WebSocket.Incoming.Chunks > 5. WebSocket.Outgoing > 6. WebSocketException > > Starting point is a class description for java.net.WebSocket. Along with > in-javadoc examples, several API test samples are provided in the webrev [4] and > named test/java/net/WebSocket/Example%.java. They are only for informational > purposes and won't be included in the final version of the API. > > I would appreciate any feedback on this API. Thanks. > > ------------------------------------------------------------------------------- > [1] http://openjdk.java.net/jeps/110 > [2] http://mail.openjdk.java.net/pipermail/net-dev/2015-March/008932.html > [3] http://cr.openjdk.java.net/~prappo/8087113/javadoc.00/ > [4] http://cr.openjdk.java.net/~prappo/8087113/webrev.00/ > > -Pavel From sebastian.sickelmann at gmx.de Fri Oct 16 13:14:46 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Fri, 16 Oct 2015 15:14:46 +0200 Subject: RFR: JDK-8022748 (new URI(u.toString()).equals(u), does not hold with paths containing colons In-Reply-To: <5613ACF3.10802@gmx.de> References: <5613ACF3.10802@gmx.de> Message-ID: <5620F846.7090606@gmx.de> Hi, is there someone who wants to sponsor/review my suggested change? -- Sebastian On 10/06/2015 01:13 PM, Sebastian Sickelmann wrote: > Hi, > > i investigated the problem described in JDK-8022748[1] i found that > the parser needed to be rescued for confusion while handling relative URIs. > > A URI created through the relativize-method is schemaless and so it > need to handle the special-case (a colon in the path-element). While > there is also another way to handle it (encode the colon as %3A) i think > we should not choose to encode everything. First it would introduce another > style of special-case handling of a colon in the path (see the method > maybeAddLeadingDot which is used while normalizing) and when resolving > it back we would need to decode it back or leave it the encoded way which > is not suggest by RFC2396 section "1.5 URI Transcribability". > > Also in Section 5 of RFC2396 it is suggested to rescue a colon in the > path-element in a relative URI through prepending a "./" just like in > maybeAddLeadingDot. > > I am not sure if it is worth to refactor the split(),join(),normalize() > methods to provide a reusable colon-detection and -handling. > > So, please find my webrev with a simpler solution at: > > http://cr.openjdk.java.net/~sebastian/8022748/webrev.00/ > > -- Sebastian > [1] https://bugs.openjdk.java.net/browse/JDK-8022748 -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.smotrakov at oracle.com Fri Oct 16 15:08:28 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Fri, 16 Oct 2015 18:08:28 +0300 Subject: [9] RFR 8138953: HttpURLConnection doesn't fallback to another auth scheme if negotiate process failed In-Reply-To: <454C81B5-67AA-43AB-8348-53BD427CC872@oracle.com> References: <5614FFCD.5070903@oracle.com> <1303268E-2E4A-442E-B260-62AA34CE0777@oracle.com> <56153F6F.1080801@oracle.com> <803BC362-2AEF-4469-B0A9-807C652C8742@oracle.com> <561FBCFC.6000601@oracle.com> <454C81B5-67AA-43AB-8348-53BD427CC872@oracle.com> Message-ID: <562112EC.2000300@oracle.com> Hi Max, Please see inline. On 10/16/2015 05:18 AM, Wang Weijun wrote: > Let's go back to the bug description: > > But no fallback happens if: > > 1. an HTTP server supports both Negotiate (via Kerberos) and Basic authentication schemes > 2. first, a user provides correct Kerberos credentials, and a connection is successfully established with Negotiate scheme > 3. then, a user provides wrong Kerberos credentials, but correct Basic credentials > > So, with #2, the HTTPP connection already succeeds. When will #3 happen? At #3, a user creates a new HttpURLConnection instance (in the same JVM), and tries to connect to the same HTTP server again. Please see the test I added for this bug: http://cr.openjdk.java.net/~asmotrak/8138953/webrev.02/ > Visiting another page on the same server and see another 401? Yes, it uses the same page on the same HTTP server. I updated the test to visit another page, and it fails on JDK 9 b83, and succeeds with the fix (please see the webrev above). > If this is a new connection, does HttpURLConnection still remember #2? Yes, HttpURLConnection is quite smart, and has a number of caches. For example, keep-alive cache, cache for auth data (for the same realms only). Artem > > Sorry for asking these. I have always been afraid of HttpURLConnection and although I've made some modifications to the code, I never dare say I fully understand it, at least not today. > > Thanks > Max > From simone.bordet at gmail.com Fri Oct 16 21:08:24 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 16 Oct 2015 23:08:24 +0200 Subject: No subject Message-ID: Hi, On Fri, Oct 16, 2015 at 12:35 PM, Pavel Rappo wrote: > Hi, > > Here's a second update on the WebSocket API: > > webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.02/ > javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.02/ > > Main differences from the previous version: > > * WebSocket has become an abstract class What is the rationale behind this ? Just to avoid subclassing ? Why subclassing is negated ? I think it's best it remains an interface: it would allow people to write wrappers, use java.lang.reflect.Proxy, etc. > * New methods in Builder: headers, connectTimeout I still don't understand the headers() signature what benefits brings. What problems would have a chainable: Builder header(name, value) ? Clear semantic and no exceptions about an odd number of String passed in. I am not sure that passing the listener to the Builder constructor is right. There are applications that just stream in one sense, so there would be no listener needed, so it appears strange that this is a required constructor parameter rather than just another builder method. > * WebSocket.Builder no longer accepts HttpRequest.Builder; only HttpClient > * One Listener instead of many onXXX handlers I tried to write a few examples with this API, and it's a bit cumbersome to use. Below my feedback: 1. I think there is a mistake in onText(CharBuffer, boolean). Should not be onText(CharSequence, boolean) ? I don't think CharBuffer can handle properly UTF-8. 2. The split of the flow control functionality into a LongConsumer is a little cumbersome to use. This LongConsumer object needs to be carried around, typically in conjunction with the WebSocket object, which forces applications to pass around 2 parameters when one would have sufficed. 3. The absence of the WebSocket parameter from onXXX() methods makes the API cumbersome to use. It forces applications to write this boilerplate over and over: new WebSocket.Builder("ws://localhost:8080/path", new WebSocket.Listener() { public WebSocket webSocket; public LongConsumer controller; @Override public void onOpen(WebSocket webSocket, LongConsumer flowController) { this.webSocket = webSocket; this.controller = flowController; flowController.accept(1); } ... } The WebSocket and LongConsumer needs to be saved from onOpen() to be used in other methods, and it's boilerplate code that needs to be written all the time. I suggest to consider to modify the signature of the onXXX() methods to: CompletableFuture onText(WebSocket, CharSequence, boolean) and to merge again LongConsumer back into WebSocket. That would make the API soo much easier to use. 4. WebSocket.Listener should have all methods implemented with a default implementation. It's just too much to have to implement them all when all I want is to receive text messages. If WebSocket is passed as parameter to onXXX() methods and LongConsumer is merged back into WebSocket, it would be trivial to write a correct default implementation of the Listener interface (which is now impossible). > * Completion handlers with custom contexts are gone; Finally :) > * send methods return CompletableFuture This parameter is typically not used. When an application calls sendXXX() it already has the WebSocket reference in scope, so it would be available anyway. Furthermore, the removal of the flow control functionality from WebSocket makes the code a little weird (a simple echo below): CompletableFuture onBinary(ByteBuffer payload, boolean isLast) { return this.ws1.sendBinary(payload, isLast).thenAccept(ws2 -> this.flowController.accept(1)); } As you can see there are two "ws" references when one would be enough, and the lambda passed to thenAccept() takes "ws2" that is useless, because the application needs the flow controller, not the WebSocket. And that flow controller is associated to "ws1" (from onOpen()), so having to deal with ws2 is confusing (is it the same reference ? it's something else ?) I think that re-merging the flow controller functionality in WebSocket and passing WebSocket as first parameter to onXXX() methods will simplify a lot, and make possible to write completely stateless listeners that would come handy when you are opening a large number of connections (you can only use one instance, rather than one per connection). Method onOpen() still has reason to exist, of course. > * onXXX methods in Listener return CompletableFuture as a means of > asynchronous completion signalling The downside of using CF is the allocation of CF instances that the application is forced to make to comply with the API. Given that this is a low-level API, I have a preference for using Consumer in all cases. > * StatusCode is now ClosureCode "Closure" typically means something different to a programmer. I would rename to CloseCode or similar. Given that exceptions are relayed via methods (and not try/catch), I find no use for WebSocketException subclasses. There may be a need for WebSocketException in case of protocol errors or spec violations, but I don't see how WebSocketException.AsynchronousClose or WebSocketException.Handshake are any useful. WebSocketException.AsynchronousClose seems a duplication of java.io.AsynchronousCloseException, and WebSocketException.Handshake can just be relayed as a WebSocketException. I would rather drop the inner classes and just keep WebSocketException only. The subclasses also break an "implicit" naming convention where exceptions class names always have the "Exception" suffix. The JDK consistently use this naming convention apart for some 1.0 class for historical reasons (e.g. ThreadDeath). Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From simone.bordet at gmail.com Fri Oct 16 21:11:11 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 16 Oct 2015 23:11:11 +0200 Subject: WebSocket client API Message-ID: Resending with the right subject... not sure what Gmail was thinking. Sorry for the double send. Hi, On Fri, Oct 16, 2015 at 12:35 PM, Pavel Rappo wrote: > Hi, > > Here's a second update on the WebSocket API: > > webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.02/ > javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.02/ > > Main differences from the previous version: > > * WebSocket has become an abstract class What is the rationale behind this ? Just to avoid subclassing ? Why subclassing is negated ? I think it's best it remains an interface: it would allow people to write wrappers, use java.lang.reflect.Proxy, etc. > * New methods in Builder: headers, connectTimeout I still don't understand the headers() signature what benefits brings. What problems would have a chainable: Builder header(name, value) ? Clear semantic and no exceptions about an odd number of String passed in. I am not sure that passing the listener to the Builder constructor is right. There are applications that just stream in one sense, so there would be no listener needed, so it appears strange that this is a required constructor parameter rather than just another builder method. > * WebSocket.Builder no longer accepts HttpRequest.Builder; only HttpClient > * One Listener instead of many onXXX handlers I tried to write a few examples with this API, and it's a bit cumbersome to use. Below my feedback: 1. I think there is a mistake in onText(CharBuffer, boolean). Should not be onText(CharSequence, boolean) ? I don't think CharBuffer can handle properly UTF-8. 2. The split of the flow control functionality into a LongConsumer is a little cumbersome to use. This LongConsumer object needs to be carried around, typically in conjunction with the WebSocket object, which forces applications to pass around 2 parameters when one would have sufficed. 3. The absence of the WebSocket parameter from onXXX() methods makes the API cumbersome to use. It forces applications to write this boilerplate over and over: new WebSocket.Builder("ws://localhost:8080/path", new WebSocket.Listener() { public WebSocket webSocket; public LongConsumer controller; @Override public void onOpen(WebSocket webSocket, LongConsumer flowController) { this.webSocket = webSocket; this.controller = flowController; flowController.accept(1); } ... } The WebSocket and LongConsumer needs to be saved from onOpen() to be used in other methods, and it's boilerplate code that needs to be written all the time. I suggest to consider to modify the signature of the onXXX() methods to: CompletableFuture onText(WebSocket, CharSequence, boolean) and to merge again LongConsumer back into WebSocket. That would make the API soo much easier to use. 4. WebSocket.Listener should have all methods implemented with a default implementation. It's just too much to have to implement them all when all I want is to receive text messages. If WebSocket is passed as parameter to onXXX() methods and LongConsumer is merged back into WebSocket, it would be trivial to write a correct default implementation of the Listener interface (which is now impossible). > * Completion handlers with custom contexts are gone; Finally :) > * send methods return CompletableFuture This parameter is typically not used. When an application calls sendXXX() it already has the WebSocket reference in scope, so it would be available anyway. Furthermore, the removal of the flow control functionality from WebSocket makes the code a little weird (a simple echo below): CompletableFuture onBinary(ByteBuffer payload, boolean isLast) { return this.ws1.sendBinary(payload, isLast).thenAccept(ws2 -> this.flowController.accept(1)); } As you can see there are two "ws" references when one would be enough, and the lambda passed to thenAccept() takes "ws2" that is useless, because the application needs the flow controller, not the WebSocket. And that flow controller is associated to "ws1" (from onOpen()), so having to deal with ws2 is confusing (is it the same reference ? it's something else ?) I think that re-merging the flow controller functionality in WebSocket and passing WebSocket as first parameter to onXXX() methods will simplify a lot, and make possible to write completely stateless listeners that would come handy when you are opening a large number of connections (you can only use one instance, rather than one per connection). Method onOpen() still has reason to exist, of course. > * onXXX methods in Listener return CompletableFuture as a means of > asynchronous completion signalling The downside of using CF is the allocation of CF instances that the application is forced to make to comply with the API. Given that this is a low-level API, I have a preference for using Consumer in all cases. > * StatusCode is now ClosureCode "Closure" typically means something different to a programmer. I would rename to CloseCode or similar. Given that exceptions are relayed via methods (and not try/catch), I find no use for WebSocketException subclasses. There may be a need for WebSocketException in case of protocol errors or spec violations, but I don't see how WebSocketException.AsynchronousClose or WebSocketException.Handshake are any useful. WebSocketException.AsynchronousClose seems a duplication of java.io.AsynchronousCloseException, and WebSocketException.Handshake can just be relayed as a WebSocketException. I would rather drop the inner classes and just keep WebSocketException only. The subclasses also break an "implicit" naming convention where exceptions class names always have the "Exception" suffix. The JDK consistently use this naming convention apart for some 1.0 class for historical reasons (e.g. ThreadDeath). Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From pavel.rappo at oracle.com Sat Oct 17 18:56:27 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Sat, 17 Oct 2015 19:56:27 +0100 Subject: WebSocket client API In-Reply-To: References: Message-ID: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> > On 16 Oct 2015, at 22:11, Simone Bordet wrote: > > Hi, > > On Fri, Oct 16, 2015 at 12:35 PM, Pavel Rappo wrote: >> Hi, >> >> Here's a second update on the WebSocket API: >> >> webrev: http://cr.openjdk.java.net/~prappo/8087113/webrev.02/ >> javadoc: http://cr.openjdk.java.net/~prappo/8087113/javadoc.02/ >> >> Main differences from the previous version: >> >> * WebSocket has become an abstract class > > What is the rationale behind this ? > Just to avoid subclassing ? > Why subclassing is negated ? > I think it's best it remains an interface: it would allow people to > write wrappers, use java.lang.reflect.Proxy, etc. This argument can be used for pretty much any class out there. You see, it's a question of whether subtyping of X should be allowed or disabled, if X has not been designed with it in mind. >> * New methods in Builder: headers, connectTimeout > > I still don't understand the headers() signature what benefits brings. > What problems would have a chainable: Builder header(name, value) ? > Clear semantic and no exceptions about an odd number of String passed in. The reason was that other Builder's methods (now only 2) have a "replace" semantics rather than "add": *

If a particular intermediate method is not called, an appropriate * default value (or behavior) is used. A repeated call to an intermediate * method overwrites the previous value (or overrides the previous * behaviour), given no exception is thrown. If Builder.header was with "add" semantics, it wouldn't be possible to remove added headers. But now I probably agree with you. In this case it would be better for the API to provide an "add" behaviour, mostly because it's more compile time checking friendly. > I am not sure that passing the listener to the Builder constructor is right. > There are applications that just stream in one sense, so there would > be no listener needed, so it appears strange that this is a required > constructor parameter rather than just another builder method. Maybe you're right. For now let's remember this argument as (A) to return to it later. >> * WebSocket.Builder no longer accepts HttpRequest.Builder; only HttpClient >> * One Listener instead of many onXXX handlers > > I tried to write a few examples with this API, and it's a bit cumbersome to use. > Below my feedback: > > 1. I think there is a mistake in onText(CharBuffer, boolean). Should > not be onText(CharSequence, boolean) ? > I don't think CharBuffer can handle properly UTF-8. I'm not sure I understand what you mean by "handle properly UTF-8". First of all, CharBuffer implements CharSequence. Secondly, if a user decides to create a ByteBuffer out of a Text message, CharBuffer is the way to go. It's a native class for charset decoding/encoding: java.nio.charset.CharsetEncoder#encode(java.nio.CharBuffer). No additional conversions are needed. On the other hand, if the user needs a String, then simple CharBuffer.toString would do. In my opinion, using CharBuffer directly is the best of two worlds. > 3. The absence of the WebSocket parameter from onXXX() methods makes > the API cumbersome to use. > It forces applications to write this boilerplate over and over: > > new WebSocket.Builder("ws://localhost:8080/path", new WebSocket.Listener() > { > public WebSocket webSocket; > public LongConsumer controller; > > @Override > public void onOpen(WebSocket webSocket, LongConsumer flowController) > { > this.webSocket = webSocket; > this.controller = flowController; > flowController.accept(1); > } > > ... > } > > The WebSocket and LongConsumer needs to be saved from onOpen() to be > used in other methods, and it's boilerplate code that needs to be > written all the time. > > I suggest to consider to modify the signature of the onXXX() methods to: > > CompletableFuture onText(WebSocket, CharSequence, boolean) > > and to merge again LongConsumer back into WebSocket. That would make > the API soo much easier to use. Easier? In some cases, probably. On the other WebSocket.request(long) is an internal part of the API that only the Listener should talk to (the same is with Flow.Subscriber and Flow.Subscription). Moreover, it would be strange (remember argument (A)?) that one could request something not having any means of listening to it. Do we have any good reasons to merge flowController with WebSocket other than boiler-plate eliminating? Because we just might lose good level of separation of concerns. > 4. WebSocket.Listener should have all methods implemented with a > default implementation. It's just too much to have to implement them > all when all I want is to receive text messages. > If WebSocket is passed as parameter to onXXX() methods and > LongConsumer is merged back into WebSocket, it would be trivial to > write a correct default implementation of the Listener interface > (which is now impossible). Summing up (A) and the passage above, I would provide some AbstractListener that would take care of a boilerplate code. >> * Completion handlers with custom contexts are gone; > > Finally :) > >> * send methods return CompletableFuture > > This parameter is typically not used. It's too early to refer to something in this API as "typically" :-) Or you mean for CF in general? > When an application calls sendXXX() it already has the WebSocket > reference in scope, so it would be available anyway. You're right. > completely stateless > listeners that would come handy when you are opening a large number of > connections (you can only use one instance, rather than one per > connection). Good argument! We may try to accomplish it a bit differently. But I'm against of merging flowController with WebSocket. > Method onOpen() still has reason to exist, of course. > >> * onXXX methods in Listener return CompletableFuture as a means of >> asynchronous completion signalling > > The downside of using CF is the allocation of CF instances that the > application is forced to make to comply with the API. > Given that this is a low-level API, I have a preference for using > Consumer in all cases. In terms of composability those approaches are pretty much the same: void onText(CharBuffer text, boolean isLast, Consumer handler) { Consumer throwableConsumer = (t) -> { if (t == null) flowController.accept(1); }; sendText(text, isLast, throwableConsumer.andThen(handler::accept)); } or CompletableFuture onText(CharBuffer text, boolean isLast) { return ws.sendText(text, isLast).thenRun(() -> flowController.accept(1)); } The difference might become serious if we think about one-shot vs reusable objects. In general. Not about CF vs handler. Reusable handlers are more prone to bugs as far as I can see now. Because they are inherently same objects but being used in different time (context). Thus their .accept(Throwable) would mean different thing in different time. Hence more caution would be needed, especially on the receiving side, when the app gets an implementation's handler. Implementation's code will have to become a lot more sophisticated in this case, because it will now have to correlate a handler to an invocation. (I know nobody cares about the implementation. I'm just saying.) As for the API, well CompletableFuture is surely more nice compared to a bare Consumer. But let me try to implement both things and we could choose later. We probably don't want to double the number of sending methods, do we? :) >> * StatusCode is now ClosureCode > > "Closure" typically means something different to a programmer. > I would rename to CloseCode or similar. I dunno. RFC 6455 uses "closure" all over the the place, e.g.: As defined in Sections 5.5.1 and 7.4, a Close control frame may contain a status code indicating a reason for closure. I think we should ask for an opinion from others. But hey, it's not a com.sun.tools.javac package we're designing a WebSocket for. I'm sure there won't be any ambiguity. > Given that exceptions are relayed via methods (and not try/catch), I > find no use for WebSocketException subclasses. What exclusive things does try-catch provide we couldn't emulate with good old instanceof/getClass()? Well, maybe just a tiny bit of help from a compiler. Or you say with method-relayed exceptions there's no need to know their type at all? > There may be a need for WebSocketException in case of protocol errors > or spec violations, but I don't see how > WebSocketException.AsynchronousClose or WebSocketException.Handshake > are any useful. Handshake provides a java.net.http.HttpResponse (Joakim [*] has noticed it might be important). And you say you don't see how a user would benefit from knowing some of their send operations have been terminated due to asynchronous close? > WebSocketException.AsynchronousClose seems a duplication of > java.io.AsynchronousCloseException, and WebSocketException.Handshake > can just be relayed as a WebSocketException. You mean java.nio.channels.AsynchronousCloseException? Well yes, but it's not reusable. The wording and the package confine this exception to channels and buffers. > I would rather drop the inner classes and just keep WebSocketException only. > > The subclasses also break an "implicit" naming convention where > exceptions class names always have the "Exception" suffix. The JDK > consistently use this naming convention apart for some 1.0 class for > historical reasons (e.g. ThreadDeath). Good argument. Will return to it later. Thanks! > Thanks ! > > -- > Simone Bordet > http://bordet.blogspot.com > --- > Finally, no matter how good the architecture and design are, > to deliver bug-free software with optimal performance and reliability, > the implementation technique must be flawless. Victoria Livschitz ------------------------------------------------------------------------------- [*] http://mail.openjdk.java.net/pipermail/net-dev/2015-August/009078.html From joakim.erdfelt at gmail.com Sat Oct 17 21:42:28 2015 From: joakim.erdfelt at gmail.com (Joakim Erdfelt) Date: Sat, 17 Oct 2015 14:42:28 -0700 Subject: WebSocket client API In-Reply-To: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: On Sat, Oct 17, 2015 at 11:56 AM, Pavel Rappo wrote: > > >> * WebSocket.Builder no longer accepts HttpRequest.Builder; only > HttpClient > >> * One Listener instead of many onXXX handlers > > > > I tried to write a few examples with this API, and it's a bit cumbersome > to use. > > Below my feedback: > > > > 1. I think there is a mistake in onText(CharBuffer, boolean). Should > > not be onText(CharSequence, boolean) ? > > I don't think CharBuffer can handle properly UTF-8. > > I'm not sure I understand what you mean by "handle properly UTF-8". First > of > all, CharBuffer implements CharSequence. Secondly, if a user decides to > create a > ByteBuffer out of a Text message, CharBuffer is the way to go. > > It's a native class for charset decoding/encoding: > > java.nio.charset.CharsetEncoder#encode(java.nio.CharBuffer). > > No additional conversions are needed. On the other hand, if the user needs > a > String, then simple CharBuffer.toString would do. In my opinion, using > CharBuffer directly is the best of two worlds. > You are required, per the RFC6455 spec, to validate incoming and outgoing TEXT messages are valid UTF8. (also Handshake and Close Reason Messages) http://tools.ietf.org/html/rfc6455#section-8.1 Relying on the JVM built-in replacement character behavior for invalid UTF8 sequences will cause many bugs. If you rely on the CharsetEncoder and CharBuffer you'll wind up with situations where you are changing the data. You need to rely on an implementation that does not use replacement characters and throws exceptions on bad Write, and on bad received TEXT messages you MUST close the connection with a 1007 error code. - Joakim -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Sat Oct 17 22:08:32 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Sat, 17 Oct 2015 23:08:32 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: Hi Joakim, > On 17 Oct 2015, at 22:42, Joakim Erdfelt wrote: > > You are required, per the RFC6455 spec, to validate incoming and outgoing TEXT messages are valid UTF8. > (also Handshake and Close Reason Messages) > > http://tools.ietf.org/html/rfc6455#section-8.1 > > Relying on the JVM built-in replacement character behavior for invalid UTF8 sequences will cause many bugs. > If you rely on the CharsetEncoder and CharBuffer you'll wind up with situations where you are changing the data. > > You need to rely on an implementation that does not use replacement characters and throws exceptions on bad Write, > and on bad received TEXT messages you MUST close the connection with a 1007 error code. The only thing I was trying to say is that in my opinion there's no extra confidence in UTF-8 representability that CharSequence or even String gives us compared to what CharBuffer does. On the other hand, compared to any other implementation of CharSequence or String, CharBuffer is the most charset-friendly thing we have: CharsetEncoder/CharsetDecoder speaks in CharBuffers. Sorry, but I believe I haven't proposed to rely on JDK built-in replacement characters. Moreover, being able to tell the decoder/encoder to throw exceptions (e.g. UnmappableCharacterException) on incorrect input was one of the main reasons to use CharsetEncoder/Decoder. And not, say, String.getBytes(StandardCharsets.UTF_8). Thanks. From andrej.golovnin at gmail.com Mon Oct 19 06:42:13 2015 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Mon, 19 Oct 2015 08:42:13 +0200 Subject: WebSocket client API In-Reply-To: References: Message-ID: Hi Pavel, the JavaDocs for the method HttpClient.Builder#proxy(java.net.ProxySelector selector) say: Sets a ProxySelector for this client. If no selector is set, then no proxies are used. If a null parameter is supplied then the system wide default proxy selector is used. Really? For me it means that nearly every application which would like to use the new HttpClient/WebSocket must call this method with null value. Just think about Java WebStart applications and applets: you never know if there is a proxy or not. IMHO the JavaDocs for this method should look like this: Sets a ProxySelector for this client. If no proxy selector is set, then the system wide default proxy selector is used. @throws IllegalArgumentException if selector is null. I'm sure the most developers out there should be able to provide their own implementation of the java.net.ProxySelector class which just returns Collections.singletonList(Proxy.NO_PROXY) in the method ProxySelector#select(URI uri) if they need direct access to a server. And if you would like to simplify the life of the developers which need a direct access to a server, then you could add a new method to the builder: HttpClient.Builder#noProxy() I think it is better than to add a special meaning to the null value. And I think that the phrase with "If no proxy selector is set, then the system wide default proxy selector is used." should be a part of the JavaDocs of the class HttpClient.Builder and the method HttpClient.getDefault() to make clear the behaviour of the created/returned HttpClient. Best regards, Andrej Golovnin From simone.bordet at gmail.com Mon Oct 19 21:37:06 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Mon, 19 Oct 2015 23:37:06 +0200 Subject: WebSocket client API In-Reply-To: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: Hi, On Sat, Oct 17, 2015 at 8:56 PM, Pavel Rappo wrote: >>> * WebSocket has become an abstract class >> >> What is the rationale behind this ? >> Just to avoid subclassing ? >> Why subclassing is negated ? >> I think it's best it remains an interface: it would allow people to >> write wrappers, use java.lang.reflect.Proxy, etc. > > This argument can be used for pretty much any class out there. You see, it's a > question of whether subtyping of X should be allowed or disabled, if X has not > been designed with it in mind. It's pretty trivial to convert WebSocket to an interface, and would allow applications much more flexibility. There are tons of frameworks out there that rely on interfaces to provide method interception (e.g. AOP, Spring Proxies, etc.). I think it would be a mistake to make WebSocket non subclassable / non implementable. There are a ton of use cases: logging, filtering, transformation of content, buffering, etc. > If Builder.header was with "add" semantics, it wouldn't be possible to remove > added headers. But now I probably agree with you. In this case it would be > better for the API to provide an "add" behaviour, mostly because it's more > compile time checking friendly. Now I am confused :) Are you saying that if you do: builder.headers("foo", "bar", "baz").headers("one", "two", "three") then "foo", "bar" and "baz" are removed ? > Easier? In some cases, probably. On the other WebSocket.request(long) is an > internal part of the API that only the Listener should talk to (the same is with > Flow.Subscriber and Flow.Subscription). In proxies and WebSocket applications in general, it's the opposite: these applications want to call request(long) after they have written data, not after they read it, to produce backpressure. And the write code is typically far, far away from the Listener code, which means that you have to carry around the LongConsumer, which is cumbersome. >>> * send methods return CompletableFuture >> >> This parameter is typically not used. > > It's too early to refer to something in this API as "typically" :-) > Or you mean for CF in general? I mean the WebSocket type parameter. It is typically not used because the WebSocket object must be present in the lexical scope, otherwise you cannot call the sendXXX methods that returns the CF. Hence the CF parameter is typically not used - it's indeed totally redundant. I'm fine with sendXXX() methods to return CF. > But I'm against of merging flowController with WebSocket. Yet, you did not bring any technical reasons for the split, and "separation of concerns" is just not enough because it can mean anything (and it may even be wrong). We are exchanging more parameters to onOpen() (the LongConsumer), more boilerplate that applications must write, more classes to avoid the boilerplate and bloat the API (an additional AbstractListener), more work for applications, more forced allocation, versus one more method in WebSocket that would get rid of all of that. If you write a chat application with these API, you will see that having WebSocket.request() simplifies the code by *a lot*. I have done that, and I am reporting my impressions. If you have an example where splitting WebSocket and LongConsumer is actually beneficial to application writing (as in less and clearer code), I am interested in learning from it. > As for the API, well CompletableFuture is surely more nice compared to a bare > Consumer. But let me try to implement both things and we could choose later. > We probably don't want to double the number of sending methods, do we? :) Agreed, either CF or Consumer, not both. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From peter.levart at gmail.com Tue Oct 20 08:39:15 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 20 Oct 2015 10:39:15 +0200 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: <5625FDB3.3080001@gmail.com> On 10/18/2015 12:08 AM, Pavel Rappo wrote: > Hi Joakim, > >> On 17 Oct 2015, at 22:42, Joakim Erdfelt wrote: >> >> You are required, per the RFC6455 spec, to validate incoming and outgoing TEXT messages are valid UTF8. >> (also Handshake and Close Reason Messages) >> >> http://tools.ietf.org/html/rfc6455#section-8.1 >> >> Relying on the JVM built-in replacement character behavior for invalid UTF8 sequences will cause many bugs. >> If you rely on the CharsetEncoder and CharBuffer you'll wind up with situations where you are changing the data. >> >> You need to rely on an implementation that does not use replacement characters and throws exceptions on bad Write, >> and on bad received TEXT messages you MUST close the connection with a 1007 error code. > The only thing I was trying to say is that in my opinion there's no extra > confidence in UTF-8 representability that CharSequence or even String gives us > compared to what CharBuffer does. On the other hand, compared to any other > implementation of CharSequence or String, CharBuffer is the most > charset-friendly thing we have: CharsetEncoder/CharsetDecoder speaks in > CharBuffers. > > Sorry, but I believe I haven't proposed to rely on JDK built-in replacement > characters. Moreover, being able to tell the decoder/encoder to throw exceptions > (e.g. UnmappableCharacterException) on incorrect input was one of the main > reasons to use CharsetEncoder/Decoder. And not, say, > String.getBytes(StandardCharsets.UTF_8). > > Thanks. > Hi, Just to clear things... The onText(..., CharBuffer cb, ...) call-back method receives a CharBuffer with content that is already UTF-8 decoded from wire message bytes, right? If it was different, it would not be right! So decoding is performed by WebSocket implementation, not by user and therefore can be performed per RFC6455 spec. CharBuffer, CharSequence, String - those object all represent characters and their API has nothing to do with UTF-8 or any other encoding. Regards, Peter From peter.levart at gmail.com Tue Oct 20 09:07:10 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 20 Oct 2015 11:07:10 +0200 Subject: WebSocket client API In-Reply-To: <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> Message-ID: <5626043E.9060600@gmail.com> Hi, Another thought about [Char|Byte]Buffer recycling. If the onXXX call-backs returned a CF so that they could asynchronously signal when they are done with consumption, the signature of the method could be: CompletionStage onText(..., CharBuffer cb, ...) Implementor of the method (the user) could do the following things: - return null or CompletableFuture.completedStage(null) to signal that the method already performed the consumption synchronously, but that it retained the CharBuffer, so WebSocket should not recycle it, but must consider it lost. - return CompletableFuture.completedStage(cb) to signal that the method already performed the consumption synchronously and the result of completed CompletionStage is the buffer that can be returned into the internal pool of WebSocket buffers. - return a CompletionStage that is yet to be completed asynchronously with the 'cb' or null as the result. The first case returns the buffer to the pool, the 2nd signals to WebSocket that the buffer is lost. This way, buffer recycling is in the domain of WebSocket implementation - not the user of WebSocket. The return type of onXXX methods should be CompletionStage not CompletableFuture. CompletionStage is not a Future and does not have methods that allow canceling of the underlying computation. All that WebSocket needs is attaching a completion that recycles the buffer, like: CharBuffer message = ... get buffer from pool or create new one ... ... fill message with data ... CompletionStage cs = listener.onText(..., message, ...); if (cs != null) { cs.thenAccept(cb -> { if (cb != null) { .... return cb to buffer pool ... } }); } What do you think? Regards, Peter On 10/13/2015 11:40 AM, Pavel Rappo wrote: > Hi Simone, > >> On 8 Oct 2015, at 20:51, Simone Bordet wrote: >> >> The *API* should provide a callback to notify when the ByteBuffer has >> been consumed. > Here is a proposed mechanism for managing buffers used by Listener. > > 1. WebSocket.Builder gets 2 new methods (may not be an actual javadoc): > > /** > * Specifies a function that provides {@code ByteBuffer}s for {@code > * WebSocket} to receive Binary, Ping and Pong messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * bytes should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder byteBuffersForListener(IntFunction provider); > > /** > * Specifies a function that provides {@code CharBuffer}s for {@code > * WebSocket} to receive Text and Close messages' payload to. > * > *

The function is called by {@code WebSocket} with a number of > * chars should remain in the required buffer. This serves as a hint > * from the implementation to a user. > * > * @param provider the providing function > * @return this builder > */ > public Builder charBuffersForListener(IntFunction provider); > > 2. If a user wants to use their own strategy of allocation/reuse they are fully > in charge of this. For example: > > IntFunction provider = (r) -> { > CharBuffer charBuffer = pool.getWithRemaining(r); > if (charBuffer == null) > charBuffer = CharBuffer.allocate(r); > return charBuffer; > }; > > ... > builder.charBuffersForListener(provider) ... .buildAsync(); > ... > > Later in the listener far, far away: > > @Override > public void onText(CharBuffer payload, boolean isLast) { > // ... > ws.sendText(payload, isLast).thenRun(() -> pool.recycle(payload)); > } > > Since the user constructs both the listener and the provider, they surely may now > of each other, so the 'pool' can be easily captured by the listener. > > 3. On the other hand we could specify a set of predefined providers, and default > behaviour like: > > * one off provider: constructs buffers on demand for one time use > * reusing provider: always returns a buffer to the implementation at the > end of the onXXX invocation > > In both cases above the user doesn't have to know about some additional > recycle-handlers. Hence there's no need for onXXX methods to change their > signatures to accommodate for it. > > What would you think about it? > > -Pavel > From simone.bordet at gmail.com Tue Oct 20 10:49:43 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Tue, 20 Oct 2015 12:49:43 +0200 Subject: WebSocket client API In-Reply-To: <5626043E.9060600@gmail.com> References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> <5626043E.9060600@gmail.com> Message-ID: Hi, On Tue, Oct 20, 2015 at 11:07 AM, Peter Levart wrote: > Hi, > > Another thought about [Char|Byte]Buffer recycling. If the onXXX call-backs > returned a CF so that they could asynchronously signal when they are done > with consumption, the signature of the method could be: > > CompletionStage onText(..., CharBuffer cb, ...) > > Implementor of the method (the user) could do the following things: > > - return null or CompletableFuture.completedStage(null) to signal that the > method already performed the consumption synchronously, but that it retained > the CharBuffer, so WebSocket should not recycle it, but must consider it > lost. > - return CompletableFuture.completedStage(cb) to signal that the method > already performed the consumption synchronously and the result of completed > CompletionStage is the buffer that can be returned into the internal pool of > WebSocket buffers. > - return a CompletionStage that is yet to be completed asynchronously with > the 'cb' or null as the result. The first case returns the buffer to the > pool, the 2nd signals to WebSocket that the buffer is lost. > > This way, buffer recycling is in the domain of WebSocket implementation - > not the user of WebSocket. > > The return type of onXXX methods should be CompletionStage not > CompletableFuture. CompletionStage is not a Future and does not have methods > that allow canceling of the underlying computation. All that WebSocket needs > is attaching a completion that recycles the buffer, like: > > CharBuffer message = ... get buffer from pool or create new one ... > > ... fill message with data ... > > CompletionStage cs = listener.onText(..., message, ...); > > if (cs != null) { > cs.thenAccept(cb -> { > if (cb != null) { > .... return cb to buffer pool ... > } > }); > } > > What do you think? The ability to return null to be completely equivalent to returning CompletableFuture.completedFuture(cb) would cover a common case (synchronous consumption) without incurring in forced allocation. I am not sure how common is the case to synchronously consume the buffer, but then not returning it. Perhaps your cases could be reduced to: * return null or CompletableFuture.completedFuture(cb) to indicate full recycle of the buffer. * return CompletableFuture.completedFuture(null) to indicate that the buffer is lost. This would reduce the allocation cost to just the fully async consumption case. With your proposal the typical async proxy case would then write from: onText(WebSocket ws, CharBuffer cb, boolean isLast) { return ws.sendText(cb, isLast).thenAccept(_ -> ws.request(1)); } to: onText(WebSocket ws, CharBuffer cb, boolean isLast) { return ws.sendText(cb, isLast).thenApply(_ -> { ws.request(1); return cb }); } Just slightly more complex. However, would it not be a potential attack vector ? Applications would be able to "inject" a buffer into the implementation buffer pool, and be able to peek/modify bytes from other connections. It should be: if (cs != null) { cs.thenAccept(cb -> { if (cb != null) { .... return *message* to buffer pool ... } }); } At that point, the return value of onText() could well be a CF like it already is, no ? The implementation would just need to test for null, but then never actually use the returned object wrapped by the CF. What do you think ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From peter.levart at gmail.com Tue Oct 20 12:01:37 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 20 Oct 2015 14:01:37 +0200 Subject: WebSocket client API In-Reply-To: References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> <5626043E.9060600@gmail.com> Message-ID: <56262D21.2010209@gmail.com> On 10/20/2015 12:49 PM, Simone Bordet wrote: > Hi, > > On Tue, Oct 20, 2015 at 11:07 AM, Peter Levart wrote: >> Hi, >> >> Another thought about [Char|Byte]Buffer recycling. If the onXXX call-backs >> returned a CF so that they could asynchronously signal when they are done >> with consumption, the signature of the method could be: >> >> CompletionStage onText(..., CharBuffer cb, ...) >> >> Implementor of the method (the user) could do the following things: >> >> - return null or CompletableFuture.completedStage(null) to signal that the >> method already performed the consumption synchronously, but that it retained >> the CharBuffer, so WebSocket should not recycle it, but must consider it >> lost. >> - return CompletableFuture.completedStage(cb) to signal that the method >> already performed the consumption synchronously and the result of completed >> CompletionStage is the buffer that can be returned into the internal pool of >> WebSocket buffers. >> - return a CompletionStage that is yet to be completed asynchronously with >> the 'cb' or null as the result. The first case returns the buffer to the >> pool, the 2nd signals to WebSocket that the buffer is lost. >> >> This way, buffer recycling is in the domain of WebSocket implementation - >> not the user of WebSocket. >> >> The return type of onXXX methods should be CompletionStage not >> CompletableFuture. CompletionStage is not a Future and does not have methods >> that allow canceling of the underlying computation. All that WebSocket needs >> is attaching a completion that recycles the buffer, like: >> >> CharBuffer message = ... get buffer from pool or create new one ... >> >> ... fill message with data ... >> >> CompletionStage cs = listener.onText(..., message, ...); >> >> if (cs != null) { >> cs.thenAccept(cb -> { >> if (cb != null) { >> .... return cb to buffer pool ... >> } >> }); >> } >> >> What do you think? > The ability to return null to be completely equivalent to returning > CompletableFuture.completedFuture(cb) would cover a common case > (synchronous consumption) without incurring in forced allocation. > I am not sure how common is the case to synchronously consume the > buffer, but then not returning it. > Perhaps your cases could be reduced to: > > * return null or CompletableFuture.completedFuture(cb) to indicate > full recycle of the buffer. > * return CompletableFuture.completedFuture(null) to indicate that the > buffer is lost. > > This would reduce the allocation cost to just the fully async consumption case. > > With your proposal the typical async proxy case would then write from: > > onText(WebSocket ws, CharBuffer cb, boolean isLast) > { > return ws.sendText(cb, isLast).thenAccept(_ -> ws.request(1)); > } > > to: > > onText(WebSocket ws, CharBuffer cb, boolean isLast) > { > return ws.sendText(cb, isLast).thenApply(_ -> { ws.request(1); return cb }); > } > > Just slightly more complex. > > However, would it not be a potential attack vector ? > Applications would be able to "inject" a buffer into the > implementation buffer pool, and be able to peek/modify bytes from > other connections. > It should be: > > if (cs != null) { > cs.thenAccept(cb -> { > if (cb != null) { > .... return *message* to buffer pool ... > } > }); > } > > At that point, the return value of onText() could well be a CF like > it already is, no ? > The implementation would just need to test for null, but then never > actually use the returned object wrapped by the CF. > > What do you think ? > You're right about potential attack vector. Then perhaps a CompletionStage is more appropriate return signature. But not more safe. Implementation could lie and return non-null (or true) indicating that buffer can be re-used, but in fact retaining it and peeking at other connection's data... To be safe, WebSocket would have to maintain separate pool of buffers for each connection. The pool could be shared among connections if it was given to WebSocket.Builder by the user. This could be implicit if API was designed with another level of indirection: interface WebSocket { static Factory newFactory() { // this could also be used to look-up the implementation with ServiceLoader } abstract class Factory { public abstract Builder newBuilder(); } abstract class Builder { ... } } The buffer pool would be on the Factory level. All sockets built by Builders constructed from the same Factory instance would share the same pool. Regards, Peter From joakim.erdfelt at gmail.com Tue Oct 20 12:50:03 2015 From: joakim.erdfelt at gmail.com (Joakim Erdfelt) Date: Tue, 20 Oct 2015 05:50:03 -0700 Subject: WebSocket client API In-Reply-To: <5625FDB3.3080001@gmail.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> Message-ID: You know that CharBuffer doesn't actually do UTF8, right? It's just a ByteBuffer split into equal 2 byte segments. CharBuffer is a way to obtain char (the 2 byte number, not the character) from the ByteBuffer or String you created it from. CharBuffer is functionally no different than ShortBuffer. On Tue, Oct 20, 2015 at 1:39 AM, Peter Levart wrote: > > > On 10/18/2015 12:08 AM, Pavel Rappo wrote: > >> Hi Joakim, >> >> On 17 Oct 2015, at 22:42, Joakim Erdfelt >>> wrote: >>> >>> You are required, per the RFC6455 spec, to validate incoming and >>> outgoing TEXT messages are valid UTF8. >>> (also Handshake and Close Reason Messages) >>> >>> http://tools.ietf.org/html/rfc6455#section-8.1 >>> >>> Relying on the JVM built-in replacement character behavior for invalid >>> UTF8 sequences will cause many bugs. >>> If you rely on the CharsetEncoder and CharBuffer you'll wind up with >>> situations where you are changing the data. >>> >>> You need to rely on an implementation that does not use replacement >>> characters and throws exceptions on bad Write, >>> and on bad received TEXT messages you MUST close the connection with a >>> 1007 error code. >>> >> The only thing I was trying to say is that in my opinion there's no extra >> confidence in UTF-8 representability that CharSequence or even String >> gives us >> compared to what CharBuffer does. On the other hand, compared to any other >> implementation of CharSequence or String, CharBuffer is the most >> charset-friendly thing we have: CharsetEncoder/CharsetDecoder speaks in >> CharBuffers. >> >> Sorry, but I believe I haven't proposed to rely on JDK built-in >> replacement >> characters. Moreover, being able to tell the decoder/encoder to throw >> exceptions >> (e.g. UnmappableCharacterException) on incorrect input was one of the main >> reasons to use CharsetEncoder/Decoder. And not, say, >> String.getBytes(StandardCharsets.UTF_8). >> >> Thanks. >> >> > Hi, > > Just to clear things... The onText(..., CharBuffer cb, ...) call-back > method receives a CharBuffer with content that is already UTF-8 decoded > from wire message bytes, right? If it was different, it would not be right! > So decoding is performed by WebSocket implementation, not by user and > therefore can be performed per RFC6455 spec. CharBuffer, CharSequence, > String - those object all represent characters and their API has nothing to > do with UTF-8 or any other encoding. > > Regards, Peter > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Tue Oct 20 12:57:18 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 20 Oct 2015 13:57:18 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> Message-ID: <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Hi Joakim, > On 20 Oct 2015, at 13:50, Joakim Erdfelt wrote: > > You know that CharBuffer doesn't actually do UTF8, right? > It's just a ByteBuffer split into equal 2 byte segments. > CharBuffer is a way to obtain char (the 2 byte number, not the character) from the ByteBuffer or String you created it from. > CharBuffer is functionally no different than ShortBuffer. Yes, I'm fully aware of the difference between java.nio.ByteBuffer#asCharBuffer and java.nio.charset.CharsetDecoder#decode(java.nio.ByteBuffer). What's your point? From simone.bordet at gmail.com Tue Oct 20 13:32:49 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Tue, 20 Oct 2015 15:32:49 +0200 Subject: WebSocket client API In-Reply-To: <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Message-ID: Hi, On Tue, Oct 20, 2015 at 2:57 PM, Pavel Rappo wrote: > Hi Joakim, > >> On 20 Oct 2015, at 13:50, Joakim Erdfelt wrote: >> >> You know that CharBuffer doesn't actually do UTF8, right? >> It's just a ByteBuffer split into equal 2 byte segments. >> CharBuffer is a way to obtain char (the 2 byte number, not the character) from the ByteBuffer or String you created it from. >> CharBuffer is functionally no different than ShortBuffer. > > Yes, I'm fully aware of the difference between java.nio.ByteBuffer#asCharBuffer > and java.nio.charset.CharsetDecoder#decode(java.nio.ByteBuffer). > > What's your point? So why choosing CharBuffer ? The only thing you can do with it is to copy, one more time and with no need to, the chars it contains before using them. Providing a CharSequence, instead, leaves the implementation much more room and has the potential to avoid unnecessary copies. The implementation could synthesize the bytes into a String, and pass that to the application without further copies (with some JDK help). Point being, if you use CharSequence, you will have much more freedom in the implementation, and you will not painting yourself in a corner with CharBuffer. Today you can use charset encoders, tomorrow you may use something more optimized that does not need CharBuffer. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From joakim.erdfelt at gmail.com Tue Oct 20 13:37:59 2015 From: joakim.erdfelt at gmail.com (Joakim Erdfelt) Date: Tue, 20 Oct 2015 06:37:59 -0700 Subject: WebSocket client API In-Reply-To: <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Message-ID: The CharBuffer requirement *was* baffling. But we *think* we understand what you are trying to do. Here's a split UTF8 scenario (just whipped up) https://gist.github.com/joakime/e34b727a6989ca7cef94 So the JVM implementation side will take the raw bytes (presumably as a ByteBuffer), and when the entire message is fully received it will convert it to a CharBuffer using the CharsetDecoder for UTF8 with REPORT logic to capture bad UTF8 sequences. Some concerns about this approach. 1. You can't fast-fail a large and fragmented TEXT message if the problematic UTF8 sequence occurs early (this is a spec test in the autobahn testsuite btw) 2. You can't use CharBuffer with partial TEXT message handling, as UTF8 sequences that are split across Frames will trigger the REPORT processing. (see gist/example above for this scenario) (also a spec test in the autobahn testsuite) 3. For each TEXT message, there's 2 data copies (ByteBuffer -> HeapCharBuffer -> String) for it to be practical to use in many 3rd party libs (eg JSON parsing). For large messages, this can get expensive. On Tue, Oct 20, 2015 at 5:57 AM, Pavel Rappo wrote: > Hi Joakim, > > > On 20 Oct 2015, at 13:50, Joakim Erdfelt > wrote: > > > > You know that CharBuffer doesn't actually do UTF8, right? > > It's just a ByteBuffer split into equal 2 byte segments. > > CharBuffer is a way to obtain char (the 2 byte number, not the > character) from the ByteBuffer or String you created it from. > > CharBuffer is functionally no different than ShortBuffer. > > Yes, I'm fully aware of the difference between > java.nio.ByteBuffer#asCharBuffer > and java.nio.charset.CharsetDecoder#decode(java.nio.ByteBuffer). > > What's your point? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Tue Oct 20 22:06:21 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 20 Oct 2015 23:06:21 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Message-ID: Hi Joakim, > On 20 Oct 2015, at 14:37, Joakim Erdfelt wrote: > > But we *think* we understand what you are trying to do. > > Here's a split UTF8 scenario (just whipped up) > https://gist.github.com/joakime/e34b727a6989ca7cef94 > > So the JVM implementation side will take the raw bytes (presumably as a ByteBuffer), and when the entire message is fully received it will convert it to a CharBuffer using the CharsetDecoder for UTF8 with REPORT logic to capture bad UTF8 sequences. > > Some concerns about this approach. > > 1. You can't fast-fail a large and fragmented TEXT message if the problematic UTF8 sequence occurs early (this is a spec test in the autobahn testsuite btw) > 2. You can't use CharBuffer with partial TEXT message handling, as UTF8 sequences that are split across Frames will trigger the REPORT processing. (see gist/example above for this scenario) (also a spec test in the autobahn testsuite) > 3. For each TEXT message, there's 2 data copies (ByteBuffer -> HeapCharBuffer -> String) for it to be practical to use in many 3rd party libs (eg JSON parsing). For large messages, this can get expensive. Joakim, If I tell you that 1. CharsetDecoder is a stateful object which is capable of incremental decoding from any given ByteBuffer into any given CharBuffer. Have a look at CharsetDecoder#decode(java.nio.ByteBuffer, java.nio.CharBuffer, boolean) method. 2. String#String(byte[], int, int, java.nio.charset.Charset) uses the same machinery underneath. The difference (among other) is, the this constructor creates Buffer wrappers. that we can preallocate a bunch of CharBuffers and reuse them (speaking of performance). 3. Probably the quickest way bytes from a Channel can end up being UTF-8 decoded chars is through the ByteBuffer, CharsetEncoder and CharBuffer (If you know better, please tell me). 4. So if, after all, one needs a String the pipeline: Channel --> ByteBuffer --> CharsetDecoder --> CharBuffer --> String would be the quickest way to it. 5. Not everyone, probably, needs a String. For some users a CharSequence would do. Consider appending it (or its subsequence) to java.lang.Appendable, or just processing Stream from cs.chars(), etc. would it change any of your concerns? If not, please try to explain these problems again, even in more details. From joakim.erdfelt at gmail.com Tue Oct 20 22:16:55 2015 From: joakim.erdfelt at gmail.com (Joakim Erdfelt) Date: Tue, 20 Oct 2015 15:16:55 -0700 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Message-ID: I'm done with the UTF8 topic, you seem to have it in hand. If you feel the existing facilities can handle it, go for it. Just don't forget to test your impl on Autobahn Testsuite. On Tue, Oct 20, 2015 at 3:06 PM, Pavel Rappo wrote: > Hi Joakim, > > > On 20 Oct 2015, at 14:37, Joakim Erdfelt > wrote: > > > > But we *think* we understand what you are trying to do. > > > > Here's a split UTF8 scenario (just whipped up) > > https://gist.github.com/joakime/e34b727a6989ca7cef94 > > > > So the JVM implementation side will take the raw bytes (presumably as a > ByteBuffer), and when the entire message is fully received it will convert > it to a CharBuffer using the CharsetDecoder for UTF8 with REPORT logic to > capture bad UTF8 sequences. > > > > Some concerns about this approach. > > > > 1. You can't fast-fail a large and fragmented TEXT message if the > problematic UTF8 sequence occurs early (this is a spec test in the autobahn > testsuite btw) > > 2. You can't use CharBuffer with partial TEXT message handling, as UTF8 > sequences that are split across Frames will trigger the REPORT processing. > (see gist/example above for this scenario) (also a spec test in the > autobahn testsuite) > > 3. For each TEXT message, there's 2 data copies (ByteBuffer -> > HeapCharBuffer -> String) for it to be practical to use in many 3rd party > libs (eg JSON parsing). For large messages, this can get expensive. > > Joakim, > > If I tell you that > > 1. CharsetDecoder is a stateful object which is capable of incremental > decoding from any given ByteBuffer into any given CharBuffer. Have a > look at > CharsetDecoder#decode(java.nio.ByteBuffer, java.nio.CharBuffer, > boolean) > method. > > 2. String#String(byte[], int, int, java.nio.charset.Charset) uses the > same > machinery underneath. The difference (among other) is, the this > constructor > creates Buffer wrappers. that we can preallocate a bunch of > CharBuffers and > reuse them (speaking of performance). > > 3. Probably the quickest way bytes from a Channel can end up being > UTF-8 > decoded chars is through the ByteBuffer, CharsetEncoder and CharBuffer > (If > you know better, please tell me). > > 4. So if, after all, one needs a String the pipeline: > > Channel --> ByteBuffer --> CharsetDecoder --> CharBuffer --> String > > would be the quickest way to it. > > 5. Not everyone, probably, needs a String. For some users a > CharSequence > would do. Consider appending it (or its subsequence) to > java.lang.Appendable, or just processing Stream from cs.chars(), etc. > > would it change any of your concerns? If not, please try to explain these > problems again, even in more details. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Wed Oct 21 09:10:55 2015 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 21 Oct 2015 10:10:55 +0100 Subject: WebSocket client API In-Reply-To: References: Message-ID: <5627569F.4010309@oracle.com> Hi, I think this is a reasonable observation. We actually changed ProxySelector itself to provide (one) simple factory method for a fixed Proxy using an InetSocketAddress for all requests and that makes setting a simple proxy very easy. But, I agree it probably makes more sense for the default behavior to be to use the system-wide default. Thanks Michael On 19/10/15 07:42, Andrej Golovnin wrote: > Hi Pavel, > > the JavaDocs for the method > HttpClient.Builder#proxy(java.net.ProxySelector selector) say: > > Sets a ProxySelector for this client. If no selector is set, then no > proxies are used. If a null parameter is supplied then the system wide > default proxy selector is used. > > Really? For me it means that nearly every application which would like > to use the new HttpClient/WebSocket must call this method with null > value. Just think about Java WebStart applications and applets: you > never know if there is a proxy or not. > > IMHO the JavaDocs for this method should look like this: > > Sets a ProxySelector for this client. If no proxy selector is set, > then the system wide default proxy selector is used. > > @throws IllegalArgumentException if selector is null. > > I'm sure the most developers out there should be able to provide their > own implementation of the java.net.ProxySelector class which just > returns Collections.singletonList(Proxy.NO_PROXY) in the method > ProxySelector#select(URI uri) if they need direct access to a server. > And if you would like to simplify the life of the developers which > need a direct access to a server, then you could add a new method to > the builder: > > HttpClient.Builder#noProxy() > > I think it is better than to add a special meaning to the null value. > > And I think that the phrase with "If no proxy selector is set, then > the system wide default proxy selector is used." should be a part of > the JavaDocs of the class HttpClient.Builder and the method > HttpClient.getDefault() to make clear the behaviour of the > created/returned HttpClient. > > Best regards, > Andrej Golovnin From ivan.gerasimov at oracle.com Wed Oct 21 10:06:01 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 21 Oct 2015 13:06:01 +0300 Subject: RFR 8139373 : [TEST_BUG] java/net/MulticastSocket/MultiDead.java failed with timeout Message-ID: <56276389.6040405@oracle.com> Hello! A few failures of the recently added regtest were observed. The failures seem to be due to slow machines. The suggested fix is to 1) increase the timeout, 2) take into account the timeout factor from the jtreg's settings, 3) measure the time of individual cycle of the loop, and give up, if it appeared to be too slow. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8139373 WEBREV: http://cr.openjdk.java.net/~igerasim/8139373/00/webrev/ Would you help review this fix? Sincerely yours, Ivan From peter.levart at gmail.com Wed Oct 21 10:29:44 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 21 Oct 2015 12:29:44 +0200 Subject: WebSocket client API In-Reply-To: References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> <5626043E.9060600@gmail.com> Message-ID: <56276918.3070206@gmail.com> On 10/20/2015 12:49 PM, Simone Bordet wrote: > if (cs != null) { > cs.thenAccept(cb -> { > if (cb != null) { > .... return*message* to buffer pool ... > } > }); > } > > At that point, the return value of onText() could well be a CF like > it already is, no ? > The implementation would just need to test for null, but then never > actually use the returned object wrapped by the CF. > > What do you think ? There is one advantage to having CompletionStage return value instead of CompletionStage or CompletionStage. In above example, the lambda has to capture the *message* and retain it until the call-back happens at which point it may be that *message* has to be ignored (not recycled). In such case the *message* may have been retained by lambda for more time than necessary. If CompletionStage is used instead, the buffer or null is passed to lambda so buffer's retention time is always optimal. Regards, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavel.rappo at oracle.com Wed Oct 21 10:52:55 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Wed, 21 Oct 2015 11:52:55 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: > On 19 Oct 2015, at 22:37, Simone Bordet wrote: > > Now I am confused :) > > Are you saying that if you do: > > builder.headers("foo", "bar", "baz").headers("one", "two", "three") > > then "foo", "bar" and "baz" are removed ? In the current version it is exactly like you've said. And we're gonna change it! The only thing is, it should've been: builder.headers("name1", "value1", "name2", "value2").headers("name3", "value3") instead of what you've written. In this case custom headers would've been ("name3", "value3") as `.headers("name3", "value3")` was the last call and therefore it would've overwritten preceding `.headers("name1", "value1", "name2", "value2")` >> Easier? In some cases, probably. On the other WebSocket.request(long) is an >> internal part of the API that only the Listener should talk to (the same is with >> Flow.Subscriber and Flow.Subscription). > > In proxies and WebSocket applications in general, it's the opposite: > these applications want to call request(long) after they have written > data, not after they read it, to produce backpressure. ... >> But I'm against of merging flowController with WebSocket. > > Yet, you did not bring any technical reasons for the split, and > "separation of concerns" is just not enough because it can mean > anything (and it may even be wrong). I see what you mean, but my passage above does not contradict this. What I wanted to say (but it seems like I've failed the attempt) is that Listener is pretty much Flow.Subscription. And as such is susceptible to Rule 3.1 [1]. There are some subtle differences though between general Flow case and out WebSocket case! The most obvious one is that WebSocket has a single Subscriber -- WebSocket.Listener. > If you write a chat application with these API, you will see that > having WebSocket.request() simplifies the code by *a lot*. > > I have done that, and I am reporting my impressions. Do you still have any examples? I would love to see them. ------------------------------------------------------------------------------- [1] https://github.com/reactive-streams/reactive-streams-jvm/ From simone.bordet at gmail.com Wed Oct 21 11:18:26 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 21 Oct 2015 13:18:26 +0200 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: Hi, On Wed, Oct 21, 2015 at 12:52 PM, Pavel Rappo wrote: >>> But I'm against of merging flowController with WebSocket. >> >> Yet, you did not bring any technical reasons for the split, and >> "separation of concerns" is just not enough because it can mean >> anything (and it may even be wrong). > > I see what you mean, but my passage above does not contradict this. What I > wanted to say (but it seems like I've failed the attempt) is that Listener is > pretty much Flow.Subscription. I don't see where you can think that Listener is anywhere near Flow.Subscription at all. Different set of methods, different purposes, different abstractions. I really hope you meant that Listener is like Flow.Subscriber. And you still have not brought any technical reason for that decision. Speaking of analogies, it's pretty common to implement Publisher *and* Subscription in the same class when you can only have one listener, which is exactly the WebSocket case. Now, Publisher is the "write" part of ReactiveStreams, and Subscription is the flow control *and* cancelling part. If you put back the flow control method into WebSocket, you get exactly that: the write part along with the flow control and cancelling part. In ReactiveStream Publisher and Subscription are divided to support multiple Subscribers. In WebSocket there is no need to, hence, no need to split them. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From pavel.rappo at oracle.com Wed Oct 21 11:22:18 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Wed, 21 Oct 2015 12:22:18 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <5625FDB3.3080001@gmail.com> <57EA7CF1-F850-4B06-8743-6A1CC18C157B@oracle.com> Message-ID: > On 20 Oct 2015, at 14:32, Simone Bordet wrote: > > So why choosing CharBuffer ? Because in general when an API provides a callback method `c` c( x) an application has to implement, it should better be the case that the is the most specific type possible (with possibly more methods and stricter guarantees) and the is the least specific type possible (weaker guarantees). Note: it's obviously the opposite with an API's method `m` the application has to call: m( x) is the most specific possible, the is the least specific possible. > The only thing you can do with it is to copy, one more time and with > no need to, the chars it contains before using them. Not true. One could use it as Appendable or CharSequence. Or, which is an extra win, encode it into a ByteBuffer (probably with different encoding) and write the resulting thing to a Channel. > Providing a CharSequence, instead, leaves the implementation much more > room and has the potential to avoid unnecessary copies. "More room"? Maybe. Unnecessary copies -- highly doubt it. What would an application (an API's user) need to do to write this CharSequence to a Channel? CompletableFuture onText(CharSequence text, boolean isLast) { ... if (text instanceof CharBuffer) { cb = (CharBuffer) text; } else { cb = CharBuffer.wrap(text); } encoder.encode(cb, bb, isLast); channel.write(bb); ... } ------------ So the implementation is very honest here. It returns the same object it's just used for a decoding. Whether a code snippet above worth weakening the type passed (from CharBuffer to CharSequence) is something to think about. As for me, I need to think about it. > The implementation could synthesize the bytes into a String, and pass > that to the application without further copies (with some JDK help). Any specific examples? From pavel.rappo at oracle.com Wed Oct 21 11:24:35 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Wed, 21 Oct 2015 12:24:35 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: > On 21 Oct 2015, at 12:18, Simone Bordet wrote: > > I really hope you meant that Listener is like Flow.Subscriber. It's obviously a typo. Thanks! Listener ~ Subscriber WebSocket ~ Publisher FlowController ~ Subscription From pavel.rappo at oracle.com Wed Oct 21 11:29:43 2015 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Wed, 21 Oct 2015 12:29:43 +0100 Subject: WebSocket client API In-Reply-To: References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> Message-ID: <2B0B522B-8DDF-420D-9B76-2557DF8617C8@oracle.com> > On 21 Oct 2015, at 12:18, Simone Bordet wrote: > > Speaking of analogies, it's pretty common to implement Publisher *and* > Subscription in the same class when you can only have one listener, > which is exactly the WebSocket case. Well, maybe, I don't know. But implementing interfaces A and B with the same class C doesn't magically bring B's methods into A, does it? From simone.bordet at gmail.com Wed Oct 21 11:42:52 2015 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 21 Oct 2015 13:42:52 +0200 Subject: WebSocket client API In-Reply-To: <2B0B522B-8DDF-420D-9B76-2557DF8617C8@oracle.com> References: <6FA72C53-E818-4FC0-9521-179C1B0421B5@oracle.com> <2B0B522B-8DDF-420D-9B76-2557DF8617C8@oracle.com> Message-ID: Hi, On Wed, Oct 21, 2015 at 1:29 PM, Pavel Rappo wrote: > Well, maybe, I don't know. But implementing interfaces A and B with the same > class C doesn't magically bring B's methods into A, does it? It does if they were not magically split without reason. My point is that you proposed a while back a handy WebSocket interface with a request(long) method. Then that method was split for no reason out of WebSocket, which also became a class for no reason. I am reporting that the previous version was better for application writers. If you're trying to be as close as possible to ReactiveStream, then just use the Flow API. LongConsumer is of course not Flow.Subscription. Half mimicking the Flow API does not bring any good. And with this I really hope you are not going to extract out of WebSocket also the cancelling/closing part to separate more concerns ;) -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From chris.hegarty at oracle.com Wed Oct 21 12:21:02 2015 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 21 Oct 2015 13:21:02 +0100 Subject: RFR 8139373 : [TEST_BUG] java/net/MulticastSocket/MultiDead.java failed with timeout In-Reply-To: <56276389.6040405@oracle.com> References: <56276389.6040405@oracle.com> Message-ID: <5627832E.4010608@oracle.com> Looks fine Ivan, Just a typo on: 48 Utils.adjustTimeout(CHILDREN_COUNT * CHILD_TIMEOT * 2); -Chris. On 21/10/15 11:06, Ivan Gerasimov wrote: > Hello! > > A few failures of the recently added regtest were observed. > The failures seem to be due to slow machines. > > The suggested fix is to > 1) increase the timeout, > 2) take into account the timeout factor from the jtreg's settings, > 3) measure the time of individual cycle of the loop, and give up, if it > appeared to be too slow. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8139373 > WEBREV: http://cr.openjdk.java.net/~igerasim/8139373/00/webrev/ > > Would you help review this fix? > > Sincerely yours, > Ivan > From peter.levart at gmail.com Wed Oct 21 13:13:15 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 21 Oct 2015 15:13:15 +0200 Subject: WebSocket client API In-Reply-To: References: <5C8C5074-321E-42BF-8F8D-A85CA3892B96@oracle.com> <0830FDB0-534C-435C-88D7-FEB29F16413B@oracle.com> <5626043E.9060600@gmail.com> Message-ID: <56278F6B.9010407@gmail.com> On 10/20/2015 12:49 PM, Simone Bordet wrote: >> >CharBuffer message = ... get buffer from pool or create new one ... >> > >> >... fill message with data ... >> > >> >CompletionStage cs = listener.onText(..., message, ...); >> > >> >if (cs != null) { >> > cs.thenAccept(cb -> { >> > if (cb != null) { >> > .... return cb to buffer pool ... >> > } >> > }); >> >} >> > >> >What do you think? > The ability to return null to be completely equivalent to returning > CompletableFuture.completedFuture(cb) would cover a common case > (synchronous consumption) without incurring in forced allocation. Correct, but from API standpoint this would be worse. The act of explicitly passing back the reference to CharBuffer is a conscious transfer of ownership. Passing null is not and could be a source of bugs. Regards, Peter From ivan.gerasimov at oracle.com Wed Oct 21 16:32:10 2015 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 21 Oct 2015 19:32:10 +0300 Subject: RFR 8139373 : [TEST_BUG] java/net/MulticastSocket/MultiDead.java failed with timeout In-Reply-To: <5627832E.4010608@oracle.com> References: <56276389.6040405@oracle.com> <5627832E.4010608@oracle.com> Message-ID: <5627BE0A.8020805@oracle.com> Thank you Chris for review and proofreading! :-) Sincerely yours, Ivan On 21.10.2015 15:21, Chris Hegarty wrote: > Looks fine Ivan, > > Just a typo on: > 48 Utils.adjustTimeout(CHILDREN_COUNT * CHILD_TIMEOT * 2); > > -Chris. > > On 21/10/15 11:06, Ivan Gerasimov wrote: >> Hello! >> >> A few failures of the recently added regtest were observed. >> The failures seem to be due to slow machines. >> >> The suggested fix is to >> 1) increase the timeout, >> 2) take into account the timeout factor from the jtreg's settings, >> 3) measure the time of individual cycle of the loop, and give up, if it >> appeared to be too slow. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8139373 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8139373/00/webrev/ >> >> Would you help review this fix? >> >> Sincerely yours, >> Ivan >> > From Roger.Riggs at Oracle.com Thu Oct 22 13:04:44 2015 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 22 Oct 2015 09:04:44 -0400 Subject: SO_REUSEPORT feature support in JDK 9 for socket communication In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2B647681C8@FMSMSX112.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B647681C8@FMSMSX112.amr.corp.intel.com> Message-ID: <5628DEEC.5080407@Oracle.com> Hi Sandhya, The folks on net-dev at openjdk.java.net will be interested too. Roger On 10/21/2015 9:08 PM, Viswanathan, Sandhya wrote: > > This is a proposal for adding SO_REUSEPORT support in JDK 9 for socket communication. The feature is supported since Linux Kernel 3.9 in OS. It is also supported in some of BSD flavors. It removes 1:1 assignment between listen socket and IP:PORT pair and enables multiple sockets listening to the same address. This improves the scalability and parallelism of network traffic handling. For more details, please refer to https://lwn.net/Articles/542629/ > > The proposal is to add the SO_REUSEPORT feature on similar lines as SO_REUSEADDR which was included in JDK7. > Add the following two API methods to java/net/ServerSocket.java and java/net/Socket.java: > > public void setReusePort(boolean on) > > throws SocketException > > Enable/disable the SO_REUSEPORT socket option. > > Enabling SO_REUSEPORT prior to binding the socket using bind(SocketAddress) allows the socket and all the sockets created after this socket being able to listen to > > same IP:PORT. Applications can use getReusePort() to determine the initial setting of SO_REUSEPORT. > > Parameters: > > on - whether to enable or disable the socket option > > Throws: > > SocketException - if an error occurs enabling or disabling the SO_RESUEPORT socket option, or the socket is closed. > > > > public boolean getReusePort() > > throws SocketException > > Tests if SO_REUSEPORT is enabled. > > Returns: > > A boolean indicating whether or not SO_REUSEPORT is enabled. > > Throws: > > SocketException - if there is an error in the underlying protocol, such as a TCP error. > > > Also add the SO_REUSEPORT to the SocketOptions in various files. > > > > We implemented a prototype on JDK 8 and see very good results in HDFS latency reduction and network connection scaling; about ~1.9x improvement in latency for OSU HiBD Benchmark (http://hibd.cse.ohio-state.edu/downloads/) on a small cluster of 1 Namenode and 2 DataNodes. > We plan to port it to JDK 9 sources and submit it for consideration. > Please let us know if there is interest and if you would like to sponsor this patch. > > > Many applications, especially Linux or BSD based webservers such as Apache httpd and Nginx are already supporting it now. Ruby and Python have it supported as well. Other Java applications such as Netty webserver have it support it via JNI function since JDK has not supported it yet. Significant throughput and latency improvement have been observed from various applications. > > > > Best Regards, > > Sandhya > > From Alan.Bateman at oracle.com Thu Oct 22 13:24:18 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 22 Oct 2015 14:24:18 +0100 Subject: SO_REUSEPORT feature support in JDK 9 for socket communication In-Reply-To: <5628DEEC.5080407@Oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B647681C8@FMSMSX112.amr.corp.intel.com> <5628DEEC.5080407@Oracle.com> Message-ID: <5628E382.3070509@oracle.com> On 22/10/2015 14:04, Roger Riggs wrote: > Hi Sandhya, > > The folks on net-dev at openjdk.java.net will be interested too. Yes, net-dev is the best list for this. One other thing to mention is the SocketOption interface and the setOption/getOption methods. This allows for platform or JDK-specific specific socket options, it also allows it to be implemented by the NIO SocketChannel and friends. -Alan From michael.x.mcmahon at oracle.com Thu Oct 22 13:33:10 2015 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 22 Oct 2015 14:33:10 +0100 Subject: SO_REUSEPORT feature support in JDK 9 for socket communication In-Reply-To: <5628E382.3070509@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B647681C8@FMSMSX112.amr.corp.intel.com> <5628DEEC.5080407@Oracle.com> <5628E382.3070509@oracle.com> Message-ID: <5628E596.5000203@oracle.com> On 22/10/15 14:24, Alan Bateman wrote: > > > On 22/10/2015 14:04, Roger Riggs wrote: >> Hi Sandhya, >> >> The folks on net-dev at openjdk.java.net will be interested too. > Yes, net-dev is the best list for this. > > One other thing to mention is the SocketOption interface and the > setOption/getOption methods. This allows for platform or JDK-specific > specific socket options, it also allows it to be implemented by the > NIO SocketChannel and friends. > > -Alan and there is the jdk.net API which extends this to the java.net socket types. I think a Java SE API would be fine if it is widely and consistently implemented across all the reference platforms and if it does not cause compatibility issues. If it only works on some platforms then maybe jdk.net could be the place for it. - Michael From mark.sheppard at oracle.com Thu Oct 22 15:05:47 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Thu, 22 Oct 2015 16:05:47 +0100 Subject: SO_REUSEPORT feature support in JDK 9 for socket communication In-Reply-To: <5628E596.5000203@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B647681C8@FMSMSX112.amr.corp.intel.com> <5628DEEC.5080407@Oracle.com> <5628E382.3070509@oracle.com> <5628E596.5000203@oracle.com> Message-ID: <5628FB4B.8090305@oracle.com> the following JBS item exists: https://bugs.openjdk.java.net/browse/JDK-6432031 search of windows documentation suggests that SO_REUSEPORT is still not an option man setsockopt on Solaris shows it as an option, but without precise description of semantics regards Mark On 22/10/2015 14:33, Michael McMahon wrote: > On 22/10/15 14:24, Alan Bateman wrote: >> >> >> On 22/10/2015 14:04, Roger Riggs wrote: >>> Hi Sandhya, >>> >>> The folks on net-dev at openjdk.java.net will be interested too. >> Yes, net-dev is the best list for this. >> >> One other thing to mention is the SocketOption interface and the >> setOption/getOption methods. This allows for platform or JDK-specific >> specific socket options, it also allows it to be implemented by the >> NIO SocketChannel and friends. >> >> -Alan > > and there is the jdk.net API which extends this to the java.net socket > types. > > I think a Java SE API would be fine if it is widely and consistently > implemented > across all the reference platforms and if it does not cause > compatibility issues. > > If it only works on some platforms then maybe jdk.net could be the > place for it. > > - Michael From mark.sheppard at oracle.com Sun Oct 25 23:32:09 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Sun, 25 Oct 2015 23:32:09 +0000 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor Message-ID: <562D6679.9090509@oracle.com> Hi, please oblige and review the following changes http://cr.openjdk.java.net/~msheppar/8134577/webrev/ which address the issue raised in https://bugs.openjdk.java.net/browse/JDK-8134577 the operative word has been "eliminate". As such, the interface and service descriptor sun.net.spi.nameservice.NameService sun.net.spi.nameservice.NameServiceDescriptor* *together with its implementation (sun.net.nameservice.dns.DNSNameService) has been remove from the JDK libraries. The immediate impact is seen in the JDK testing framework. To facilitate testing activity, and provide a replacement for the customized NameService implementations in the JDK tests, the default NameService has been extended to support the retrieval of host to IP address mappings from a file. The file path is specified with a system property " jdk.internal.hosts". Previously a nameservice provider was specified by setting the system property "sun.net.spi.nameservice.provider.", as per the documentation http://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html InetAddress now tests to determine if this property is set and will throw a ServiceConfigurationError indicating that this functionality is no longer supported. The choice of ServideConfigurationError may cause some debate, or disagreement. The rationale was that InternalError, is documented to relate to a JVM error, and javax.naming.NamingException has a context of DirContext. A possible alternative candidate could be javax.naming.ServiceUnavailableException. As such, the setting of the property "sun.net.spi.nameservice.provider." was used, previously, as a configuration parameter for the loading of a NamerService service provider, and as this is now (considered) an error, ServiceConfigurationError, seemed a best fit! These changes impacted a number of jdk security tests, also. The affected tetsts have been amended to adopt the changes, with the exception of test/sun/security/x509/URICertStore/ExtensionsWithLDAP.java which will require some rewrite. regards Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.smotrakov at oracle.com Mon Oct 26 00:24:07 2015 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Mon, 26 Oct 2015 03:24:07 +0300 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562D6679.9090509@oracle.com> References: <562D6679.9090509@oracle.com> Message-ID: <562D72A7.1040302@oracle.com> Hi Mark, I am not a reviewer, just have a couple of comments about InetAddress.java 1. It may be better to create an instance of Scanner in try-with-resource block to be sure that Scanner.close() method is called. 2. Lines 909-923: There are two similar "if" blocks in the loop. Looks like the first one is not necessary (I also see similar code in lookupAllHostAddr() method, maybe this code could be moved to a separate method). 3. extractHostAddr() and extractHost() methods: The methods assume that "hostEntry" contains at least one whitespace, and access first and second elements of "mapping " array. It may be better to check that length of "mapping" is more than one to avoid a possible ArrayIndexOutOfBoundsException. Looks like those methods may also be static. Artem On 10/26/2015 02:32 AM, Mark Sheppard wrote: > Hi, > please oblige and review the following changes > http://cr.openjdk.java.net/~msheppar/8134577/webrev/ > > which address the issue raised in > https://bugs.openjdk.java.net/browse/JDK-8134577 > > the operative word has been "eliminate". > As such, the interface and service descriptor > sun.net.spi.nameservice.NameService > sun.net.spi.nameservice.NameServiceDescriptor* > *together with its implementation (sun.net.nameservice.dns.DNSNameService) > has been remove from the JDK libraries. > > The immediate impact is seen in the JDK testing framework. > > To facilitate testing activity, and provide a replacement for the > customized NameService implementations in the > JDK tests, the default NameService has been extended to support > the retrieval of host to IP address mappings from a file. > The file path is specified with a system property " jdk.internal.hosts". > > Previously a nameservice provider was specified by setting the system > property > "sun.net.spi.nameservice.provider.", as per the documentation > http://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html > > InetAddress now tests to determine if this property is set and will > throw a ServiceConfigurationError > indicating that this functionality is no longer supported. The choice > of ServideConfigurationError may cause > some debate, or disagreement. The rationale was that InternalError, > is documented to relate to a JVM error, > and javax.naming.NamingException has a context of DirContext. > A possible alternative candidate could be > javax.naming.ServiceUnavailableException. > As such, the setting of the property > "sun.net.spi.nameservice.provider." was used, previously, as a > configuration > parameter for the loading of a NamerService service provider, and as > this is now (considered) an error, ServiceConfigurationError, > seemed a best fit! > > These changes impacted a number of jdk security tests, also. The > affected tetsts have been amended to adopt the > changes, with the exception of > test/sun/security/x509/URICertStore/ExtensionsWithLDAP.java > which will require some rewrite. > > regards > Mark > -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at oracle.com Mon Oct 26 01:57:23 2015 From: weijun.wang at oracle.com (Wang Weijun) Date: Mon, 26 Oct 2015 09:57:23 +0800 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562D72A7.1040302@oracle.com> References: <562D6679.9090509@oracle.com> <562D72A7.1040302@oracle.com> Message-ID: I see a lot of krb5 tests modified. Basically, the NameServiceDescriptor inside KDC.java maps everything to localhost except for one (I guess Artem invented the 2nd feature). Can we expand the grammar a little bit to support this? For example: not.existing.host UnknownHostException * 127.0.0.1 I see you've added all names the tests have touched in test/sun/security/krb5/auto/hosts, but I am not sure if we might add randomly generated hostnames in any test later. BTW, the standard /etc/hosts has lines like "127.0.0.1 localhost" which has a different order than yours. Shall we rename hosts to something else so that people does not get confused? Thanks Max >> http://cr.openjdk.java.net/~msheppar/8134577/webrev/ >> >> which address the issue raised in >> https://bugs.openjdk.java.net/browse/JDK-8134577 >> From mark.sheppard at oracle.com Tue Oct 27 12:22:15 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Tue, 27 Oct 2015 12:22:15 +0000 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562D72A7.1040302@oracle.com> References: <562D6679.9090509@oracle.com> <562D72A7.1040302@oracle.com> Message-ID: <562F6C77.6000805@oracle.com> Hi Artem, thanks for the feedback, I'll make the changes for 2 & 3. I'll look into rework of try-with-resources. regards Mark On 26/10/2015 00:24, Artem Smotrakov wrote: > Hi Mark, > > I am not a reviewer, just have a couple of comments about InetAddress.java > > 1. It may be better to create an instance of Scanner in > try-with-resource block to be sure that Scanner.close() method is called. > > 2. Lines 909-923: > > There are two similar "if" blocks in the loop. Looks like the first > one is not necessary (I also see similar code in lookupAllHostAddr() > method, maybe this code could be moved to a separate method). > > 3. extractHostAddr() and extractHost() methods: > > The methods assume that "hostEntry" contains at least one whitespace, > and access first and second elements of "mapping " array. It may be > better to check that length of "mapping" is more than one to avoid a > possible ArrayIndexOutOfBoundsException. > > Looks like those methods may also be static. > > Artem > > On 10/26/2015 02:32 AM, Mark Sheppard wrote: >> Hi, >> please oblige and review the following changes >> http://cr.openjdk.java.net/~msheppar/8134577/webrev/ >> >> which address the issue raised in >> https://bugs.openjdk.java.net/browse/JDK-8134577 >> >> the operative word has been "eliminate". >> As such, the interface and service descriptor >> sun.net.spi.nameservice.NameService >> sun.net.spi.nameservice.NameServiceDescriptor* >> *together with its implementation >> (sun.net.nameservice.dns.DNSNameService) >> has been remove from the JDK libraries. >> >> The immediate impact is seen in the JDK testing framework. >> >> To facilitate testing activity, and provide a replacement for the >> customized NameService implementations in the >> JDK tests, the default NameService has been extended to support >> the retrieval of host to IP address mappings from a file. >> The file path is specified with a system property " jdk.internal.hosts". >> >> Previously a nameservice provider was specified by setting the system >> property >> "sun.net.spi.nameservice.provider.", as per the documentation >> http://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html >> >> InetAddress now tests to determine if this property is set and will >> throw a ServiceConfigurationError >> indicating that this functionality is no longer supported. The choice >> of ServideConfigurationError may cause >> some debate, or disagreement. The rationale was that InternalError, >> is documented to relate to a JVM error, >> and javax.naming.NamingException has a context of DirContext. >> A possible alternative candidate could be >> javax.naming.ServiceUnavailableException. >> As such, the setting of the property >> "sun.net.spi.nameservice.provider." was used, previously, as a >> configuration >> parameter for the loading of a NamerService service provider, and as >> this is now (considered) an error, ServiceConfigurationError, >> seemed a best fit! >> >> These changes impacted a number of jdk security tests, also. The >> affected tetsts have been amended to adopt the >> changes, with the exception of >> test/sun/security/x509/URICertStore/ExtensionsWithLDAP.java >> which will require some rewrite. >> >> regards >> Mark >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Tue Oct 27 14:51:05 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 27 Oct 2015 09:51:05 -0500 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor Message-ID: <562F8F59.7040804@redhat.com> > Hi, > please oblige and review the following changes > http://cr.openjdk.java.net/~msheppar/8134577/webrev/ > > which address the issue raised in > https://bugs.openjdk.java.net/browse/JDK-8134577 > > the operative word has been "eliminate". > As such, the interface and service descriptor > sun.net.spi.nameservice.NameService > sun.net.spi.nameservice.NameServiceDescriptor* > *together with its implementation (sun.net.nameservice.dns.DNSNameService) > has been remove from the JDK libraries. This is unfortunate. There's really no other way to hook into name resolution. What's wrong with putting this stuff in a com.sun or org.openjdk module? Having modules doesn't *really* require that everything non-standard be hidden or killed off, just that it be separated from the standard modules in a way that makes some kind of sense. -- - DML From Alan.Bateman at oracle.com Tue Oct 27 15:05:41 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Oct 2015 15:05:41 +0000 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562D6679.9090509@oracle.com> References: <562D6679.9090509@oracle.com> Message-ID: <562F92C5.3060105@oracle.com> On 25/10/2015 23:32, Mark Sheppard wrote: > Hi, > please oblige and review the following changes > http://cr.openjdk.java.net/~msheppar/8134577/webrev/ > > which address the issue raised in > https://bugs.openjdk.java.net/browse/JDK-8134577 > > the operative word has been "eliminate". This has been a very troublesome mechanism so good to see this reduced down to a hosts file that we can we use for testing. Shouldn't the reading of sun.net.spi.nameservice.provider. be removed so that the throwing of ServiceConfigurationError can be removed too? Also just on the property name, I would assume it should be something like "jdk.net.hosts.file" rather than "jdk.internal.hosts". Do we have tests that are using the illegal_state_exception token? I didn't spot any in the webrev. Just wondering if this is the right exception for testing. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.sickelmann at gmx.de Tue Oct 27 04:24:38 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 27 Oct 2015 05:24:38 +0100 Subject: RFR: JDK-8022748 (new URI(u.toString()).equals(u), does not hold with paths containing colons Message-ID: <562EFC86.5060003@gmx.de> Hi, i investigated the problem described in JDK-8022748[1] i found that the parser needed to be rescued for confusion while handling relative URIs. A URI created through the relativize-method is schemaless and so it need to handle the special-case (a colon in the path-element). While there is also another way to handle it (encode the colon as %3A) i think we should not choose to encode everything. First it would introduce another style of special-case handling of a colon in the path (see the method maybeAddLeadingDot which is used while normalizing) and when resolving it back we would need to decode it back or leave it the encoded way which is not suggest by RFC2396 section "1.5 URI Transcribability". Also in Section 5 of RFC2396 it is suggested to rescue a colon in the path-element in a relative URI through prepending a "./" just like in maybeAddLeadingDot. I am not sure if it is worth to refactor the split(),join(),normalize() methods to provide a reusable colon-detection and -handling. So, instead of split/join/normalize please find my webrev with a simpler approach at: http://cr.openjdk.java.net/~sebastian/8022748/webrev.00/ -- Sebastian [1] https://bugs.openjdk.java.net/browse/JDK-8022748 From sebastian.sickelmann at gmx.de Tue Oct 27 04:28:31 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 27 Oct 2015 05:28:31 +0100 Subject: RFR: 5108778 Too many instances of java.lang.Boolean created in Java application(net) Message-ID: <562EFD6F.4000304@gmx.de> Hello, Actually I am searching through the JBS for low hanging fruits. Right now i am looking through the openjdk-sources and try to evaluate if i can make something about JDK-5108778. Please find my webrevs for the jdk(net) repos at: http://cr.openjdk.java.net/~sebastian/5108778/net/webrev.00/ The changes are javadoc only. -- Sebastian From mark.sheppard at oracle.com Tue Oct 27 15:25:17 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Tue, 27 Oct 2015 15:25:17 +0000 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562F92C5.3060105@oracle.com> References: <562D6679.9090509@oracle.com> <562F92C5.3060105@oracle.com> Message-ID: <562F975D.4010805@oracle.com> thanks Alan, I'll double check the illegal state token recognition. There was a security test that used this. property name of "jdk.net.hosts.file" is fine. we can remove the code for reading the service provider property sun.net.spi.nameservice.provider. and ignore its setting if we think that is most appropriate. I considered the ServiceConfigurationError, not quite "user friendly", but a reasonable form of feedback, to convey that the functionality was not available and so that an application could be amended to align with NameService retirement. On 27/10/2015 15:05, Alan Bateman wrote: > On 25/10/2015 23:32, Mark Sheppard wrote: >> Hi, >> please oblige and review the following changes >> http://cr.openjdk.java.net/~msheppar/8134577/webrev/ >> >> which address the issue raised in >> https://bugs.openjdk.java.net/browse/JDK-8134577 >> >> the operative word has been "eliminate". > This has been a very troublesome mechanism so good to see this reduced > down to a hosts file that we can we use for testing. > > Shouldn't the reading of sun.net.spi.nameservice.provider. be > removed so that the throwing of ServiceConfigurationError can be > removed too? > > Also just on the property name, I would assume it should be something > like "jdk.net.hosts.file" rather than "jdk.internal.hosts". > > Do we have tests that are using the illegal_state_exception token? I > didn't spot any in the webrev. Just wondering if this is the right > exception for testing. > > -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.sheppard at oracle.com Tue Oct 27 20:56:25 2015 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Tue, 27 Oct 2015 20:56:25 +0000 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: References: <562D6679.9090509@oracle.com> <562D72A7.1040302@oracle.com> Message-ID: <562FE4F9.1080205@oracle.com> Hi Max, thanks for the reply. I'll change the hosts file names used to correspond with an associated test dynamic update of host / ip mappings updates are accommodated as seen per the tests http://cr.openjdk.java.net/~msheppar/8134577/webrev/test/sun/net/InetAddress/nameservice/simple/DefaultCaching.java.sdiff.html http://cr.openjdk.java.net/~msheppar/8134577/webrev/test/sun/net/InetAddress/nameservice/simple/CacheTest.java.sdiff.html yes, that's correct the structure of hosts file does not align with /etc/hosts (ip address list of aliases), it aligns with the various customized NameService implementations in the test framework. I'll look into the UnknownHostException suggestion regards Mark On 26/10/2015 01:57, Wang Weijun wrote: > I see a lot of krb5 tests modified. > > Basically, the NameServiceDescriptor inside KDC.java maps everything to localhost except for one (I guess Artem invented the 2nd feature). Can we expand the grammar a little bit to support this? For example: > > not.existing.host UnknownHostException > * 127.0.0.1 > > I see you've added all names the tests have touched in test/sun/security/krb5/auto/hosts, but I am not sure if we might add randomly generated hostnames in any test later. > > BTW, the standard /etc/hosts has lines like "127.0.0.1 localhost" which has a different order than yours. Shall we rename hosts to something else so that people does not get confused? > > Thanks > Max > >>> http://cr.openjdk.java.net/~msheppar/8134577/webrev/ >>> >>> which address the issue raised in >>> https://bugs.openjdk.java.net/browse/JDK-8134577 >>> From kubota.yuji at gmail.com Wed Oct 28 18:22:59 2015 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 28 Oct 2015 11:22:59 -0700 Subject: Unexpected BindException in Endpoint.publish Message-ID: Hi all, I'm at the HackerGarten @ JavaOne15, and write a patch for OpenJDK community. This's second times from JavaOne14. :) We find an unexpected exception in JAX-WS, so I write a patch to fix it. We think that this issue may block the migration to JDK9 from JDK7. If we bind 0.0.0.0 ( using as wildcard ) to publish multiple as the following test code, JDK9 (and JDK8) returns "java.net.BindException: Address already in use.? as the below. But JDK7 does NOT return the exception. - Test code for reproduce --- import javax.jws.*; import javax.xml.ws.*; public class WSTest{ @WebService public static class Method1{ @WebMethod public String getMethod1Value(){ return "from Method1"; } } @WebService public static class Method2{ @WebMethod public String getMethod2Value(){ return "from Method2"; } } public static void main(String[] args) throws Exception{ Endpoint endPoint1 = Endpoint.publish("http://0.0.0.0:8081/method1", new Method1()); Endpoint endPoint2 = Endpoint.publish("http://0.0.0.0:8081/method2", new Method2()); System.out.println("Sleep 3 secs..."); Thread.sleep(3000); endPoint2.stop(); endPoint1.stop(); } } --- - StackTrace --- Exception in thread "main" com.sun.xml.internal.ws.server.ServerRtException: Server Runtime Error: java.net.BindException: Address already in use at com.sun.xml.internal.ws.transport.http.server.ServerMgr.createContext(ServerMgr.java:117) at com.sun.xml.internal.ws.transport.http.server.HttpEndpoint.publish(HttpEndpoint.java:64) at com.sun.xml.internal.ws.transport.http.server.EndpointImpl.publish(EndpointImpl.java:232) at com.sun.xml.internal.ws.spi.ProviderImpl.createAndPublishEndpoint(ProviderImpl.java:126) at javax.xml.ws.Endpoint.publish(Endpoint.java:240) at wstest.WSTest.main(WSTest.java:27) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:432) at sun.nio.ch.Net.bind(Net.java:424) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at sun.net.httpserver.ServerImpl.(ServerImpl.java:102) at sun.net.httpserver.HttpServerImpl.(HttpServerImpl.java:50) at sun.net.httpserver.DefaultHttpServerProvider.createHttpServer(DefaultHttpServerProvider.java:35) at com.sun.net.httpserver.HttpServer.create(HttpServer.java:130) at com.sun.xml.internal.ws.transport.http.server.ServerMgr.createContext(ServerMgr.java:86) ... 5 more ----- To publishes the Endpoint, JAX-WS checks whether the HttpContext has been created by given address, then creates a HttpContext if do not exist. If we sets 0.0.0.0 as given address, JAX-WS checks by ServerSocket#getLocalSocketAddress() (server local address), so returns BindException when 0.0.0.0 has been blinded already. Why so? JAX_WS-941[1] fixes NPE in Endpoint.stop but do not think about above situation. And JAX_WS-941 does not back port to JDK7. So I write a patch which is based jdk9/dev/jaxws (changeset: 637:2d84c6f4cbba) to fix the BindException with JAX_WS-941. Please review this patch :) [1]: https://java.net/jira/browse/JAX_WS-941 - Patch --- diff -r 2d84c6f4cbba src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java --- a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java Thu Oct 22 08:47:47 2015 -0700 +++ b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java Tue Oct 27 19:48:35 2015 +0900 @@ -38,6 +38,7 @@ import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.logging.Logger; +import java.util.Optional; /** * Manages all the WebService HTTP servers created by JAXWS runtime. @@ -81,24 +82,38 @@ synchronized(servers) { state = servers.get(inetAddress); if (state == null) { - logger.fine("Creating new HTTP Server at "+inetAddress); - // Creates server with default socket backlog - server = HttpServer.create(inetAddress, 0); - server.setExecutor(Executors.newCachedThreadPool()); - String path = url.toURI().getPath(); - logger.fine("Creating HTTP Context at = "+path); - HttpContext context = server.createContext(path); - server.start(); + final int finalPortNum = port; + Optional stateOpt = + servers.values() + .stream() + .filter(s -> s.getServer() + .getAddress() + .getPort() == finalPortNum) + .findAny(); - // we have to get actual inetAddress from server, which can differ from the original in some cases. - // e.g. A port number of zero will let the system pick up an ephemeral port in a bind operation, - // or IP: 0.0.0.0 - which is used to monitor network traffic from any valid IP address - inetAddress = server.getAddress(); + if (inetAddress.getAddress().isAnyLocalAddress() && + stateOpt.isPresent()) { + state = stateOpt.get(); + } else { + logger.fine("Creating new HTTP Server at "+inetAddress); + // Creates server with default socket backlog + server = HttpServer.create(inetAddress, 0); + server.setExecutor(Executors.newCachedThreadPool()); + String path = url.toURI().getPath(); + logger.fine("Creating HTTP Context at = "+path); + HttpContext context = server.createContext(path); + server.start(); - logger.fine("HTTP server started = "+inetAddress); - state = new ServerState(server, path); - servers.put(inetAddress, state); - return context; + // we have to get actual inetAddress from server, which can differ from the original in some cases. + // e.g. A port number of zero will let the system pick up an ephemeral port in a bind operation, + // or IP: 0.0.0.0 - which is used to monitor network traffic from any valid IP address + inetAddress = server.getAddress(); + + logger.fine("HTTP server started = "+inetAddress); + state = new ServerState(server, path); + servers.put(inetAddress, state); + return context; + } } } server = state.getServer(); --- From weijun.wang at oracle.com Thu Oct 29 02:08:20 2015 From: weijun.wang at oracle.com (Wang Weijun) Date: Thu, 29 Oct 2015 10:08:20 +0800 Subject: RFR: JDK-8134577 - Eliminate or standardize a replacement for sun.net.spi.nameservice.NameServiceDescriptor In-Reply-To: <562FE4F9.1080205@oracle.com> References: <562D6679.9090509@oracle.com> <562D72A7.1040302@oracle.com> <562FE4F9.1080205@oracle.com> Message-ID: <8D6810A3-EC43-4725-9EA3-C2CAC282FD04@oracle.com> > On Oct 28, 2015, at 4:56 AM, Mark Sheppard wrote: > > dynamic update of host / ip mappings updates are accommodated as seen per the tests > http://cr.openjdk.java.net/~msheppar/8134577/webrev/test/sun/net/InetAddress/nameservice/simple/DefaultCaching.java.sdiff.html > http://cr.openjdk.java.net/~msheppar/8134577/webrev/test/sun/net/InetAddress/nameservice/simple/CacheTest.java.sdiff.html Oh, I would say the old SimpleNameService class is quite good. Since the current fix is meant for test only and the new system property is called jdk.internal something, can we just introduce an internal method that looks like SimpleNameService? Who wants to use it can add "@modules java.base/jdk.internal.net.SimpleNameService". Yes I know this is quite against API encapsulation, but so is the internal system property. Thanks Max