From kubota.yuji at gmail.com Thu Mar 1 11:40:46 2018 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Thu, 1 Mar 2018 20:40:46 +0900 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> Message-ID: Hi all, Could you please review my patch(s)? Thanks, Yuji 2018-02-20 14:21 GMT+09:00 KUBOTA Yuji : > Hi Daniel, > > Thank you for your comment. > > Dispatcher is package-private class and handle method is called at > only this file in the package (sun.net.httpserver), and all call sites > look like to handle exception suitably. So we can remove `throws > IOException` clause without modifying the call site logic as like > webrev.03. > http://cr.openjdk.java.net/~ykubota/8169358/webrev.03 > > However, I could not find whether can I change a signature of public > method without specified steps. If we need to write CSR to remove > `throws IOException` clause, we should remove the clause by other > issue. And I want to commit webrev.02 at this issue. > http://cr.openjdk.java.net/~ykubota/8169358/webrev.02 > > I'd like to get your thoughts on that. > > P.S. I'm an author, not a committer, so I cannot access Mach5. > > Thanks, > Yuji > > 2018-02-17 1:11 GMT+09:00 Daniel Fuchs : >> Hi, >> >> On the surface I'd say that this change looks reasonable. >> However, handle is declared to throw IOException, and >> with this change it is guaranteed to never throw even >> RuntimeException. >> >> It seems that the code that calls handle() - at least in >> this file, has some logic to handle IOException - or >> other exception possibly thrown by Dispatcher::handle, >> which as far as I can see is not doing much more than what >> closeConnection does. So it looks as if calling >> closeConnection in handle() instead of throwing could be OK. >> >> However it would be good to at least try to figure out >> whether there are any other call sites for Dispatcher::handle, >> validate that no longer throwing will not modify the call site >> logic, and possibly investigate if the 'throws IOException' clause >> can be removed from Dispatcher::handle (that is: look >> whether Dispatcher is exposed and/or can be subclassed and >> if there are any existing subclasses). >> >> best regards, >> >> -- daniel >> >> >> On 16/02/2018 15:29, KUBOTA Yuji wrote: >>> >>> Hi Chris and Yasumasa, >>> >>> Sorry to have remained this issue for a long time. I come back. >>> >>> I updated my patch for the following three reasons. >>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.02/ >>> >>> * applies to the latest jdk/jdk instead of jdk9/dev >>> * adds test by modifying reproducer as like tests on >>> test/jdk/com/sun/net/httpserver >>> * prevents potential connection leak as Yasumasa said. For example, a >>> user sets own implementation of the thread pool to HttpServer and then >>> throws unexpected exceptions and errors. I think that we should save >>> existing exception handler to keep compatibility and minimize the risk >>> of connection leak by catching Throwable. >>> >>> I've tested test/jdk/com/sun/net/httpserver and passed. >>> I'm not a committer, so I can not access March 5. >>> >>> Could you review and sponsor it? >>> >>> Thanks, >>> Yuji >>> >>> 2016-11-11 12:11 GMT+09:00 KUBOTA Yuji : >>>> >>>> Hi Chris and Yasumasa, >>>> >>>> Thank you for your comments! >>>> >>>> I had faced connection leak on production by this handler. It seems >>>> like a corner-case but it's a real risk to me. >>>> I had focused REE on this issue, but it is a subclass of >>>> RuntimeException, so I think that we should investigate >>>> RuntimeException, too. >>>> >>>> If Chris agrees with the covering RuntimeException by JDK-8169358, I >>>> will investigate it and update my patch. >>>> If we should investigate on another issue, I want to add a test case >>>> to check fixing about REE like existing jtreg, and I will create a new >>>> issue after an investigation about RuntimeException. >>>> >>>> Any thoughts? >>>> >>>> Thank you! >>>> Yuji >>>> >>>> 2016-11-11 0:56 GMT+09:00 Chris Hegarty : >>>>> >>>>> >>>>>> On 10 Nov 2016, at 14:43, Yasumasa Suenaga wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>>> I think it best to just handle the specific case of REE, as it done in >>>>>>> Yuji?s patch. >>>>>> >>>>>> >>>>>> Will it be a cause of connection leak if RuntimeException is occurred >>>>>> in handler method? >>>>> >>>>> >>>>> No, at least not from this point in the code. >>>>> >>>>>> I think we should catch RuntimeException at least. >>>>> >>>>> >>>>> Possibly, but it seems like a corner-case of a corner-case. >>>>> >>>>>>>> I think you can use finally statement to close the connection in this >>>>>>>> case. >>>>>>> >>>>>>> >>>>>>> I don?t believe that this is possible. The handling of the connection >>>>>>> may be >>>>>>> done in a separate thread, some time after execute() returns. >>>>>> >>>>>> >>>>>> So I said we need to check the implementation of HTTP connection and >>>>>> dispatcher. >>>>> >>>>> >>>>> To me, this goes beyond the scope of the issue describe in JIRA, but if >>>>> you want to investigate that, then that?s fine. >>>>> >>>>> -Chris. >>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/11/10 23:00, Chris Hegarty wrote: >>>>>>> >>>>>>> >>>>>>>> On 9 Nov 2016, at 12:38, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> Hi Yuji, >>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ >>>>>>> >>>>>>> >>>>>>> I think Yuji?s patch is good as is. >>>>>>> >>>>>>>> I think you can use finally statement to close the connection in this >>>>>>>> case. >>>>>>> >>>>>>> >>>>>>> I don?t believe that this is possible. The handling of the connection >>>>>>> may be >>>>>>> done in a separate thread, some time after execute() returns. I think >>>>>>> it best >>>>>>> to just handle the specific case of REE, as it done in Yuji?s patch. >>>>>>> >>>>>>>> Your patch cannot close the connection when any other runtime >>>>>>>> exceptions are occurred. >>>>>>>> >>>>>>>> However, it is concerned double close if custom handler which is >>>>>>>> implemented by the user throws runtime exception after closing the >>>>>>>> connection. >>>>>>>> IMHO, you need to check the implementation of HTTP connection and >>>>>>>> dispatcher. >>>>>>> >>>>>>> >>>>>>> -Chris. >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2016/11/08 18:53, KUBOTA Yuji wrote: >>>>>>>>> >>>>>>>>> Hi Chris, >>>>>>>>> >>>>>>>>> Thank you for your review and updating this issues on JBS. >>>>>>>>> >>>>>>>>> I filed an issue: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8169358 >>>>>>>>> >>>>>>>>> I don't assign to me because I'm not a committer. >>>>>>>>> >>>>>>>>> 2016-11-08 0:28 GMT+09:00 Chris Hegarty >>>>>>>> oracle.com>: >>>>>>>>>>> >>>>>>>>>>> * patch >>>>>>>>>>> diff --git >>>>>>>>>>> >>>>>>>>>>> a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >>>>>>>>>>> >>>>>>>>>>> b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >>>>>>>>>>> --- >>>>>>>>>>> a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >>>>>>>>>>> +++ >>>>>>>>>>> b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >>>>>>>>>>> @@ -442,10 +442,13 @@ >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (4)", e1); >>>>>>>>>>> closeConnection(conn); >>>>>>>>>>> } catch (IOException e) { >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (5)", e); >>>>>>>>>>> closeConnection(conn); >>>>>>>>>>> + } catch (RejectedExecutionException e) { >>>>>>>>>>> + logger.log (Level.TRACE, "Dispatcher (9)", e); >>>>>>>>>>> + closeConnection(conn); >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> _ >>>>>>>>>>> static boolean debug = ServerConfig.debugEnabled (); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This looks ok. I wonder if some of these exceptions could be >>>>>>>>>> refactored >>>>>>>>>> into a catch clause with several exception types? >>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, I agree to keep simple. I updated my patch as below. >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ >>>>>>>>> Could you review it again? >>>>>>>>> >>>>>>>>> Thank you, >>>>>>>>> Yuji >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Chris. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> * steps to reproduce >>>>>>>>>>> 1. java -Djava.util.logging.config.file=logging.properties >>>>>>>>>>> SmallHttpServer >>>>>>>>>>> 2. post tcp connections by curl or other ways >>>>>>>>>>> e.g.: while true; do curl -XPOST --noproxy 127.0.0.1 >>>>>>>>>>> http://127.0.0.1:8080/; done >>>>>>>>>>> 3. wait RejectedExecutionException occurs as below and then >>>>>>>>>>> SmallHttpServer stops by this issue. >>>>>>>>>>> ---- >>>>>>>>>>> Nov 05, 2016 12:01:48 PM sun.net.httpserver.ServerImpl$Dispatcher >>>>>>>>>>> run >>>>>>>>>>> FINER: Dispatcher (7) >>>>>>>>>>> java.util.concurrent.RejectedExecutionException: Task >>>>>>>>>>> sun.net.httpserver.ServerImpl$Exchange at 37b50d9e rejected from >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor at 1b3178d4[Running, pool >>>>>>>>>>> size = >>>>>>>>>>> 1, active threads = 0, queued tasks = 0, completed tasks = 7168] >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(java.base/ThreadPoolExecutor.java:2076) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.reject(java.base/ThreadPoolExecutor.java:842) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.execute(java.base/ThreadPoolExecutor.java:1388) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>>>> sun.net.httpserver.ServerImpl$Dispatcher.handle(jdk.httpserver/ServerImpl.java:440) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>>>> sun.net.httpserver.ServerImpl$Dispatcher.run(jdk.httpserver/ServerImpl.java:405) >>>>>>>>>>> at java.lang.Thread.run(java.base/Thread.java:844) >>>>>>>>>>> (SmallHttpServer is stopping by not closing socket) >>>>>>>>>>> ---- >>>>>>>>>>> >>>>>>>>>>> *logging.properties >>>>>>>>>>> handlers = java.util.logging.ConsoleHandler >>>>>>>>>>> com.sun.net.httpserver.level = FINEST >>>>>>>>>>> java.util.logging.ConsoleHandler.level = FINEST >>>>>>>>>>> java.util.logging.ConsoleHandler.formatter = >>>>>>>>>>> java.util.logging.SimpleFormatter >>>>>>>>>>> >>>>>>>>>>> * SmallHttpServer.java >>>>>>>>>>> import com.sun.net.httpserver.HttpExchange; >>>>>>>>>>> import com.sun.net.httpserver.HttpHandler; >>>>>>>>>>> import com.sun.net.httpserver.HttpServer; >>>>>>>>>>> >>>>>>>>>>> import java.net.InetSocketAddress; >>>>>>>>>>> import java.util.Scanner; >>>>>>>>>>> import java.util.concurrent.LinkedBlockingQueue; >>>>>>>>>>> import java.util.concurrent.ThreadPoolExecutor; >>>>>>>>>>> import java.util.concurrent.TimeUnit; >>>>>>>>>>> >>>>>>>>>>> public class SmallHttpServer { >>>>>>>>>>> >>>>>>>>>>> public static void main(String[] args) throws Exception { >>>>>>>>>>> int POOL_SIZE = 1; >>>>>>>>>>> String HOST = args.length < 1 ? "127.0.0.1" : args[0]; >>>>>>>>>>> int PORT = args.length < 2 ? 8080 : >>>>>>>>>>> Integer.valueOf(args[1]); >>>>>>>>>>> >>>>>>>>>>> // Setup a minimum thread pool to rise >>>>>>>>>>> RejectExecutionException in httpserver >>>>>>>>>>> ThreadPoolExecutor miniHttpPoolExecutor >>>>>>>>>>> = new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, 0L, >>>>>>>>>>> TimeUnit.MICROSECONDS, >>>>>>>>>>> new LinkedBlockingQueue<>(1), (Runnable r) >>>>>>>>>>> -> { >>>>>>>>>>> return new Thread(r); >>>>>>>>>>> }); >>>>>>>>>>> HttpServer httpServer = HttpServer.create(new >>>>>>>>>>> InetSocketAddress(HOST, PORT), 0); >>>>>>>>>>> httpServer.setExecutor(miniHttpPoolExecutor); >>>>>>>>>>> >>>>>>>>>>> HttpHandler res200handler = (HttpExchange exchange) -> { >>>>>>>>>>> exchange.sendResponseHeaders(200, 0); >>>>>>>>>>> exchange.close(); >>>>>>>>>>> }; >>>>>>>>>>> httpServer.createContext("/", res200handler); >>>>>>>>>>> >>>>>>>>>>> httpServer.start(); >>>>>>>>>>> // Wait stdin to exit >>>>>>>>>>> Scanner in = new Scanner(System.in); >>>>>>>>>>> while (in.hasNext()) { >>>>>>>>>>> System.out.println(in.nextLine()); >>>>>>>>>>> } >>>>>>>>>>> httpServer.stop(0); >>>>>>>>>>> miniHttpPoolExecutor.shutdownNow(); >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Yuji >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> From pallavi.sonal at oracle.com Fri Mar 2 01:54:38 2018 From: pallavi.sonal at oracle.com (Pallavi Sonal) Date: Thu, 1 Mar 2018 17:54:38 -0800 (PST) Subject: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start Message-ID: Hi, Please review the following change to fix JDK-8144300. Bug : https://bugs.openjdk.java.net/browse/JDK-8144300 Webrev : http://cr.openjdk.java.net/~rpatil/8144300/webrev.00/ Whenever there is a wildcard(*) at either the beginning or the end of a hostname specified in the list of http.nonProxyHosts , the code works as expected and DIRECT connection is used for them. But in scenarios, where there is a wildcard both at the beginning and end of the hostname, proxy is used instead of DIRECT connection. All the tier1 and tier2 Mach 5 tests have passed. Thanks, Pallavi Sonal -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.gerasimov at oracle.com Fri Mar 2 04:43:09 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 1 Mar 2018 20:43:09 -0800 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> Message-ID: <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Hello! I'd like to do the next step toward removing the TwoStacks socket implementation on Windows. It would be aligning the two implementations (DualStack and TwoStacks), so they can be easier merged together during the next step. There are three PlainSocketImpl implementations in JDK: java.base/windows/classes/java/net/DualStackPlainSocketImpl.java java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java java.base/unix/classes/java/net/PlainSocketImpl.java While two later have very similar organization (in particular, set of native methods), the former is organized slightly differently. In order to merge the two Windows implementation together, they first need to be organized in a similar way. For consistency, DualStack implementation will be reorganized to be aligned with TwoStacks and unix/PlainSocketImpl. Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ The change looks somewhat messy, but in fact it was a series of incremental changes, which I still keep in the mercurial 'mq'. (I wish the webrev could be made incremental based on the mq patches, to make it easier to review.) The patched JDK builds fine and all the regression tests pass Okay. Thanks in advance! -- With kind regards, Ivan Gerasimov From james at lightbend.com Fri Mar 2 06:23:51 2018 From: james at lightbend.com (James Roper) Date: Fri, 2 Mar 2018 17:23:51 +1100 Subject: websockets In-Reply-To: References: <6F0393D9-DD02-45BD-B5CC-CE07E19EA6A3@oracle.com> Message-ID: Hi Simone, Sorry, I think there's been some missed information here. I am in no way suggesting that the API as provided currently should be changed or replaced. What I'm suggesting is that a high level adapter be provided out of the box that provides Reactive Streams support on top of this API. Yes, there's no formal definition of high/low level, but in the case of WebSockets, we can draw some lines based roughly on what web browsers (which is the target application developer experience that WebSockets were designed for) support as being high level, and what the WebSocket protocol allows beyond this as low level. I think a Reactive Streams adapter on top of the existing low level API that has feature parity with the JavaScript API for WebSockets offered by browsers is a worthwhile thing to aim for. Would you agree with that? So, for example, the browser APIs offer one error callback for handling errors from both directions. All WebSocket messages handled by the browser APIs are discrete, fixed length in memory, any chunking is handled transparently underneath. Developers in the browser don't have to explicitly send or implement any sending or receiving of the close messages, they just handle the high level concept of the socket being closed, and the browser implements the handshake underneath transparently. I'd say WebSocket proxies would be completely out of scope for a high level API, they definitely would want to deal with splitting messages, custom close handling, very precise error semantics etc, and such a use case would be well served by the current low level API. Regards, James On 26 February 2018 at 22:15, Simone Bordet wrote: > James, > > On Mon, Feb 26, 2018 at 4:37 AM, James Roper wrote: > > On the topic of error handling. A high level API doesn't need to report > each > > individual error with sending. > > Depends on what you mean by "high level APIs"... I guess there is no > formal definition of that, as one can even think that the Java socket > APIs are high level. > > > So firstly, it is impossible to report *all* > > errors with sending, since it's impossible to know, once you send a > message > > across the network, whether it got there or not. So if an application > > developer has a requirement to handle errors in sending (and > realistically, > > there is never a business require to handle just the errors that can be > > detected in sending, it's always about handling all errors), do we expect > > them to implement that logic in two places, both on the sending side to > > handle errors that can be detected on the sending side, and then > > additionally write logic to handle errors that can't be detected on the > > sending side (such as, for example, having the remote side send ACKs, and > > track the ACKs on the receiving end)? I doubt it, having to handle logic > on > > both sides is going to mean the application developer has to spend more > time > > implementing boiler plate that could otherwise be spent solving their > actual > > business problem. > > I think you are oversimplifying this. > > If my business is to write a WebSocket Proxy, I may totally want to > care about writing code that handles errors differently whether they > happened on the read side or on the write side, if just for logging > reasons. > > If my business is to write a protocol on top of WebSocket, I may want > to use custom WebSocket error codes (depending on errors) when I close > the connection. > I may want to send a WS message the moment I receive a WebSocket > "close" event from a peer. > > Sure, maybe in 80% of the cases read and write errors are handled the > same way, but as an application writer I would like to have the choice > between using an API that is "lower" level and allows me full > flexibility with respect to the WebSocket protocol, and an opinionated > (correctly opinionated in the majority of the cases) framework that > for some case reduces the API surface at the cost of something else > like flexibility, increased allocation, ecc. > Both have their use cases. > > I believe the JDK net group always stated they went for the first choice. > The lack of the second choice is under discussion, and as I have > already stated may be a matter of resources. > > > If an application developer needs to handle errors in > > sending messages, it would be much simpler for them to treat both types > of > > errors in the same way. And hence, a high level API should expose > detectable > > errors in sending on the receiving end, by cancelling upstream, > terminating > > the WebSocket with an error, and then emitting the error in the > downstream > > onError. > > > > When it comes to protocol specific concerns - a high level API should not > > expose that to developers. A developer should not be allowed to send a > text > > message between two split binary frames. A developer should not be > > responsible for implementing the closing handshake protocol. That is the > > responsibility of a high level API. WebSocket close messages are designed > > such that high level APIs don't need to expose them to end developers - > you > > have one status for success, and then many statuses for close. This > means, > > an application developer can signal a normal close by simply terminating > the > > stream, and can signal any of the other closes by terminating with an > error > > that gets mapped to an error status code - likewise, closes received can > be > > mapped in the same way. And, most of the error codes are for protocol > level > > errors anyway that a user should never be generating and can't really > > handle. So I don't think, if providing a high level API, it makes any > sense > > to expose close messages or the close handshake to application > developers - > > the WebSocket protocol by design is not meant to be used that way. > > I don't understand what you're saying here. > You want to notify application code if the other peer closed the > connection. > If you do, then you're exposing the application code to the WebSocket > close event, which is part of the WebSocket close handshake. > An application can use custom WebSocket error codes, or may want to > reply with a different error code, or may want to send a last message > with information of what has been already processed so far (similar to > HTTP/2 last stream id). > All that can only be provided by application code so the API must > allow for that. > Again, if you want to write an opinionated framework that covers 80% > of the cases in a simpler way, great, but I'd like to have other > choices too. > > > For fragmented frames, once again, I don't think the designers of the > > WebSocket protocol ever intended that this protocol level detail would > ever > > be exposed to application developers. My reading of the spec is that the > > reason fragmentation is allowed is to allow endpoints to work with fixed > > buffer sizes, allowing messages to be split if they choose. In fact, > this is > > exactly what the RFC says: > > > >> The primary purpose of fragmentation is to allow sending a message > >> that is of unknown size when the message is started without having to > >> buffer that message. If messages couldn't be fragmented, then an > >> endpoint would have to buffer the entire message so its length could > >> be counted before the first byte is sent. With fragmentation, a > >> server or intermediary may choose a reasonable size buffer and, when > >> the buffer is full, write a fragment to the network. > > > > > > https://tools.ietf.org/search/rfc6455#section-5.4 > > > > This is a protocol level detail, not something that is supposed to be > > exposed to application developers. > > It again depends on what you mean by "application developers". > If I'm writing a WebSocket Proxy, receiving WebSocket frames rather > than whole messages is exactly what I want. > > > It is reasonable to expect that an > > implementation will buffer and put the fragments back together, up to a > > configured buffer size, for application consumption. If a message does > > exceed that buffer size, then the implementation can close the socket > with > > the close code designed exactly for that purpose - 1009. > > Our experience with Jetty and its JSR 356 implementation is the opposite. > People are not aware of the fact that WebSocket frames can be > exbibytes large, and are not aware that there even is a limit. > They come from HTTP and they can send/receive any arbitrary large content. > When they see error 1009 they are extremely surprised. > So "reasonable" here may not be something we all agree on. > Rather than having the implementation take a decision for me (to > buffer frames and possibly discard them because of some implementation > little known limit), "reasonable" may mean "pass me all the WebSocket > frames, I know what to do", sticking to least surprise. > > One last orthogonal comment about usage of the Flow APIs in the new HTTP > client. > As far as I know the Flow APIs are only used for request and response > content, not the for HTTP protocol lifecycle. > HTTP requests/responses are not modeled as Publishers, only their content > is. > > In that light, perhaps we should reflect whether WebSocket should be > modeled in the same way; that is, the Flow API be used only to model > message content, and not the WebSocket protocol lifecycle. > > Thanks ! > > -- > Simone Bordet > --- > Finally, no matter how good the architecture and design are, > to deliver bug-free software with optimal performance and reliability, > the implementation technique must be flawless. Victoria Livschitz > -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrej.golovnin at gmail.com Fri Mar 2 08:28:21 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Fri, 2 Mar 2018 09:28:21 +0100 Subject: RFE: Add PATCH to the list of the supported HTTP methods Message-ID: Hi all, it would be nice if could add PATCH (https://tools.ietf.org/html/rfc5789) to the list of the supported HTTP methods in the HttpURLConnection class. I have attached two patches: - one for the HttpURLConnection class: java_net_HttpURLConnection_PATCH_method.diff - and one for the HTTP Client branch in the JDK sandbox repository which adds PATCH to the HttpRequest builder: java_net_http_HttpRequest_PATCH_method.diff Best regards, Andrej Golovnin -------------- next part -------------- A non-text attachment was scrubbed... Name: java_net_HttpURLConnection_PATCH_method.diff Type: application/octet-stream Size: 530 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: java_net_http_HttpRequest_PATCH_method.diff Type: application/octet-stream Size: 4885 bytes Desc: not available URL: From cjgunzel at gmail.com Sat Mar 3 18:49:19 2018 From: cjgunzel at gmail.com (Chuck Davis) Date: Sat, 3 Mar 2018 10:49:19 -0800 Subject: websockets In-Reply-To: References: <6F0393D9-DD02-45BD-B5CC-CE07E19EA6A3@oracle.com> Message-ID: James, this does not make any sense to me. Last I heard browsers no longer run java. Why would JSE give a hang about designing a communications mechanism for browsers on the client side? That is the job of the browser developers. On the other hand, JSE WebSocket is the best thing to come along for java clients since RMI which was released I think in 1.1. Java WebSocket is NOT for browsers....it's for Java clients!!!! ?? And I'm quite pleased with the progress I've made refactoring to use the jdk9 WebSocket api together with wildfly11 on the server (and still blazingly FAST -- seems to be snappier than jdk8 implementation). Now if the IDEs would get in gear to fully implement jdk9 I'd be in fat city. And my thanks to everyone who has participated in this particular conversation on this list and all the expertise that has been shared. >Yes, there's no formal definition of high/low level, but in the case of WebSockets, we can draw some lines based roughly on what web browsers (which is the target application developer experience that >WebSockets were designed for) support as being high level, and what the WebSocket protocol allows beyond this as low level. > > Regards, > > James > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james at lightbend.com Mon Mar 5 00:24:44 2018 From: james at lightbend.com (James Roper) Date: Mon, 5 Mar 2018 11:24:44 +1100 Subject: websockets In-Reply-To: References: <6F0393D9-DD02-45BD-B5CC-CE07E19EA6A3@oracle.com> Message-ID: Hi Chuck, I'm definitely not saying that the Java client is for browsers. I'm saying that the WebSocket protocol was designed to be used by the API provided by web browsers. No protocol is ever implemented on an island, it is implemented with an intended use case. To best understand a protocol, to understand why it is the way it is, to understand what would be a good way to use it, and what might be a bad way to use it, and therefore to understand what an API for it should look like, it is good to understand what use case the protocol was designed for. And that use case was to be used from browsers. So when you create a WebSocket client, or server, in any language, it is very valuable to look at the browser provided APIs to understand how the protocol was intended to be used, and therefore what features you should expose to users. To put in perspective, most server implementations of WebSockets are designed to be connected to from browsers, non browser clients are still a relatively niche use case for WebSockets. And so servers are going to look to the browser for what features that should give priority to. If a browser doesn't support X, then it's likely that a server implemention either won't support X, or won't provide very good or easy to use support for X. Consequently, it's of dubious value to provide a new client where support for X complicates the API, since servers aren't going to support that well anyway, it's providing a feature that you won't be able to use at the cost of making other features harder to use. Regards, James On 4 March 2018 at 05:49, Chuck Davis wrote: > James, this does not make any sense to me. Last I heard browsers no > longer run java. Why would JSE give a hang about designing a > communications mechanism for browsers on the client side? That is the job > of the browser developers. On the other hand, JSE WebSocket is the best > thing to come along for java clients since RMI which was released I think > in 1.1. Java WebSocket is NOT for browsers....it's for Java clients!!!! > ?? And I'm quite pleased with the progress I've made refactoring to use > the jdk9 WebSocket api together with wildfly11 on the server (and still > blazingly FAST -- seems to be snappier than jdk8 implementation). Now if > the IDEs would get in gear to fully implement jdk9 I'd be in fat city. > > And my thanks to everyone who has participated in this particular > conversation on this list and all the expertise that has been shared. > > >Yes, there's no formal definition of high/low level, but in the case of > WebSockets, we can draw some lines based roughly on what web browsers > (which is the target application developer experience that >WebSockets were > designed for) support as being high level, and what the WebSocket protocol > allows beyond this as low level. > >> >> Regards, >> >> James >> >> > -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjgunzel at gmail.com Mon Mar 5 01:59:41 2018 From: cjgunzel at gmail.com (Chuck Davis) Date: Sun, 4 Mar 2018 17:59:41 -0800 Subject: websockets In-Reply-To: References: <6F0393D9-DD02-45BD-B5CC-CE07E19EA6A3@oracle.com> Message-ID: Well, yes, WebSocket on the server is an entirely different beast (at this time -- although perhaps jdk9 WebSocket will make it's way to JEE someday) -- and that definitely has to work with browsers. But the whole world is not going to browsers as much as some would have us believe. The desktop is alive and well for internal users at businesses. And the quality of most of that software is so incredibly dismal one wonders how any of it gets sold. JavaFX and WebSockets can put the FIX on that stuff besides being cross-platform with update repositories! Long live desktop applications and long live the servers that power them!! ?? Oh, yeah, and long live WebSockets!! On Sun, Mar 4, 2018 at 4:24 PM, James Roper wrote: > Hi Chuck, > > I'm definitely not saying that the Java client is for browsers. I'm saying > that the WebSocket protocol was designed to be used by the API provided by > web browsers. No protocol is ever implemented on an island, it is > implemented with an intended use case. To best understand a protocol, to > understand why it is the way it is, to understand what would be a good way > to use it, and what might be a bad way to use it, and therefore to > understand what an API for it should look like, it is good to understand > what use case the protocol was designed for. And that use case was to be > used from browsers. So when you create a WebSocket client, or server, in > any language, it is very valuable to look at the browser provided APIs to > understand how the protocol was intended to be used, and therefore what > features you should expose to users. > > To put in perspective, most server implementations of WebSockets are > designed to be connected to from browsers, non browser clients are still a > relatively niche use case for WebSockets. And so servers are going to look > to the browser for what features that should give priority to. If a browser > doesn't support X, then it's likely that a server implemention either won't > support X, or won't provide very good or easy to use support for X. > Consequently, it's of dubious value to provide a new client where support > for X complicates the API, since servers aren't going to support that well > anyway, it's providing a feature that you won't be able to use at the cost > of making other features harder to use. > > Regards, > > James > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Tue Mar 6 11:00:58 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 6 Mar 2018 11:00:58 +0000 Subject: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start In-Reply-To: References: Message-ID: Hi Pallavi, this change looks good to me. The only thing left to mention is that you will need to update the copyright years - I guess you would have done it on submit anyway. Best regards Christoph From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of Pallavi Sonal Sent: Freitag, 2. M?rz 2018 02:55 To: net-dev at openjdk.java.net Subject: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start Hi, Please review the following change to fix JDK-8144300. Bug : https://bugs.openjdk.java.net/browse/JDK-8144300 Webrev : http://cr.openjdk.java.net/~rpatil/8144300/webrev.00/ Whenever there is a wildcard(*) at either the beginning or the end of a hostname specified in the list of http.nonProxyHosts , the code works as expected and DIRECT connection is used for them. But in scenarios, where there is a wildcard both at the beginning and end of the hostname, proxy is used instead of DIRECT connection. All the tier1 and tier2 Mach 5 tests have passed. Thanks, Pallavi Sonal -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Tue Mar 6 11:38:34 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 6 Mar 2018 11:38:34 +0000 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> Message-ID: Hi Yuji, I had a look and to me it seems safe to do like webrev.03 suggests. That is, remove the throws clause. As Daniel pointed out, it seems that all callers of the "handle" method would do merely the same. The only thing I'm not sure about is the CancelledKeyException caught in line 413. In that case no exception would be logged currently. With your change, it would now inside "handle". Maybe you want to handle a CancelledKeyException explicitly in "handle" now and suppress the log? What tests did you run? BTW: you could remove line 396: ' boolean closed;' while you are touching this file. It is not needed. Best regards Christoph > -----Original Message----- > From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of > KUBOTA Yuji > Sent: Donnerstag, 1. M?rz 2018 12:41 > To: OpenJDK Network Dev list > Cc: Yasumasa Suenaga > Subject: Re: httpserver does not close connections when > RejectedExecutionException occurs > > Hi all, > > Could you please review my patch(s)? > > Thanks, > Yuji > > 2018-02-20 14:21 GMT+09:00 KUBOTA Yuji : > > Hi Daniel, > > > > Thank you for your comment. > > > > Dispatcher is package-private class and handle method is called at > > only this file in the package (sun.net.httpserver), and all call sites > > look like to handle exception suitably. So we can remove `throws > > IOException` clause without modifying the call site logic as like > > webrev.03. > > http://cr.openjdk.java.net/~ykubota/8169358/webrev.03 > > > > However, I could not find whether can I change a signature of public > > method without specified steps. If we need to write CSR to remove > > `throws IOException` clause, we should remove the clause by other > > issue. And I want to commit webrev.02 at this issue. > > http://cr.openjdk.java.net/~ykubota/8169358/webrev.02 > > > > I'd like to get your thoughts on that. > > > > P.S. I'm an author, not a committer, so I cannot access Mach5. > > > > Thanks, > > Yuji > > > > 2018-02-17 1:11 GMT+09:00 Daniel Fuchs : > >> Hi, > >> > >> On the surface I'd say that this change looks reasonable. > >> However, handle is declared to throw IOException, and > >> with this change it is guaranteed to never throw even > >> RuntimeException. > >> > >> It seems that the code that calls handle() - at least in > >> this file, has some logic to handle IOException - or > >> other exception possibly thrown by Dispatcher::handle, > >> which as far as I can see is not doing much more than what > >> closeConnection does. So it looks as if calling > >> closeConnection in handle() instead of throwing could be OK. > >> > >> However it would be good to at least try to figure out > >> whether there are any other call sites for Dispatcher::handle, > >> validate that no longer throwing will not modify the call site > >> logic, and possibly investigate if the 'throws IOException' clause > >> can be removed from Dispatcher::handle (that is: look > >> whether Dispatcher is exposed and/or can be subclassed and > >> if there are any existing subclasses). > >> > >> best regards, > >> > >> -- daniel > >> > >> > >> On 16/02/2018 15:29, KUBOTA Yuji wrote: > >>> > >>> Hi Chris and Yasumasa, > >>> > >>> Sorry to have remained this issue for a long time. I come back. > >>> > >>> I updated my patch for the following three reasons. > >>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.02/ > >>> > >>> * applies to the latest jdk/jdk instead of jdk9/dev > >>> * adds test by modifying reproducer as like tests on > >>> test/jdk/com/sun/net/httpserver > >>> * prevents potential connection leak as Yasumasa said. For example, a > >>> user sets own implementation of the thread pool to HttpServer and > then > >>> throws unexpected exceptions and errors. I think that we should save > >>> existing exception handler to keep compatibility and minimize the risk > >>> of connection leak by catching Throwable. > >>> > >>> I've tested test/jdk/com/sun/net/httpserver and passed. > >>> I'm not a committer, so I can not access March 5. > >>> > >>> Could you review and sponsor it? > >>> > >>> Thanks, > >>> Yuji > >>> > >>> 2016-11-11 12:11 GMT+09:00 KUBOTA Yuji : > >>>> > >>>> Hi Chris and Yasumasa, > >>>> > >>>> Thank you for your comments! > >>>> > >>>> I had faced connection leak on production by this handler. It seems > >>>> like a corner-case but it's a real risk to me. > >>>> I had focused REE on this issue, but it is a subclass of > >>>> RuntimeException, so I think that we should investigate > >>>> RuntimeException, too. > >>>> > >>>> If Chris agrees with the covering RuntimeException by JDK-8169358, I > >>>> will investigate it and update my patch. > >>>> If we should investigate on another issue, I want to add a test case > >>>> to check fixing about REE like existing jtreg, and I will create a new > >>>> issue after an investigation about RuntimeException. > >>>> > >>>> Any thoughts? > >>>> > >>>> Thank you! > >>>> Yuji > >>>> > >>>> 2016-11-11 0:56 GMT+09:00 Chris Hegarty : > >>>>> > >>>>> > >>>>>> On 10 Nov 2016, at 14:43, Yasumasa Suenaga > wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>>> I think it best to just handle the specific case of REE, as it done in > >>>>>>> Yuji?s patch. > >>>>>> > >>>>>> > >>>>>> Will it be a cause of connection leak if RuntimeException is occurred > >>>>>> in handler method? > >>>>> > >>>>> > >>>>> No, at least not from this point in the code. > >>>>> > >>>>>> I think we should catch RuntimeException at least. > >>>>> > >>>>> > >>>>> Possibly, but it seems like a corner-case of a corner-case. > >>>>> > >>>>>>>> I think you can use finally statement to close the connection in this > >>>>>>>> case. > >>>>>>> > >>>>>>> > >>>>>>> I don?t believe that this is possible. The handling of the connection > >>>>>>> may be > >>>>>>> done in a separate thread, some time after execute() returns. > >>>>>> > >>>>>> > >>>>>> So I said we need to check the implementation of HTTP connection > and > >>>>>> dispatcher. > >>>>> > >>>>> > >>>>> To me, this goes beyond the scope of the issue describe in JIRA, but if > >>>>> you want to investigate that, then that?s fine. > >>>>> > >>>>> -Chris. > >>>>> > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2016/11/10 23:00, Chris Hegarty wrote: > >>>>>>> > >>>>>>> > >>>>>>>> On 9 Nov 2016, at 12:38, Yasumasa Suenaga > wrote: > >>>>>>>> > >>>>>>>> Hi Yuji, > >>>>>>>> > >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ > >>>>>>> > >>>>>>> > >>>>>>> I think Yuji?s patch is good as is. > >>>>>>> > >>>>>>>> I think you can use finally statement to close the connection in this > >>>>>>>> case. > >>>>>>> > >>>>>>> > >>>>>>> I don?t believe that this is possible. The handling of the connection > >>>>>>> may be > >>>>>>> done in a separate thread, some time after execute() returns. I > think > >>>>>>> it best > >>>>>>> to just handle the specific case of REE, as it done in Yuji?s patch. > >>>>>>> > >>>>>>>> Your patch cannot close the connection when any other runtime > >>>>>>>> exceptions are occurred. > >>>>>>>> > >>>>>>>> However, it is concerned double close if custom handler which is > >>>>>>>> implemented by the user throws runtime exception after closing > the > >>>>>>>> connection. > >>>>>>>> IMHO, you need to check the implementation of HTTP connection > and > >>>>>>>> dispatcher. > >>>>>>> > >>>>>>> > >>>>>>> -Chris. > >>>>>>> > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2016/11/08 18:53, KUBOTA Yuji wrote: > >>>>>>>>> > >>>>>>>>> Hi Chris, > >>>>>>>>> > >>>>>>>>> Thank you for your review and updating this issues on JBS. > >>>>>>>>> > >>>>>>>>> I filed an issue: > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8169358 > >>>>>>>>> > >>>>>>>>> I don't assign to me because I'm not a committer. > >>>>>>>>> > >>>>>>>>> 2016-11-08 0:28 GMT+09:00 Chris Hegarty >>>>>>>>> oracle.com>: > >>>>>>>>>>> > >>>>>>>>>>> * patch > >>>>>>>>>>> diff --git > >>>>>>>>>>> > >>>>>>>>>>> > a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > >>>>>>>>>>> > >>>>>>>>>>> > b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > >>>>>>>>>>> --- > >>>>>>>>>>> > a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > >>>>>>>>>>> +++ > >>>>>>>>>>> > b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > >>>>>>>>>>> @@ -442,10 +442,13 @@ > >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (4)", e1); > >>>>>>>>>>> closeConnection(conn); > >>>>>>>>>>> } catch (IOException e) { > >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (5)", e); > >>>>>>>>>>> closeConnection(conn); > >>>>>>>>>>> + } catch (RejectedExecutionException e) { > >>>>>>>>>>> + logger.log (Level.TRACE, "Dispatcher (9)", e); > >>>>>>>>>>> + closeConnection(conn); > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> _ > >>>>>>>>>>> static boolean debug = ServerConfig.debugEnabled (); > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> This looks ok. I wonder if some of these exceptions could be > >>>>>>>>>> refactored > >>>>>>>>>> into a catch clause with several exception types? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Yes, I agree to keep simple. I updated my patch as below. > >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ > >>>>>>>>> Could you review it again? > >>>>>>>>> > >>>>>>>>> Thank you, > >>>>>>>>> Yuji > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -Chris. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> * steps to reproduce > >>>>>>>>>>> 1. java -Djava.util.logging.config.file=logging.properties > >>>>>>>>>>> SmallHttpServer > >>>>>>>>>>> 2. post tcp connections by curl or other ways > >>>>>>>>>>> e.g.: while true; do curl -XPOST --noproxy 127.0.0.1 > >>>>>>>>>>> http://127.0.0.1:8080/; done > >>>>>>>>>>> 3. wait RejectedExecutionException occurs as below and then > >>>>>>>>>>> SmallHttpServer stops by this issue. > >>>>>>>>>>> ---- > >>>>>>>>>>> Nov 05, 2016 12:01:48 PM > sun.net.httpserver.ServerImpl$Dispatcher > >>>>>>>>>>> run > >>>>>>>>>>> FINER: Dispatcher (7) > >>>>>>>>>>> java.util.concurrent.RejectedExecutionException: Task > >>>>>>>>>>> sun.net.httpserver.ServerImpl$Exchange at 37b50d9e > rejected from > >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor at > 1b3178d4[Running, pool > >>>>>>>>>>> size = > >>>>>>>>>>> 1, active threads = 0, queued tasks = 0, completed tasks = > 7168] > >>>>>>>>>>> at > >>>>>>>>>>> > >>>>>>>>>>> > java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(java > .base/ThreadPoolExecutor.java:2076) > >>>>>>>>>>> at > >>>>>>>>>>> > >>>>>>>>>>> > java.util.concurrent.ThreadPoolExecutor.reject(java.base/ThreadPoolExecut > or.java:842) > >>>>>>>>>>> at > >>>>>>>>>>> > >>>>>>>>>>> > java.util.concurrent.ThreadPoolExecutor.execute(java.base/ThreadPoolExec > utor.java:1388) > >>>>>>>>>>> at > >>>>>>>>>>> > >>>>>>>>>>> > sun.net.httpserver.ServerImpl$Dispatcher.handle(jdk.httpserver/ServerImp > l.java:440) > >>>>>>>>>>> at > >>>>>>>>>>> > >>>>>>>>>>> > sun.net.httpserver.ServerImpl$Dispatcher.run(jdk.httpserver/ServerImpl.ja > va:405) > >>>>>>>>>>> at java.lang.Thread.run(java.base/Thread.java:844) > >>>>>>>>>>> (SmallHttpServer is stopping by not closing socket) > >>>>>>>>>>> ---- > >>>>>>>>>>> > >>>>>>>>>>> *logging.properties > >>>>>>>>>>> handlers = java.util.logging.ConsoleHandler > >>>>>>>>>>> com.sun.net.httpserver.level = FINEST > >>>>>>>>>>> java.util.logging.ConsoleHandler.level = FINEST > >>>>>>>>>>> java.util.logging.ConsoleHandler.formatter = > >>>>>>>>>>> java.util.logging.SimpleFormatter > >>>>>>>>>>> > >>>>>>>>>>> * SmallHttpServer.java > >>>>>>>>>>> import com.sun.net.httpserver.HttpExchange; > >>>>>>>>>>> import com.sun.net.httpserver.HttpHandler; > >>>>>>>>>>> import com.sun.net.httpserver.HttpServer; > >>>>>>>>>>> > >>>>>>>>>>> import java.net.InetSocketAddress; > >>>>>>>>>>> import java.util.Scanner; > >>>>>>>>>>> import java.util.concurrent.LinkedBlockingQueue; > >>>>>>>>>>> import java.util.concurrent.ThreadPoolExecutor; > >>>>>>>>>>> import java.util.concurrent.TimeUnit; > >>>>>>>>>>> > >>>>>>>>>>> public class SmallHttpServer { > >>>>>>>>>>> > >>>>>>>>>>> public static void main(String[] args) throws Exception { > >>>>>>>>>>> int POOL_SIZE = 1; > >>>>>>>>>>> String HOST = args.length < 1 ? "127.0.0.1" : args[0]; > >>>>>>>>>>> int PORT = args.length < 2 ? 8080 : > >>>>>>>>>>> Integer.valueOf(args[1]); > >>>>>>>>>>> > >>>>>>>>>>> // Setup a minimum thread pool to rise > >>>>>>>>>>> RejectExecutionException in httpserver > >>>>>>>>>>> ThreadPoolExecutor miniHttpPoolExecutor > >>>>>>>>>>> = new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, > 0L, > >>>>>>>>>>> TimeUnit.MICROSECONDS, > >>>>>>>>>>> new LinkedBlockingQueue<>(1), (Runnable r) > >>>>>>>>>>> -> { > >>>>>>>>>>> return new Thread(r); > >>>>>>>>>>> }); > >>>>>>>>>>> HttpServer httpServer = HttpServer.create(new > >>>>>>>>>>> InetSocketAddress(HOST, PORT), 0); > >>>>>>>>>>> httpServer.setExecutor(miniHttpPoolExecutor); > >>>>>>>>>>> > >>>>>>>>>>> HttpHandler res200handler = (HttpExchange exchange) -> > { > >>>>>>>>>>> exchange.sendResponseHeaders(200, 0); > >>>>>>>>>>> exchange.close(); > >>>>>>>>>>> }; > >>>>>>>>>>> httpServer.createContext("/", res200handler); > >>>>>>>>>>> > >>>>>>>>>>> httpServer.start(); > >>>>>>>>>>> // Wait stdin to exit > >>>>>>>>>>> Scanner in = new Scanner(System.in); > >>>>>>>>>>> while (in.hasNext()) { > >>>>>>>>>>> System.out.println(in.nextLine()); > >>>>>>>>>>> } > >>>>>>>>>>> httpServer.stop(0); > >>>>>>>>>>> miniHttpPoolExecutor.shutdownNow(); > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Yuji > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >> From christoph.langer at sap.com Tue Mar 6 12:53:47 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 6 Mar 2018 12:53:47 +0000 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: <16f1c79c5d134aa0a7670bf9f256af2d@sap.com> Hi Ivan, I went through the changes a bit and I would think it is really a good cleanup and will make the future merge of the implementations easier. One thing I saw was that TwoStacksPlainSocketImpl.c has an #include in line 25. I think that can be removed!? I had problems importing the patch(set) via mercurial queues - but maybe it is an issue of my local mercurial version. Best regards Christoph > -----Original Message----- > From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of > Ivan Gerasimov > Sent: Freitag, 2. M?rz 2018 05:43 > To: Chris Hegarty ; net-dev at openjdk.java.net > Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl > with TwoStacksPlainSocketImp [win] > > Hello! > > I'd like to do the next step toward removing the TwoStacks socket > implementation on Windows. > > It would be aligning the two implementations (DualStack and TwoStacks), > so they can be easier merged together during the next step. > > There are three PlainSocketImpl implementations in JDK: > java.base/windows/classes/java/net/DualStackPlainSocketImpl.java > java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java > java.base/unix/classes/java/net/PlainSocketImpl.java > > While two later have very similar organization (in particular, set of > native methods), the former is organized slightly differently. > In order to merge the two Windows implementation together, they first > need to be organized in a similar way. > For consistency, DualStack implementation will be reorganized to be > aligned with TwoStacks and unix/PlainSocketImpl. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 > Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ > > The change looks somewhat messy, but in fact it was a series of > incremental changes, which I still keep in the mercurial 'mq'. > > (I wish the webrev could be made incremental based on the mq patches, to > make it easier to review.) > > The patched JDK builds fine and all the regression tests pass Okay. > > > Thanks in advance! > -- > > With kind regards, > Ivan Gerasimov From daniel.fuchs at oracle.com Tue Mar 6 13:07:25 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 6 Mar 2018 13:07:25 +0000 Subject: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start In-Reply-To: References: Message-ID: <953e2ed8-4e1c-3f62-8fcc-9bee700ca4f2@oracle.com> Hi, Looks reasonable to me. best regards, -- daniel On 01/03/2018 14:46, Pallavi Sonal wrote: > Hi, > > Please review the following change to fix JDK-8144300. > > > > Bug : https://bugs.openjdk.java.net/browse/JDK-8144300 > > Webrev : http://cr.openjdk.java.net/~rpatil/8144300/webrev.00/ > > > > Whenever there is a wildcard(*) at either the beginning or the end of a hostname specified in the list of http.nonProxyHosts , the code works as expected and DIRECT connection is used for them. But in scenarios, where there is a wildcard both at the beginning and end of the hostname, proxy is used instead of DIRECT connection. > > > > All the tier1 and tier2 Mach 5 tests have passed. > > > > Thanks, > > Pallavi Sonal > From brian.burkhalter at oracle.com Tue Mar 6 15:26:44 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 6 Mar 2018 07:26:44 -0800 Subject: RFR 8198302: VS2017 (C4477) java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect printf format strings Message-ID: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8198302 Changes are in the diff below. Thanks, Brian --- a/src/java.base/windows/native/libnet/NetworkInterface_winXP.c +++ b/src/java.base/windows/native/libnet/NetworkInterface_winXP.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -39,14 +39,15 @@ #ifdef DEBUG void printnif (netif *nif) { #ifdef _WIN64 - printf ("nif:0x%I64x name:%s\n", nif,nif->name); + printf ("nif:0x%I64x name:%s\n", (UINT_PTR)nif, nif->name); #else - printf ("nif:0x%x name:%s\n", nif,nif->name); + printf ("nif:0x%x name:%s\n", nif, nif->name); #endif if (nif->dNameIsUnicode) { - printf ("dName:%S index:%d ", nif->displayName,nif->index); + printf ("dName:%S index:%d ", (unsigned short *)nif->displayName, + nif->index); } else { - printf ("dName:%s index:%d ", nif->displayName,nif->index); + printf ("dName:%s index:%d ", nif->displayName, nif->index); } printf ("naddrs:%d\n", nif->naddrs); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Tue Mar 6 15:57:40 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 6 Mar 2018 15:57:40 +0000 Subject: RFR 8198302: VS2017 (C4477) java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect printf format strings In-Reply-To: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> References: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> Message-ID: Looks good, Brian. From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of Brian Burkhalter Sent: Dienstag, 6. M?rz 2018 16:27 To: OpenJDK Network Dev list Subject: RFR 8198302: VS2017 (C4477) java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect printf format strings https://bugs.openjdk.java.net/browse/JDK-8198302 Changes are in the diff below. Thanks, Brian --- a/src/java.base/windows/native/libnet/NetworkInterface_winXP.c +++ b/src/java.base/windows/native/libnet/NetworkInterface_winXP.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -39,14 +39,15 @@ #ifdef DEBUG void printnif (netif *nif) { #ifdef _WIN64 - printf ("nif:0x%I64x name:%s\n", nif,nif->name); + printf ("nif:0x%I64x name:%s\n", (UINT_PTR)nif, nif->name); #else - printf ("nif:0x%x name:%s\n", nif,nif->name); + printf ("nif:0x%x name:%s\n", nif, nif->name); #endif if (nif->dNameIsUnicode) { - printf ("dName:%S index:%d ", nif->displayName,nif->index); + printf ("dName:%S index:%d ", (unsigned short *)nif->displayName, + nif->index); } else { - printf ("dName:%s index:%d ", nif->displayName,nif->index); + printf ("dName:%s index:%d ", nif->displayName, nif->index); } printf ("naddrs:%d\n", nif->naddrs); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Riggs at Oracle.com Tue Mar 6 16:06:10 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 6 Mar 2018 11:06:10 -0500 Subject: RFR 8198302: VS2017 (C4477) java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect printf format strings In-Reply-To: References: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> Message-ID: +1 On 3/6/2018 10:57 AM, Langer, Christoph wrote: > > Looks good, Brian. > > *From:*net-dev [mailto:net-dev-bounces at openjdk.java.net] *On Behalf Of > *Brian Burkhalter > *Sent:* Dienstag, 6. M?rz 2018 16:27 > *To:* OpenJDK Network Dev list > *Subject:* RFR 8198302: VS2017 (C4477) > java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect > printf format strings > > https://bugs.openjdk.java.net/browse/JDK-8198302 > > Changes are in the diff below. > > Thanks, > > Brian > > --- a/src/java.base/windows/native/libnet/NetworkInterface_winXP.c > > +++ b/src/java.base/windows/native/libnet/NetworkInterface_winXP.c > > @@ -1,5 +1,5 @@ > > ?/* > > - * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights > reserved. > > + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > * > > * This code is free software; you can redistribute it and/or modify it > > @@ -39,14 +39,15 @@ > > ?#ifdef DEBUG > > ?void printnif (netif *nif) { > > ?#ifdef _WIN64 > > - ? ? ? printf ("nif:0x%I64x name:%s\n", nif,nif->name); > > + ? ? ? printf ("nif:0x%I64x name:%s\n", (UINT_PTR)nif, nif->name); > > ?#else > > - ? ? ? printf ("nif:0x%x name:%s\n", nif,nif->name); > > + ? ? ? printf ("nif:0x%x name:%s\n", nif, nif->name); > > ?#endif > > ? ? ? if (nif->dNameIsUnicode) { > > - ? ? ? ? ? printf ("dName:%S index:%d ", nif->displayName,nif->index); > > + ? ? ? ? ? printf ("dName:%S index:%d ", (unsigned short > *)nif->displayName, > > + ? ? ? ? ? ? ? nif->index); > > ? ? ? } else { > > - ? ? ? ? ? printf ("dName:%s index:%d ", nif->displayName,nif->index); > > + ? ? ? ? ? printf ("dName:%s index:%d ", nif->displayName, nif->index); > > ? ? ? } > > ? ? ? printf ("naddrs:%d\n", nif->naddrs); > > ?} > From ivan.gerasimov at oracle.com Tue Mar 6 17:03:15 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 6 Mar 2018 09:03:15 -0800 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <16f1c79c5d134aa0a7670bf9f256af2d@sap.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <16f1c79c5d134aa0a7670bf9f256af2d@sap.com> Message-ID: Thank you Christoph for reviewing this! On 3/6/18 4:53 AM, Langer, Christoph wrote: > Hi Ivan, > > I went through the changes a bit and I would think it is really a good cleanup and will make the future merge of the implementations easier. > > One thing I saw was that TwoStacksPlainSocketImpl.c has an #include in line 25. I think that can be removed!? Certainly! Probably it's a leftover from already removed code. Thanks for spotting this. With kind regards, Ivan > I had problems importing the patch(set) via mercurial queues - but maybe it is an issue of my local mercurial version. > Best regards > Christoph > >> -----Original Message----- >> From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of >> Ivan Gerasimov >> Sent: Freitag, 2. M?rz 2018 05:43 >> To: Chris Hegarty ; net-dev at openjdk.java.net >> Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl >> with TwoStacksPlainSocketImp [win] >> >> Hello! >> >> I'd like to do the next step toward removing the TwoStacks socket >> implementation on Windows. >> >> It would be aligning the two implementations (DualStack and TwoStacks), >> so they can be easier merged together during the next step. >> >> There are three PlainSocketImpl implementations in JDK: >> java.base/windows/classes/java/net/DualStackPlainSocketImpl.java >> java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java >> java.base/unix/classes/java/net/PlainSocketImpl.java >> >> While two later have very similar organization (in particular, set of >> native methods), the former is organized slightly differently. >> In order to merge the two Windows implementation together, they first >> need to be organized in a similar way. >> For consistency, DualStack implementation will be reorganized to be >> aligned with TwoStacks and unix/PlainSocketImpl. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 >> Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ >> >> The change looks somewhat messy, but in fact it was a series of >> incremental changes, which I still keep in the mercurial 'mq'. >> >> (I wish the webrev could be made incremental based on the mq patches, to >> make it easier to review.) >> >> The patched JDK builds fine and all the regression tests pass Okay. >> >> >> Thanks in advance! >> -- >> >> With kind regards, >> Ivan Gerasimov > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Tue Mar 6 18:12:30 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 6 Mar 2018 18:12:30 +0000 Subject: RFR 8198302: VS2017 (C4477) java.base/windows/native/libnet/NetworkInterface_winXP.c incorrect printf format strings In-Reply-To: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> References: <3E8CDDE2-B9B6-482E-9D0D-448EFE5B3B39@oracle.com> Message-ID: Brian, I think this is fine. -Chris. On 06/03/18 15:26, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8198302 > > Changes are in the diff below. > > Thanks, > > Brian > > --- a/src/java.base/windows/native/libnet/NetworkInterface_winXP.c > +++ b/src/java.base/windows/native/libnet/NetworkInterface_winXP.c > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2003, 2017, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -39,14 +39,15 @@ > #ifdef DEBUG > void printnif (netif *nif) { > #ifdef _WIN64 > - printf ("nif:0x%I64x name:%s\n", nif,nif->name); > + printf ("nif:0x%I64x name:%s\n", (UINT_PTR)nif, nif->name); > #else > - printf ("nif:0x%x name:%s\n", nif,nif->name); > + printf ("nif:0x%x name:%s\n", nif, nif->name); > #endif > if (nif->dNameIsUnicode) { > - printf ("dName:%S index:%d ", nif->displayName,nif->index); > + printf ("dName:%S index:%d ", (unsigned short > *)nif->displayName, > + nif->index); > } else { > - printf ("dName:%s index:%d ", nif->displayName,nif->index); > + printf ("dName:%s index:%d ", nif->displayName, nif->index); > } > printf ("naddrs:%d\n", nif->naddrs); > } > From chris.hegarty at oracle.com Tue Mar 6 18:16:57 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 6 Mar 2018 18:16:57 +0000 Subject: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start In-Reply-To: References: Message-ID: <9ddafa91-d7f0-401e-3821-f141880701f5@oracle.com> Pallavi, On 01/03/18 14:46, Pallavi Sonal wrote: > Hi, > > Please review the following change to fix JDK-8144300. > > > > Bug : https://bugs.openjdk.java.net/browse/JDK-8144300 > > Webrev : http://cr.openjdk.java.net/~rpatil/8144300/webrev.00/ Looks ok to me. Please add the bugId to the @bug tag in the test before pushing. Thanks, -Chris. From chris.hegarty at oracle.com Tue Mar 6 18:40:43 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 6 Mar 2018 18:40:43 +0000 Subject: RFE: Add PATCH to the list of the supported HTTP methods In-Reply-To: References: Message-ID: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> Hi Andrej, On 02/03/18 08:28, Andrej Golovnin wrote: > Hi all, > > it would be nice if could add PATCH > (https://tools.ietf.org/html/rfc5789) to the list of the supported > ... > - and one for the HTTP Client branch in the JDK sandbox repository > which adds PATCH to the HttpRequest builder: > java_net_http_HttpRequest_PATCH_method.diff The JDK HTTP Client has: `HttpRequest.Builder::method(String method, BodyPublisher publisher)` ,so it is currently possible to use the `PATCH` method. Is `PATCH` sufficiently popular to warrant its own self-named method in the request builder? -Chris. From ivan.gerasimov at oracle.com Tue Mar 6 20:31:05 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 6 Mar 2018 12:31:05 -0800 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: In order to make is easier to review the fix, I made the webrev.ksh to generate a series of incremental webrevs from the mq patch stack. Here's the list of the incremental changes with a brief comments: WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/00/000/webrev/ Only changing the order of methods declaration WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/00/001/webrev/ Renaming initIDs to initProto WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/00/002/webrev/ Changing socketBind WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/00/003/webrev/ Changing socketCreate WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/00/004/webrev/ Changing socketListen WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/00/005/webrev/ Changin socketAvailable WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/00/006/webrev/ Changing socketClose0 WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/00/007/webrev/ Changing socketShutdown WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/00/008/webrev/ Changing socketSendUrgentData WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/00/009/webrev/ Changing socketAccept WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/00/010/webrev/ Changing socketConnect WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/00/011/webrev/ Minor editing, comments, moving code WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/00/012/webrev/ Changing socketSetOption WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/00/013/webrev/ Changing socketGetOption WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/00/014/webrev/ Moving a few methods one more time Accumulative webrev with all the changes above is available here: http://cr.openjdk.java.net/~igerasim/8198358/01/webrev/ Thanks in advance! Ivan On 3/1/18 8:43 PM, Ivan Gerasimov wrote: > Hello! > > I'd like to do the next step toward removing the TwoStacks socket > implementation on Windows. > > It would be aligning the two implementations (DualStack and > TwoStacks), so they can be easier merged together during the next step. > > There are three PlainSocketImpl implementations in JDK: > java.base/windows/classes/java/net/DualStackPlainSocketImpl.java > java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java > java.base/unix/classes/java/net/PlainSocketImpl.java > > While two later have very similar organization (in particular, set of > native methods), the former is organized slightly differently. > In order to merge the two Windows implementation together, they first > need to be organized in a similar way. > For consistency, DualStack implementation will be reorganized to be > aligned with TwoStacks and unix/PlainSocketImpl. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 > Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ > > The change looks somewhat messy, but in fact it was a series of > incremental changes, which I still keep in the mercurial 'mq'. > > (I wish the webrev could be made incremental based on the mq patches, > to make it easier to review.) > > The patched JDK builds fine and all the regression tests pass Okay. > > > Thanks in advance! -- With kind regards, Ivan Gerasimov From james at lightbend.com Wed Mar 7 00:55:13 2018 From: james at lightbend.com (James Roper) Date: Wed, 7 Mar 2018 11:55:13 +1100 Subject: RFE: Add PATCH to the list of the supported HTTP methods In-Reply-To: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> References: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> Message-ID: I don't have any opinion on the matter, but a datapoint to the popularity of PATCH is its use in the GitHub API: https://developer.github.com/v3/ GitHub is often looked to as a gold standard for how to write a REST API, so if they do it, it's likely that many others do too. On 7 March 2018 at 05:40, Chris Hegarty wrote: > Hi Andrej, > > On 02/03/18 08:28, Andrej Golovnin wrote: > >> Hi all, >> >> it would be nice if could add PATCH >> (https://tools.ietf.org/html/rfc5789) to the list of the supported >> ... >> - and one for the HTTP Client branch in the JDK sandbox repository >> which adds PATCH to the HttpRequest builder: >> java_net_http_HttpRequest_PATCH_method.diff >> > > The JDK HTTP Client has: > `HttpRequest.Builder::method(String method, BodyPublisher publisher)` > > ,so it is currently possible to use the `PATCH` method. > > Is `PATCH` sufficiently popular to warrant its own self-named > method in the request builder? > > -Chris. > -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From kubota.yuji at gmail.com Wed Mar 7 01:07:26 2018 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 7 Mar 2018 10:07:26 +0900 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> Message-ID: Hi Christoph, Thank you for your review and comment! 2018-03-06 20:38 GMT+09:00 Langer, Christoph : > I had a look and to me it seems safe to do like webrev.03 suggests. That is, remove the throws clause. As Daniel pointed out, it seems that all callers of the "handle" method would do merely the same. The only thing I'm not sure about is the CancelledKeyException caught in line 413. In that case no exception would be logged currently. With your change, it would now inside "handle". Maybe you want to handle a CancelledKeyException explicitly in "handle" now and suppress the log? SelectionKey::isReadable might throw CancelledKeyException in line 397, so the logging in line 413 still work except the exceptions from inside "handle". I want to handle unexpected exceptions and errors from "handle" to minimize the risk of connection leak. Separating logging also would help developers. > What tests did you run? I've tested test/jdk/com/sun/net/httpserver and passed. I can not access Mach5 because I'm not a committer. > BTW: you could remove line 396: ' boolean closed;' while you are touching this file. It is not needed. Nice catch! I removed this unused variable. http://cr.openjdk.java.net/~ykubota/8169358/webrev.04/ Best regards Yuji > Best regards > Christoph > >> -----Original Message----- >> From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of >> KUBOTA Yuji >> Sent: Donnerstag, 1. M?rz 2018 12:41 >> To: OpenJDK Network Dev list >> Cc: Yasumasa Suenaga >> Subject: Re: httpserver does not close connections when >> RejectedExecutionException occurs >> >> Hi all, >> >> Could you please review my patch(s)? >> >> Thanks, >> Yuji >> >> 2018-02-20 14:21 GMT+09:00 KUBOTA Yuji : >> > Hi Daniel, >> > >> > Thank you for your comment. >> > >> > Dispatcher is package-private class and handle method is called at >> > only this file in the package (sun.net.httpserver), and all call sites >> > look like to handle exception suitably. So we can remove `throws >> > IOException` clause without modifying the call site logic as like >> > webrev.03. >> > http://cr.openjdk.java.net/~ykubota/8169358/webrev.03 >> > >> > However, I could not find whether can I change a signature of public >> > method without specified steps. If we need to write CSR to remove >> > `throws IOException` clause, we should remove the clause by other >> > issue. And I want to commit webrev.02 at this issue. >> > http://cr.openjdk.java.net/~ykubota/8169358/webrev.02 >> > >> > I'd like to get your thoughts on that. >> > >> > P.S. I'm an author, not a committer, so I cannot access Mach5. >> > >> > Thanks, >> > Yuji >> > >> > 2018-02-17 1:11 GMT+09:00 Daniel Fuchs : >> >> Hi, >> >> >> >> On the surface I'd say that this change looks reasonable. >> >> However, handle is declared to throw IOException, and >> >> with this change it is guaranteed to never throw even >> >> RuntimeException. >> >> >> >> It seems that the code that calls handle() - at least in >> >> this file, has some logic to handle IOException - or >> >> other exception possibly thrown by Dispatcher::handle, >> >> which as far as I can see is not doing much more than what >> >> closeConnection does. So it looks as if calling >> >> closeConnection in handle() instead of throwing could be OK. >> >> >> >> However it would be good to at least try to figure out >> >> whether there are any other call sites for Dispatcher::handle, >> >> validate that no longer throwing will not modify the call site >> >> logic, and possibly investigate if the 'throws IOException' clause >> >> can be removed from Dispatcher::handle (that is: look >> >> whether Dispatcher is exposed and/or can be subclassed and >> >> if there are any existing subclasses). >> >> >> >> best regards, >> >> >> >> -- daniel >> >> >> >> >> >> On 16/02/2018 15:29, KUBOTA Yuji wrote: >> >>> >> >>> Hi Chris and Yasumasa, >> >>> >> >>> Sorry to have remained this issue for a long time. I come back. >> >>> >> >>> I updated my patch for the following three reasons. >> >>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.02/ >> >>> >> >>> * applies to the latest jdk/jdk instead of jdk9/dev >> >>> * adds test by modifying reproducer as like tests on >> >>> test/jdk/com/sun/net/httpserver >> >>> * prevents potential connection leak as Yasumasa said. For example, a >> >>> user sets own implementation of the thread pool to HttpServer and >> then >> >>> throws unexpected exceptions and errors. I think that we should save >> >>> existing exception handler to keep compatibility and minimize the risk >> >>> of connection leak by catching Throwable. >> >>> >> >>> I've tested test/jdk/com/sun/net/httpserver and passed. >> >>> I'm not a committer, so I can not access March 5. >> >>> >> >>> Could you review and sponsor it? >> >>> >> >>> Thanks, >> >>> Yuji >> >>> >> >>> 2016-11-11 12:11 GMT+09:00 KUBOTA Yuji : >> >>>> >> >>>> Hi Chris and Yasumasa, >> >>>> >> >>>> Thank you for your comments! >> >>>> >> >>>> I had faced connection leak on production by this handler. It seems >> >>>> like a corner-case but it's a real risk to me. >> >>>> I had focused REE on this issue, but it is a subclass of >> >>>> RuntimeException, so I think that we should investigate >> >>>> RuntimeException, too. >> >>>> >> >>>> If Chris agrees with the covering RuntimeException by JDK-8169358, I >> >>>> will investigate it and update my patch. >> >>>> If we should investigate on another issue, I want to add a test case >> >>>> to check fixing about REE like existing jtreg, and I will create a new >> >>>> issue after an investigation about RuntimeException. >> >>>> >> >>>> Any thoughts? >> >>>> >> >>>> Thank you! >> >>>> Yuji >> >>>> >> >>>> 2016-11-11 0:56 GMT+09:00 Chris Hegarty : >> >>>>> >> >>>>> >> >>>>>> On 10 Nov 2016, at 14:43, Yasumasa Suenaga >> wrote: >> >>>>>> >> >>>>>> Hi, >> >>>>>> >> >>>>>>> I think it best to just handle the specific case of REE, as it done in >> >>>>>>> Yuji?s patch. >> >>>>>> >> >>>>>> >> >>>>>> Will it be a cause of connection leak if RuntimeException is occurred >> >>>>>> in handler method? >> >>>>> >> >>>>> >> >>>>> No, at least not from this point in the code. >> >>>>> >> >>>>>> I think we should catch RuntimeException at least. >> >>>>> >> >>>>> >> >>>>> Possibly, but it seems like a corner-case of a corner-case. >> >>>>> >> >>>>>>>> I think you can use finally statement to close the connection in this >> >>>>>>>> case. >> >>>>>>> >> >>>>>>> >> >>>>>>> I don?t believe that this is possible. The handling of the connection >> >>>>>>> may be >> >>>>>>> done in a separate thread, some time after execute() returns. >> >>>>>> >> >>>>>> >> >>>>>> So I said we need to check the implementation of HTTP connection >> and >> >>>>>> dispatcher. >> >>>>> >> >>>>> >> >>>>> To me, this goes beyond the scope of the issue describe in JIRA, but if >> >>>>> you want to investigate that, then that?s fine. >> >>>>> >> >>>>> -Chris. >> >>>>> >> >>>>>> >> >>>>>> Yasumasa >> >>>>>> >> >>>>>> >> >>>>>> On 2016/11/10 23:00, Chris Hegarty wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>>> On 9 Nov 2016, at 12:38, Yasumasa Suenaga >> wrote: >> >>>>>>>> >> >>>>>>>> Hi Yuji, >> >>>>>>>> >> >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ >> >>>>>>> >> >>>>>>> >> >>>>>>> I think Yuji?s patch is good as is. >> >>>>>>> >> >>>>>>>> I think you can use finally statement to close the connection in this >> >>>>>>>> case. >> >>>>>>> >> >>>>>>> >> >>>>>>> I don?t believe that this is possible. The handling of the connection >> >>>>>>> may be >> >>>>>>> done in a separate thread, some time after execute() returns. I >> think >> >>>>>>> it best >> >>>>>>> to just handle the specific case of REE, as it done in Yuji?s patch. >> >>>>>>> >> >>>>>>>> Your patch cannot close the connection when any other runtime >> >>>>>>>> exceptions are occurred. >> >>>>>>>> >> >>>>>>>> However, it is concerned double close if custom handler which is >> >>>>>>>> implemented by the user throws runtime exception after closing >> the >> >>>>>>>> connection. >> >>>>>>>> IMHO, you need to check the implementation of HTTP connection >> and >> >>>>>>>> dispatcher. >> >>>>>>> >> >>>>>>> >> >>>>>>> -Chris. >> >>>>>>> >> >>>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> >> >>>>>>>> Yasumasa >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On 2016/11/08 18:53, KUBOTA Yuji wrote: >> >>>>>>>>> >> >>>>>>>>> Hi Chris, >> >>>>>>>>> >> >>>>>>>>> Thank you for your review and updating this issues on JBS. >> >>>>>>>>> >> >>>>>>>>> I filed an issue: >> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8169358 >> >>>>>>>>> >> >>>>>>>>> I don't assign to me because I'm not a committer. >> >>>>>>>>> >> >>>>>>>>> 2016-11-08 0:28 GMT+09:00 Chris Hegarty > >>>>>>>>> oracle.com>: >> >>>>>>>>>>> >> >>>>>>>>>>> * patch >> >>>>>>>>>>> diff --git >> >>>>>>>>>>> >> >>>>>>>>>>> >> a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> >>>>>>>>>>> >> >>>>>>>>>>> >> b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> >>>>>>>>>>> --- >> >>>>>>>>>>> >> a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> >>>>>>>>>>> +++ >> >>>>>>>>>>> >> b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> >>>>>>>>>>> @@ -442,10 +442,13 @@ >> >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (4)", e1); >> >>>>>>>>>>> closeConnection(conn); >> >>>>>>>>>>> } catch (IOException e) { >> >>>>>>>>>>> logger.log (Level.TRACE, "Dispatcher (5)", e); >> >>>>>>>>>>> closeConnection(conn); >> >>>>>>>>>>> + } catch (RejectedExecutionException e) { >> >>>>>>>>>>> + logger.log (Level.TRACE, "Dispatcher (9)", e); >> >>>>>>>>>>> + closeConnection(conn); >> >>>>>>>>>>> } >> >>>>>>>>>>> } >> >>>>>>>>>>> } >> >>>>>>>>>>> _ >> >>>>>>>>>>> static boolean debug = ServerConfig.debugEnabled (); >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> This looks ok. I wonder if some of these exceptions could be >> >>>>>>>>>> refactored >> >>>>>>>>>> into a catch clause with several exception types? >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Yes, I agree to keep simple. I updated my patch as below. >> >>>>>>>>> http://cr.openjdk.java.net/~ykubota/8169358/webrev.01/ >> >>>>>>>>> Could you review it again? >> >>>>>>>>> >> >>>>>>>>> Thank you, >> >>>>>>>>> Yuji >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> -Chris. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>>> * steps to reproduce >> >>>>>>>>>>> 1. java -Djava.util.logging.config.file=logging.properties >> >>>>>>>>>>> SmallHttpServer >> >>>>>>>>>>> 2. post tcp connections by curl or other ways >> >>>>>>>>>>> e.g.: while true; do curl -XPOST --noproxy 127.0.0.1 >> >>>>>>>>>>> http://127.0.0.1:8080/; done >> >>>>>>>>>>> 3. wait RejectedExecutionException occurs as below and then >> >>>>>>>>>>> SmallHttpServer stops by this issue. >> >>>>>>>>>>> ---- >> >>>>>>>>>>> Nov 05, 2016 12:01:48 PM >> sun.net.httpserver.ServerImpl$Dispatcher >> >>>>>>>>>>> run >> >>>>>>>>>>> FINER: Dispatcher (7) >> >>>>>>>>>>> java.util.concurrent.RejectedExecutionException: Task >> >>>>>>>>>>> sun.net.httpserver.ServerImpl$Exchange at 37b50d9e >> rejected from >> >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor at >> 1b3178d4[Running, pool >> >>>>>>>>>>> size = >> >>>>>>>>>>> 1, active threads = 0, queued tasks = 0, completed tasks = >> 7168] >> >>>>>>>>>>> at >> >>>>>>>>>>> >> >>>>>>>>>>> >> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(java >> .base/ThreadPoolExecutor.java:2076) >> >>>>>>>>>>> at >> >>>>>>>>>>> >> >>>>>>>>>>> >> java.util.concurrent.ThreadPoolExecutor.reject(java.base/ThreadPoolExecut >> or.java:842) >> >>>>>>>>>>> at >> >>>>>>>>>>> >> >>>>>>>>>>> >> java.util.concurrent.ThreadPoolExecutor.execute(java.base/ThreadPoolExec >> utor.java:1388) >> >>>>>>>>>>> at >> >>>>>>>>>>> >> >>>>>>>>>>> >> sun.net.httpserver.ServerImpl$Dispatcher.handle(jdk.httpserver/ServerImp >> l.java:440) >> >>>>>>>>>>> at >> >>>>>>>>>>> >> >>>>>>>>>>> >> sun.net.httpserver.ServerImpl$Dispatcher.run(jdk.httpserver/ServerImpl.ja >> va:405) >> >>>>>>>>>>> at java.lang.Thread.run(java.base/Thread.java:844) >> >>>>>>>>>>> (SmallHttpServer is stopping by not closing socket) >> >>>>>>>>>>> ---- >> >>>>>>>>>>> >> >>>>>>>>>>> *logging.properties >> >>>>>>>>>>> handlers = java.util.logging.ConsoleHandler >> >>>>>>>>>>> com.sun.net.httpserver.level = FINEST >> >>>>>>>>>>> java.util.logging.ConsoleHandler.level = FINEST >> >>>>>>>>>>> java.util.logging.ConsoleHandler.formatter = >> >>>>>>>>>>> java.util.logging.SimpleFormatter >> >>>>>>>>>>> >> >>>>>>>>>>> * SmallHttpServer.java >> >>>>>>>>>>> import com.sun.net.httpserver.HttpExchange; >> >>>>>>>>>>> import com.sun.net.httpserver.HttpHandler; >> >>>>>>>>>>> import com.sun.net.httpserver.HttpServer; >> >>>>>>>>>>> >> >>>>>>>>>>> import java.net.InetSocketAddress; >> >>>>>>>>>>> import java.util.Scanner; >> >>>>>>>>>>> import java.util.concurrent.LinkedBlockingQueue; >> >>>>>>>>>>> import java.util.concurrent.ThreadPoolExecutor; >> >>>>>>>>>>> import java.util.concurrent.TimeUnit; >> >>>>>>>>>>> >> >>>>>>>>>>> public class SmallHttpServer { >> >>>>>>>>>>> >> >>>>>>>>>>> public static void main(String[] args) throws Exception { >> >>>>>>>>>>> int POOL_SIZE = 1; >> >>>>>>>>>>> String HOST = args.length < 1 ? "127.0.0.1" : args[0]; >> >>>>>>>>>>> int PORT = args.length < 2 ? 8080 : >> >>>>>>>>>>> Integer.valueOf(args[1]); >> >>>>>>>>>>> >> >>>>>>>>>>> // Setup a minimum thread pool to rise >> >>>>>>>>>>> RejectExecutionException in httpserver >> >>>>>>>>>>> ThreadPoolExecutor miniHttpPoolExecutor >> >>>>>>>>>>> = new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, >> 0L, >> >>>>>>>>>>> TimeUnit.MICROSECONDS, >> >>>>>>>>>>> new LinkedBlockingQueue<>(1), (Runnable r) >> >>>>>>>>>>> -> { >> >>>>>>>>>>> return new Thread(r); >> >>>>>>>>>>> }); >> >>>>>>>>>>> HttpServer httpServer = HttpServer.create(new >> >>>>>>>>>>> InetSocketAddress(HOST, PORT), 0); >> >>>>>>>>>>> httpServer.setExecutor(miniHttpPoolExecutor); >> >>>>>>>>>>> >> >>>>>>>>>>> HttpHandler res200handler = (HttpExchange exchange) -> >> { >> >>>>>>>>>>> exchange.sendResponseHeaders(200, 0); >> >>>>>>>>>>> exchange.close(); >> >>>>>>>>>>> }; >> >>>>>>>>>>> httpServer.createContext("/", res200handler); >> >>>>>>>>>>> >> >>>>>>>>>>> httpServer.start(); >> >>>>>>>>>>> // Wait stdin to exit >> >>>>>>>>>>> Scanner in = new Scanner(System.in); >> >>>>>>>>>>> while (in.hasNext()) { >> >>>>>>>>>>> System.out.println(in.nextLine()); >> >>>>>>>>>>> } >> >>>>>>>>>>> httpServer.stop(0); >> >>>>>>>>>>> miniHttpPoolExecutor.shutdownNow(); >> >>>>>>>>>>> } >> >>>>>>>>>>> } >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks, >> >>>>>>>>>>> Yuji >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> >>>>> >> >> From andrej.golovnin at gmail.com Wed Mar 7 08:36:38 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Wed, 7 Mar 2018 09:36:38 +0100 Subject: RFE: Add PATCH to the list of the supported HTTP methods In-Reply-To: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> References: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> Message-ID: Hi Chris, > The JDK HTTP Client has: > `HttpRequest.Builder::method(String method, BodyPublisher publisher)` > > ,so it is currently possible to use the `PATCH` method. > > Is `PATCH` sufficiently popular to warrant its own self-named > method in the request builder? I don't know how popular is it. We have just started to use it with one of our services and discovered that we cannot use it with HttpURLConnection at all. And I thought when we add the support for PATCH to HttpURLConnection, then it would be nice to have it in HTTP Client too. When you decided not to add it to HTTP Client, then it is OK for me too. But I think that HttpURLConnection should be changed. Best regards, Andrej Golovnin From daniel.fuchs at oracle.com Wed Mar 7 15:56:13 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 7 Mar 2018 15:56:13 +0000 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> Message-ID: <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> Hi Yuji, On 07/03/2018 01:07, KUBOTA Yuji wrote: > Nice catch! I removed this unused variable. > http://cr.openjdk.java.net/~ykubota/8169358/webrev.04/ > > Best regards > Yuji > This looks good to me as well. I have imported your patch and sent it to our test system and things are looking fine. Could you prepare a final changeset with a properly formatted comment [1], use jcheck [2] to verify that your changeset conforms to the OpenJDK rules (no trailing whitespaces, no tabs etc...), and upload it somewhere to http://cr.openjdk.java.net/~ykubota/8169358 ? If there aren't any other review feedback in the mean time, I'll be happy to sponsor your changes when that's ready. best regards, -- daniel [1] http://openjdk.java.net/guide/producingChangeset.html [2] http://openjdk.java.net/projects/code-tools/jcheck/ From chris.hegarty at oracle.com Wed Mar 7 15:55:55 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 7 Mar 2018 15:55:55 +0000 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> Message-ID: On 07/03/18 01:07, KUBOTA Yuji wrote: >.. > Nice catch! I removed this unused variable. > http://cr.openjdk.java.net/~ykubota/8169358/webrev.04/ I think is good. Thanks. -Chris. From kubota.yuji at gmail.com Thu Mar 8 02:56:36 2018 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Thu, 8 Mar 2018 11:56:36 +0900 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> Message-ID: Hi Daniel, Thank you for your sponsoring! 2018-03-08 0:56 GMT+09:00 Daniel Fuchs : > Could you prepare a final changeset with a properly formatted > comment [1], use jcheck [2] to verify that your changeset conforms > to the OpenJDK rules (no trailing whitespaces, no tabs etc...), > and upload it somewhere to http://cr.openjdk.java.net/~ykubota/8169358 ? My final changeset is below. (no jcheck error from webrev.04) http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked If Christoph could not check webrev.04, please use the following changeset. http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked-two-reviewers Best regards Yuji From christoph.langer at sap.com Thu Mar 8 07:47:38 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 8 Mar 2018 07:47:38 +0000 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> Message-ID: Hi Yuji, looks good. Best regards Christoph > -----Original Message----- > From: KUBOTA Yuji [mailto:kubota.yuji at gmail.com] > Sent: Donnerstag, 8. M?rz 2018 03:57 > To: Daniel Fuchs > Cc: Langer, Christoph ; Yasumasa Suenaga > ; OpenJDK Network Dev list dev at openjdk.java.net> > Subject: Re: httpserver does not close connections when > RejectedExecutionException occurs > > Hi Daniel, > > Thank you for your sponsoring! > > 2018-03-08 0:56 GMT+09:00 Daniel Fuchs : > > Could you prepare a final changeset with a properly formatted > > comment [1], use jcheck [2] to verify that your changeset conforms > > to the OpenJDK rules (no trailing whitespaces, no tabs etc...), > > and upload it somewhere to http://cr.openjdk.java.net/~ykubota/8169358 > ? > > My final changeset is below. (no jcheck error from webrev.04) > http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked > > If Christoph could not check webrev.04, please use the following changeset. > http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked- > two-reviewers > > Best regards > Yuji From kubota.yuji at gmail.com Thu Mar 8 09:28:44 2018 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Thu, 8 Mar 2018 18:28:44 +0900 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <4a4587b6-048c-b337-f289-09582dc95abe@oracle.com> <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> Message-ID: 2018-03-08 16:47 GMT+09:00 Langer, Christoph : > looks good. Thank you! Let's use the following changeset if there aren't any problems. http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked Best regards Yuji > > Best regards > Christoph > >> -----Original Message----- >> From: KUBOTA Yuji [mailto:kubota.yuji at gmail.com] >> Sent: Donnerstag, 8. M?rz 2018 03:57 >> To: Daniel Fuchs >> Cc: Langer, Christoph ; Yasumasa Suenaga >> ; OpenJDK Network Dev list > dev at openjdk.java.net> >> Subject: Re: httpserver does not close connections when >> RejectedExecutionException occurs >> >> Hi Daniel, >> >> Thank you for your sponsoring! >> >> 2018-03-08 0:56 GMT+09:00 Daniel Fuchs : >> > Could you prepare a final changeset with a properly formatted >> > comment [1], use jcheck [2] to verify that your changeset conforms >> > to the OpenJDK rules (no trailing whitespaces, no tabs etc...), >> > and upload it somewhere to http://cr.openjdk.java.net/~ykubota/8169358 >> ? >> >> My final changeset is below. (no jcheck error from webrev.04) >> http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked >> >> If Christoph could not check webrev.04, please use the following changeset. >> http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked- >> two-reviewers >> >> Best regards >> Yuji From pallavi.sonal at oracle.com Thu Mar 8 12:27:48 2018 From: pallavi.sonal at oracle.com (Pallavi Sonal) Date: Thu, 8 Mar 2018 04:27:48 -0800 (PST) Subject: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start In-Reply-To: <9ddafa91-d7f0-401e-3821-f141880701f5@oracle.com> References: <9ddafa91-d7f0-401e-3821-f141880701f5@oracle.com> Message-ID: <8155c9ca-35bc-4fe4-84b0-d3ffba74c54d@default> Thanks Chris, Daniel and Christoph for the review. I will add bug id to the test case and update copyright year before pushing the patch. Thanks, Pallavi Sonal -----Original Message----- From: Chris Hegarty Sent: 06 March 2018 23:47 To: Pallavi Sonal ; net-dev at openjdk.java.net Cc: core-libs-dev at openjdk.java.net Subject: Re: RFR: JDK-8144300 : Java does not honor http.nonProxyHosts value having wildcard * both at end and start Pallavi, On 01/03/18 14:46, Pallavi Sonal wrote: > Hi, > > Please review the following change to fix JDK-8144300. > > > > Bug : https://bugs.openjdk.java.net/browse/JDK-8144300 > > Webrev : http://cr.openjdk.java.net/~rpatil/8144300/webrev.00/ Looks ok to me. Please add the bugId to the @bug tag in the test before pushing. Thanks, -Chris. From daniel.fuchs at oracle.com Thu Mar 8 16:26:38 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 8 Mar 2018 16:26:38 +0000 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: References: <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> Message-ID: <67b6a9b6-ed0d-37bc-ad29-a60dbe01a2cf@oracle.com> Thank you Yuji! This has been pushed. For the record - since you are an author, you don't need to list your address in the Contributed-by section as you already appear as the author of the changeset: http://hg.openjdk.java.net/jdk/jdk/rev/5447851ff0f6 ``` author ykubota ``` best regards, -- daniel On 08/03/2018 09:28, KUBOTA Yuji wrote: > Thank you! > > Let's use the following changeset if there aren't any problems. > http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked > > Best regards > Yuji From kubota.yuji at gmail.com Fri Mar 9 01:30:27 2018 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Fri, 9 Mar 2018 10:30:27 +0900 Subject: httpserver does not close connections when RejectedExecutionException occurs In-Reply-To: <67b6a9b6-ed0d-37bc-ad29-a60dbe01a2cf@oracle.com> References: <5596fa72-1a84-ab26-1eaa-247e3fd3314a@gmail.com> <2c757092-069e-ca33-3ac7-d1f8c94e1c65@oracle.com> <67b6a9b6-ed0d-37bc-ad29-a60dbe01a2cf@oracle.com> Message-ID: Thank you, Daniel and everyone! I will keep how to make the suitable changeset when next contributions. Best regards, Yuji 2018-03-09 1:26 GMT+09:00 Daniel Fuchs : > Thank you Yuji! > > This has been pushed. > > For the record - since you are an author, you don't need > to list your address in the Contributed-by section as you > already appear as the author of the changeset: > > http://hg.openjdk.java.net/jdk/jdk/rev/5447851ff0f6 > ``` > author ykubota > ``` > > best regards, > > -- daniel > > > On 08/03/2018 09:28, KUBOTA Yuji wrote: >> >> Thank you! >> >> Let's use the following changeset if there aren't any problems. >> http://cr.openjdk.java.net/~ykubota/8169358/final_changeset_jckecked >> >> Best regards >> Yuji > > From chris.hegarty at oracle.com Fri Mar 9 11:50:44 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 9 Mar 2018 11:50:44 +0000 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: Ivan, Thanks for breaking up the changes, it makes it easier to review. I disagree with changing DualStackPlainSocketImp to conform with TwoStacks (and unix/PlainSocketImpl). I would prefer to move the latter to the former. Specifically, around the use of fdAccess. -Chris. On 06/03/18 20:31, Ivan Gerasimov wrote: > In order to make is easier to review the fix, I made the webrev.ksh to > generate a series of incremental webrevs from the mq patch stack. > > Here's the list of the incremental changes with a brief comments: > > WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/00/000/webrev/ > Only changing the order of methods declaration > > WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/00/001/webrev/ > Renaming initIDs to initProto > > WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/00/002/webrev/ > Changing socketBind > > WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/00/003/webrev/ > Changing socketCreate > > WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/00/004/webrev/ > Changing socketListen > > WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/00/005/webrev/ > Changin socketAvailable > > WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/00/006/webrev/ > Changing socketClose0 > > WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/00/007/webrev/ > Changing socketShutdown > > WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/00/008/webrev/ > Changing socketSendUrgentData > > WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/00/009/webrev/ > Changing socketAccept > > WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/00/010/webrev/ > Changing socketConnect > > WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/00/011/webrev/ > Minor editing, comments, moving code > > WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/00/012/webrev/ > Changing socketSetOption > > WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/00/013/webrev/ > Changing socketGetOption > > WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/00/014/webrev/ > Moving a few methods one more time > > > Accumulative webrev with all the changes above is available here: > http://cr.openjdk.java.net/~igerasim/8198358/01/webrev/ > > > Thanks in advance! > > Ivan > > On 3/1/18 8:43 PM, Ivan Gerasimov wrote: >> Hello! >> >> I'd like to do the next step toward removing the TwoStacks socket >> implementation on Windows. >> >> It would be aligning the two implementations (DualStack and >> TwoStacks), so they can be easier merged together during the next step. >> >> There are three PlainSocketImpl implementations in JDK: >> java.base/windows/classes/java/net/DualStackPlainSocketImpl.java >> java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java >> java.base/unix/classes/java/net/PlainSocketImpl.java >> >> While two later have very similar organization (in particular, set of >> native methods), the former is organized slightly differently. >> In order to merge the two Windows implementation together, they first >> need to be organized in a similar way. >> For consistency, DualStack implementation will be reorganized to be >> aligned with TwoStacks and unix/PlainSocketImpl. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 >> Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ >> >> The change looks somewhat messy, but in fact it was a series of >> incremental changes, which I still keep in the mercurial 'mq'. >> >> (I wish the webrev could be made incremental based on the mq patches, >> to make it easier to review.) >> >> The patched JDK builds fine and all the regression tests pass Okay. >> >> >> Thanks in advance! > From ivan.gerasimov at oracle.com Fri Mar 9 17:21:38 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 9 Mar 2018 09:21:38 -0800 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: <422fbb96-e767-6b37-e697-0003939487e9@oracle.com> Hi Chris! Thank you for reviewing the patch set! On 3/9/18 3:50 AM, Chris Hegarty wrote: > Ivan, > > Thanks for breaking up the changes, it makes it easier to review. > > I disagree with changing DualStackPlainSocketImp to conform with > TwoStacks (and unix/PlainSocketImpl). I would prefer to move the > latter to the former. Specifically, around the use of fdAccess. > Do you think unix/PlainSocketImpl should also be refactored that way at some point later? It looks tempting to make Windows and Unix implementation similar, so that any further modifications should they need to be done to both will be easier to apply. That was the primary reason why I choose TwoStacks as the basis and aligned DualStack with it. It can surely be done the opposite way, though I want to make sure it is the preferable way to go. With kind regards, Ivan > -Chris. > > > On 06/03/18 20:31, Ivan Gerasimov wrote: >> In order to make is easier to review the fix, I made the webrev.ksh >> to generate a series of incremental webrevs from the mq patch stack. >> >> Here's the list of the incremental changes with a brief comments: >> >> WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/00/000/webrev/ >> Only changing the order of methods declaration >> >> WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/00/001/webrev/ >> Renaming initIDs to initProto >> >> WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/00/002/webrev/ >> Changing socketBind >> >> WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/00/003/webrev/ >> Changing socketCreate >> >> WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/00/004/webrev/ >> Changing socketListen >> >> WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/00/005/webrev/ >> Changin socketAvailable >> >> WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/00/006/webrev/ >> Changing socketClose0 >> >> WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/00/007/webrev/ >> Changing socketShutdown >> >> WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/00/008/webrev/ >> Changing socketSendUrgentData >> >> WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/00/009/webrev/ >> Changing socketAccept >> >> WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/00/010/webrev/ >> Changing socketConnect >> >> WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/00/011/webrev/ >> Minor editing, comments, moving code >> >> WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/00/012/webrev/ >> Changing socketSetOption >> >> WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/00/013/webrev/ >> Changing socketGetOption >> >> WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/00/014/webrev/ >> Moving a few methods one more time >> >> >> Accumulative webrev with all the changes above is available here: >> http://cr.openjdk.java.net/~igerasim/8198358/01/webrev/ >> >> >> Thanks in advance! >> >> Ivan >> >> On 3/1/18 8:43 PM, Ivan Gerasimov wrote: >>> Hello! >>> >>> I'd like to do the next step toward removing the TwoStacks socket >>> implementation on Windows. >>> >>> It would be aligning the two implementations (DualStack and >>> TwoStacks), so they can be easier merged together during the next step. >>> >>> There are three PlainSocketImpl implementations in JDK: >>> java.base/windows/classes/java/net/DualStackPlainSocketImpl.java >>> java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java >>> java.base/unix/classes/java/net/PlainSocketImpl.java >>> >>> While two later have very similar organization (in particular, set >>> of native methods), the former is organized slightly differently. >>> In order to merge the two Windows implementation together, they >>> first need to be organized in a similar way. >>> For consistency, DualStack implementation will be reorganized to be >>> aligned with TwoStacks and unix/PlainSocketImpl. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 >>> Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ >>> >>> The change looks somewhat messy, but in fact it was a series of >>> incremental changes, which I still keep in the mercurial 'mq'. >>> >>> (I wish the webrev could be made incremental based on the mq >>> patches, to make it easier to review.) >>> >>> The patched JDK builds fine and all the regression tests pass Okay. >>> >>> >>> Thanks in advance! >> > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Mon Mar 12 16:45:23 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 12 Mar 2018 16:45:23 +0000 Subject: RFE: Add PATCH to the list of the supported HTTP methods In-Reply-To: References: <9ee4df79-ae2c-1ea7-3b45-aee2ad239681@oracle.com> Message-ID: <83904536-fa11-dad2-abc5-9ea785792397@oracle.com> On 07/03/18 08:36, Andrej Golovnin wrote: > Hi Chris, > >> The JDK HTTP Client has: >> `HttpRequest.Builder::method(String method, BodyPublisher publisher)` >> >> ,so it is currently possible to use the `PATCH` method. >> >> Is `PATCH` sufficiently popular to warrant its own self-named >> method in the request builder? > > I don't know how popular is it. We have just started to use it with > one of our services and discovered that we cannot use it with > HttpURLConnection at all. And I thought when we add the support for > PATCH to HttpURLConnection, then it would be nice to have it in HTTP > Client too. When you decided not to add it to HTTP Client, then it is > OK for me too. But I think that HttpURLConnection should be changed. I'm not opposed adding support for PATCH to HttpURLConnection. You will need to add a test, and I think a CSR to add it to list of supported methods. -Chris. From chris.hegarty at oracle.com Wed Mar 14 13:16:30 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 14 Mar 2018 13:16:30 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl Message-ID: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> An odd assertion has been observed in the com.sun.net HTTP Server code, that is currently unexplainable. I'd like to add some additional diagnostics information to the assertion so that it may be more helpful if seen again. --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java @@ -408,7 +408,9 @@ } handle (chan, conn); } else { - assert false; + assert false : String.format("Unexpected non-readable key, where " + + " key's channel:%s, isValid:%b, interestOps:%d, readyOps:%d", + key.channel(), key.isValid(), key.interestOps(), key.readyOps()); } } catch (CancelledKeyException e) { handleException(key, null); -Chris. From Alan.Bateman at oracle.com Wed Mar 14 14:05:13 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2018 14:05:13 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> Message-ID: <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> Looks good. On 14/03/2018 13:16, Chris Hegarty wrote: > An odd assertion has been observed in the com.sun.net > HTTP Server code, that is currently unexplainable. I'd > like to add some additional diagnostics information to > the assertion so that it may be more helpful if seen > again. > > > --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > @@ -408,7 +408,9 @@ > ???????????????????????????????????? } > ???????????????????????????????????? handle (chan, conn); > ???????????????????????????????? } else { > -??????????????????????????????????? assert false; > +??????????????????????????????????? assert false : > String.format("Unexpected non-readable key, where " > +??????????????????????????????????????? + " key's channel:%s, > isValid:%b, interestOps:%d, readyOps:%d", > +??????????????????????????????????????? key.channel(), key.isValid(), > key.interestOps(), key.readyOps()); > ???????????????????????????????? } > ???????????????????????????? } catch (CancelledKeyException e) { > ???????????????????????????????? handleException(key, null); > > -Chris. From Alan.Bateman at oracle.com Wed Mar 14 14:30:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2018 14:30:42 +0000 Subject: 8199329: Remove code that attempts to read bytes after connection reset reported Message-ID: Classic networking has some curious code to deal with connection resets. I needed to dig into ancient history to find the issues and original motivations. When a connection reset is reported (usually ECONNRESET) then further attempts to read from the socket will typically return -1 (EOF). Classic networking papers over this so that further attempts with the socket APIs to read bytes will continue to throw SocketException "connection reset". Furthermore, it allows for cases where the platform can read bytes from the socket buffer after ECONNRESET has been reported. None of the main stream platforms do this and it's hard to imagine anything depending on such unpredictable behavior. I'm running into this odd code as part of cleanup to the read/write code paths and reducing them down to a single blocking call. I would like to remove the legacy/undocumented behavior that attempts to read beyond the connection reset. The code to consistently throw IOException once ECONNRESET is returned is retained as it's possible that code relies on this. The proposed changes are here: ?? http://cr.openjdk.java.net/~alanb/8199329/webrev/ All existing tests pass with these changes. -Alan From daniel.fuchs at oracle.com Wed Mar 14 14:39:10 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 14 Mar 2018 14:39:10 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> Message-ID: Hi Chris, key.interestOps() might throw CancelledKeyException in which case you may trade an AssertionError for a CancelledKeyException when building the assertion message. Maybe some of the other methods you call on key need to be checked as well. best regards, -- daniel On 14/03/2018 14:05, Alan Bateman wrote: > Looks good. > > On 14/03/2018 13:16, Chris Hegarty wrote: >> An odd assertion has been observed in the com.sun.net >> HTTP Server code, that is currently unexplainable. I'd >> like to add some additional diagnostics information to >> the assertion so that it may be more helpful if seen >> again. >> >> >> --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java >> @@ -408,7 +408,9 @@ >> ???????????????????????????????????? } >> ???????????????????????????????????? handle (chan, conn); >> ???????????????????????????????? } else { >> -??????????????????????????????????? assert false; >> +??????????????????????????????????? assert false : >> String.format("Unexpected non-readable key, where " >> +??????????????????????????????????????? + " key's channel:%s, >> isValid:%b, interestOps:%d, readyOps:%d", >> +??????????????????????????????????????? key.channel(), key.isValid(), >> key.interestOps(), key.readyOps()); >> ???????????????????????????????? } >> ???????????????????????????? } catch (CancelledKeyException e) { >> ???????????????????????????????? handleException(key, null); >> >> -Chris. > From david.lloyd at redhat.com Wed Mar 14 14:47:15 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 14 Mar 2018 09:47:15 -0500 Subject: 8199329: Remove code that attempts to read bytes after connection reset reported In-Reply-To: References: Message-ID: +1 from me (non-reviewer). Semantically, it is impossible to distinguish the difference between the OS dropping bytes when it receives an RST compared to Java doing the same thing, so there is no observable behavior change here AFAICT. On Wed, Mar 14, 2018 at 9:30 AM, Alan Bateman wrote: > > Classic networking has some curious code to deal with connection resets. I > needed to dig into ancient history to find the issues and original > motivations. > > When a connection reset is reported (usually ECONNRESET) then further > attempts to read from the socket will typically return -1 (EOF). Classic > networking papers over this so that further attempts with the socket APIs to > read bytes will continue to throw SocketException "connection reset". > Furthermore, it allows for cases where the platform can read bytes from the > socket buffer after ECONNRESET has been reported. None of the main stream > platforms do this and it's hard to imagine anything depending on such > unpredictable behavior. I'm running into this odd code as part of cleanup to > the read/write code paths and reducing them down to a single blocking call. > > I would like to remove the legacy/undocumented behavior that attempts to > read beyond the connection reset. The code to consistently throw IOException > once ECONNRESET is returned is retained as it's possible that code relies on > this. > > The proposed changes are here: > http://cr.openjdk.java.net/~alanb/8199329/webrev/ > > All existing tests pass with these changes. > > -Alan -- - DML From chris.hegarty at oracle.com Wed Mar 14 14:51:38 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 14 Mar 2018 14:51:38 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> Message-ID: <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> On 14/03/18 14:39, Daniel Fuchs wrote: > Hi Chris, > > key.interestOps() might throw CancelledKeyException in > which case you may trade an AssertionError for a > CancelledKeyException when building the assertion > message. > > Maybe some of the other methods you call on key need > to be checked as well. Good catch Daniel. Updated code: --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java @@ -280,6 +280,16 @@ } } + // assertion diagnostic information + private static String opsFromKey(SelectionKey key) { + try { + return String.format("interestOps:%d, readyOps:%d", + key.interestOps(), key.readyOps()); + } catch (CancellationException cce) { + return "key has been cancelled"; + } + } + /* main server listener task */ class Dispatcher implements Runnable { @@ -408,7 +418,9 @@ } handle (chan, conn); } else { - assert false; + assert false : String.format("Unexpected non-readable key," + + " where key's channel:%s, isValid:%b, %s", + key.channel(), key.isValid(), opsFromKey(key)); } } catch (CancelledKeyException e) { handleException(key, null); -Chris. From daniel.fuchs at oracle.com Wed Mar 14 14:56:57 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 14 Mar 2018 14:56:57 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> Message-ID: Looks good Chris! cheers, -- daniel On 14/03/2018 14:51, Chris Hegarty wrote: > On 14/03/18 14:39, Daniel Fuchs wrote: >> Hi Chris, >> >> key.interestOps() might throw CancelledKeyException in >> which case you may trade an AssertionError for a >> CancelledKeyException when building the assertion >> message. >> >> Maybe some of the other methods you call on key need >> to be checked as well. > > Good catch Daniel. Updated code: > > --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > @@ -280,6 +280,16 @@ > ???????? } > ???? } > > +??? // assertion diagnostic information > +??? private static String opsFromKey(SelectionKey key) { > +??????? try { > +??????????? return String.format("interestOps:%d, readyOps:%d", > +???????????????????????????????? key.interestOps(), key.readyOps()); > +??????? } catch (CancellationException cce) { > +??????????? return "key has been cancelled"; > +??????? } > +??? } > + > ???? /* main server listener task */ > > ???? class Dispatcher implements Runnable { > @@ -408,7 +418,9 @@ > ???????????????????????????????????? } > ???????????????????????????????????? handle (chan, conn); > ???????????????????????????????? } else { > -??????????????????????????????????? assert false; > +??????????????????????????????????? assert false : > String.format("Unexpected non-readable key," > +??????????????????????????????????????? + " where key's channel:%s, > isValid:%b, %s", > +??????????????????????????????????????? key.channel(), key.isValid(), > opsFromKey(key)); > ???????????????????????????????? } > ???????????????????????????? } catch (CancelledKeyException e) { > ???????????????????????????????? handleException(key, null); > > > -Chris. From Alan.Bateman at oracle.com Wed Mar 14 14:58:32 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2018 14:58:32 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> Message-ID: We can add a toString() to SelectionKeyImpl if needed as it can access the interest and readys ops without concern for cancel. -Alan On 14/03/2018 14:51, Chris Hegarty wrote: > On 14/03/18 14:39, Daniel Fuchs wrote: >> Hi Chris, >> >> key.interestOps() might throw CancelledKeyException in >> which case you may trade an AssertionError for a >> CancelledKeyException when building the assertion >> message. >> >> Maybe some of the other methods you call on key need >> to be checked as well. > > Good catch Daniel. Updated code: > > --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > @@ -280,6 +280,16 @@ > ???????? } > ???? } > > +??? // assertion diagnostic information > +??? private static String opsFromKey(SelectionKey key) { > +??????? try { > +??????????? return String.format("interestOps:%d, readyOps:%d", > +???????????????????????????????? key.interestOps(), key.readyOps()); > +??????? } catch (CancellationException cce) { > +??????????? return "key has been cancelled"; > +??????? } > +??? } > + > ???? /* main server listener task */ > > ???? class Dispatcher implements Runnable { > @@ -408,7 +418,9 @@ > ???????????????????????????????????? } > ???????????????????????????????????? handle (chan, conn); > ???????????????????????????????? } else { > -??????????????????????????????????? assert false; > +??????????????????????????????????? assert false : > String.format("Unexpected non-readable key," > +??????????????????????????????????????? + " where key's channel:%s, > isValid:%b, %s", > +??????????????????????????????????????? key.channel(), key.isValid(), > opsFromKey(key)); > ???????????????????????????????? } > ???????????????????????????? } catch (CancelledKeyException e) { > ???????????????????????????????? handleException(key, null); > > > -Chris. From claes.redestad at oracle.com Wed Mar 14 15:02:36 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 14 Mar 2018 16:02:36 +0100 Subject: 8199329: Remove code that attempts to read bytes after connection reset reported In-Reply-To: References: Message-ID: <584146a8-c6f4-1ad7-03ae-d22892e59f91@oracle.com> Looks good to me. /Claes On 2018-03-14 15:30, Alan Bateman wrote: > > Classic networking has some curious code to deal with connection > resets. I needed to dig into ancient history to find the issues and > original motivations. > > When a connection reset is reported (usually ECONNRESET) then further > attempts to read from the socket will typically return -1 (EOF). > Classic networking papers over this so that further attempts with the > socket APIs to read bytes will continue to throw SocketException > "connection reset". Furthermore, it allows for cases where the > platform can read bytes from the socket buffer after ECONNRESET has > been reported. None of the main stream platforms do this and it's hard > to imagine anything depending on such unpredictable behavior. I'm > running into this odd code as part of cleanup to the read/write code > paths and reducing them down to a single blocking call. > > I would like to remove the legacy/undocumented behavior that attempts > to read beyond the connection reset. The code to consistently throw > IOException once ECONNRESET is returned is retained as it's possible > that code relies on this. > > The proposed changes are here: > ?? http://cr.openjdk.java.net/~alanb/8199329/webrev/ > > All existing tests pass with these changes. > > -Alan From chris.hegarty at oracle.com Wed Mar 14 15:11:47 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 14 Mar 2018 15:11:47 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> Message-ID: <821fe888-4e66-710d-18dc-d7a19f37270b@oracle.com> On 14/03/18 14:58, Alan Bateman wrote: > We can add a toString() to SelectionKeyImpl if needed as it can access > the interest and readys ops without concern for cancel. Yes, that would be fine either. Since you have an outstanding review that contains changes to SelectionKeyImpl, do you wanna include the change there, and then http will simply invoke toString. Or will I include the toString change here? Either is fine for me. -Chris. From chris.hegarty at oracle.com Wed Mar 14 15:29:22 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 14 Mar 2018 15:29:22 +0000 Subject: 8199329: Remove code that attempts to read bytes after connection reset reported In-Reply-To: References: Message-ID: <6cf16391-9227-c25b-5dbe-8226ba512779@oracle.com> On 14/03/18 14:30, Alan Bateman wrote: > > Classic networking has some curious code to deal with connection resets. > I needed to dig into ancient history to find the issues and original > motivations. > > When a connection reset is reported (usually ECONNRESET) then further > attempts to read from the socket will typically return -1 (EOF). Classic > networking papers over this so that further attempts with the socket > APIs to read bytes will continue to throw SocketException "connection > reset". Furthermore, it allows for cases where the platform can read > bytes from the socket buffer after ECONNRESET has been reported. None of > the main stream platforms do this and it's hard to imagine anything > depending on such unpredictable behavior. I'm running into this odd code > as part of cleanup to the read/write code paths and reducing them down > to a single blocking call. > > I would like to remove the legacy/undocumented behavior that attempts to > read beyond the connection reset. The code to consistently throw > IOException once ECONNRESET is returned is retained as it's possible > that code relies on this. > > The proposed changes are here: > http://cr.openjdk.java.net/~alanb/8199329/webrev/ Looks fine. Curiously, I cannot find a test that checks the read-after-reset-throws-SE, but your changes look good. -Chris. From christoph.langer at sap.com Wed Mar 14 15:42:22 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 14 Mar 2018 15:42:22 +0000 Subject: 8199329: Remove code that attempts to read bytes after connection reset reported In-Reply-To: References: Message-ID: Hi Alan, looks good to me, +1. Best regards Christoph > -----Original Message----- > From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of > Alan Bateman > Sent: Mittwoch, 14. M?rz 2018 15:31 > To: OpenJDK Network Dev list > Subject: 8199329: Remove code that attempts to read bytes after connection > reset reported > > > Classic networking has some curious code to deal with connection resets. > I needed to dig into ancient history to find the issues and original > motivations. > > When a connection reset is reported (usually ECONNRESET) then further > attempts to read from the socket will typically return -1 (EOF). Classic > networking papers over this so that further attempts with the socket > APIs to read bytes will continue to throw SocketException "connection > reset". Furthermore, it allows for cases where the platform can read > bytes from the socket buffer after ECONNRESET has been reported. None of > the main stream platforms do this and it's hard to imagine anything > depending on such unpredictable behavior. I'm running into this odd code > as part of cleanup to the read/write code paths and reducing them down > to a single blocking call. > > I would like to remove the legacy/undocumented behavior that attempts to > read beyond the connection reset. The code to consistently throw > IOException once ECONNRESET is returned is retained as it's possible > that code relies on this. > > The proposed changes are here: > ?? http://cr.openjdk.java.net/~alanb/8199329/webrev/ > > All existing tests pass with these changes. > > -Alan From Alan.Bateman at oracle.com Wed Mar 14 15:48:28 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 14 Mar 2018 15:48:28 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: <821fe888-4e66-710d-18dc-d7a19f37270b@oracle.com> References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> <821fe888-4e66-710d-18dc-d7a19f37270b@oracle.com> Message-ID: On 14/03/2018 15:11, Chris Hegarty wrote: > > On 14/03/18 14:58, Alan Bateman wrote: >> We can add a toString() to SelectionKeyImpl if needed as it can >> access the interest and readys ops without concern for cancel. > > Yes, that would be fine either. Since you have an outstanding > review that contains changes to SelectionKeyImpl, do you wanna > include the change there, and then http will simply invoke > toString. Or will I include the toString change here? Either > is fine for me. I'm happy to include it in JDK-8199611 as that it doing several cleanups in this area. -Alan From Alan.Bateman at oracle.com Thu Mar 15 11:20:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Mar 2018 11:20:06 +0000 Subject: RFR of JDK-8199215: Re-examine getFreePort method in test infrastructure library In-Reply-To: References: <5a52026a-b3a3-8e16-2a85-1c2bd239e266@oracle.com> Message-ID: <067292b0-a167-8877-df46-73a081c99301@oracle.com> On 15/03/2018 08:43, Hamlin Li wrote: > : > > Hi Alan, > > Thank you for reviewing, I have updated the webrev in place. ( cc'ing serviceability-dev and net-dev as these are the other areas that use the getFreePort method in the test library. For context, the patch that we are discussing is: ??? http://cr.openjdk.java.net/~mli/8199215/webrev.00/ ) The new implementation of getFreePort looks good but it no longer throws InterruptedException and so might need some of the usages (esp. in the serviceability tests) to be updated. Also the comment "The function will spin ..." is no longer relevant and can be removed. Moving refusingEndpoint() from the NIO test to Utils looks okay. The "it's much more stable ..." in the method description looks a it inconsistent with the other wording. An alternative is "This method is better choice than getFreePort for tests that need an endpoint that refuses connections". The update to the tests look okay to me. -Alan From huaming.li at oracle.com Fri Mar 16 01:54:48 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 16 Mar 2018 09:54:48 +0800 Subject: RFR of JDK-8199215: Re-examine getFreePort method in test infrastructure library In-Reply-To: <067292b0-a167-8877-df46-73a081c99301@oracle.com> References: <5a52026a-b3a3-8e16-2a85-1c2bd239e266@oracle.com> <067292b0-a167-8877-df46-73a081c99301@oracle.com> Message-ID: On 15/03/2018 7:20 PM, Alan Bateman wrote: > On 15/03/2018 08:43, Hamlin Li wrote: >> : >> >> Hi Alan, >> >> Thank you for reviewing, I have updated the webrev in place. > ( cc'ing serviceability-dev and net-dev as these are the other areas > that use the getFreePort method in the test library. For context, the > patch that we are discussing is: > ??? http://cr.openjdk.java.net/~mli/8199215/webrev.00/ ) > > The new implementation of getFreePort looks good but it no longer > throws InterruptedException and so might need some of the usages (esp. > in the serviceability tests) to be updated. Also the comment "The > function will spin ..." is no longer relevant and can be removed. > > Moving refusingEndpoint() from the NIO test to Utils looks okay. The > "it's much more stable ..." in the method description looks a it > inconsistent with the other wording. An alternative is "This method is > better choice than getFreePort for tests that need an endpoint that > refuses connections". > > The update to the tests look okay to me. Hi Alan, Thank you for detailed reviewing. I have updated the webrev in place. (http://cr.openjdk.java.net/~mli/8199215/webrev.00/) Thank you -Hamlin > > -Alan > From Alan.Bateman at oracle.com Fri Mar 16 08:05:48 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Mar 2018 08:05:48 +0000 Subject: RFR of JDK-8199215: Re-examine getFreePort method in test infrastructure library In-Reply-To: References: <5a52026a-b3a3-8e16-2a85-1c2bd239e266@oracle.com> <067292b0-a167-8877-df46-73a081c99301@oracle.com> Message-ID: <9ef60dff-7d40-f772-2ecf-74eb13e79391@oracle.com> On 16/03/2018 01:54, Hamlin Li wrote: > : > > Hi Alan, > Thank you for detailed reviewing. I have updated the webrev in place. > (http://cr.openjdk.java.net/~mli/8199215/webrev.00/) Looks good, just a minor typo "is better choice" -> "is a better choice". Just to confirm, have you run the serviceability and http client tests to make sure that they compile with this change? -Alan From huaming.li at oracle.com Fri Mar 16 09:00:42 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 16 Mar 2018 17:00:42 +0800 Subject: RFR of JDK-8199215: Re-examine getFreePort method in test infrastructure library In-Reply-To: <9ef60dff-7d40-f772-2ecf-74eb13e79391@oracle.com> References: <5a52026a-b3a3-8e16-2a85-1c2bd239e266@oracle.com> <067292b0-a167-8877-df46-73a081c99301@oracle.com> <9ef60dff-7d40-f772-2ecf-74eb13e79391@oracle.com> Message-ID: <668c3c63-ff56-2e00-dc60-5e85c478fd48@oracle.com> On 16/03/2018 4:05 PM, Alan Bateman wrote: > On 16/03/2018 01:54, Hamlin Li wrote: >> : >> >> Hi Alan, >> Thank you for detailed reviewing. I have updated the webrev in place. >> (http://cr.openjdk.java.net/~mli/8199215/webrev.00/) > Looks good, just a minor typo "is better choice" -> "is a better choice". Hi Alan, Thank you, I will modify it when push. > > Just to confirm, have you run the serviceability and http client tests > to make sure that they compile with this change? Yes, I ran tier1,tier2,tier3 tests(I think it includes httpclient tests), and also specific tests using Utils.getFreePort in svc area. I think I'm good to push? Thank you -Hamlin > > -Alan From Alan.Bateman at oracle.com Fri Mar 16 09:23:13 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 16 Mar 2018 09:23:13 +0000 Subject: RFR of JDK-8199215: Re-examine getFreePort method in test infrastructure library In-Reply-To: <668c3c63-ff56-2e00-dc60-5e85c478fd48@oracle.com> References: <5a52026a-b3a3-8e16-2a85-1c2bd239e266@oracle.com> <067292b0-a167-8877-df46-73a081c99301@oracle.com> <9ef60dff-7d40-f772-2ecf-74eb13e79391@oracle.com> <668c3c63-ff56-2e00-dc60-5e85c478fd48@oracle.com> Message-ID: On 16/03/2018 09:00, Hamlin Li wrote: > : >> >> Just to confirm, have you run the serviceability and http client >> tests to make sure that they compile with this change? > Yes, I ran tier1,tier2,tier3 tests(I think it includes httpclient > tests), and also specific tests using Utils.getFreePort in svc area. > I think I'm good to push? Thanks for confirming, I was just surprised that none of the tests in the other areas needed changes.? So yes, go ahead! -Alan From chris.hegarty at oracle.com Fri Mar 16 12:54:22 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 16 Mar 2018 12:54:22 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> <821fe888-4e66-710d-18dc-d7a19f37270b@oracle.com> Message-ID: On 14/03/18 15:48, Alan Bateman wrote: > On 14/03/2018 15:11, Chris Hegarty wrote: >> ... > I'm happy to include it in JDK-8199611 as that it doing several cleanups > in this area. With JDK-8199611 pushed, including SelectionKey::toString, the changes here reduce to simply: --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java @@ -407,7 +407,7 @@ } handle (chan, conn); } else { - assert false; + assert false : "Unexpected non-readable key:" + key; } } catch (CancelledKeyException e) { handleException(key, null); -Chris. From daniel.fuchs at oracle.com Fri Mar 16 13:02:20 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 16 Mar 2018 13:02:20 +0000 Subject: RFR 8199437 [11]Improve diagnostic system assertion message in com.sun.net.httpserver impl In-Reply-To: References: <8e933539-1707-9b6e-673f-dfdf03c88e34@oracle.com> <47f3e3a6-4b39-3f80-0188-3b208d4c4f90@oracle.com> <48aa38d4-a804-f486-cdd8-c6392528c11a@oracle.com> <821fe888-4e66-710d-18dc-d7a19f37270b@oracle.com> Message-ID: <52396f86-0ee6-859c-524c-894b7fa04595@oracle.com> Looks good Chris! -- daniel On 16/03/2018 12:54, Chris Hegarty wrote: > On 14/03/18 15:48, Alan Bateman wrote: >> On 14/03/2018 15:11, Chris Hegarty wrote: >>> ?... >> I'm happy to include it in JDK-8199611 as that it doing several >> cleanups in this area. > > With JDK-8199611 pushed, including SelectionKey::toString, the > changes here reduce to simply: > > --- a/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > +++ b/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java > @@ -407,7 +407,7 @@ > ???????????????????????????????????? } > ???????????????????????????????????? handle (chan, conn); > ???????????????????????????????? } else { > -??????????????????????????????????? assert false; > +??????????????????????????????????? assert false : "Unexpected > non-readable key:" + key; > ???????????????????????????????? } > ???????????????????????????? } catch (CancelledKeyException e) { > ???????????????????????????????? handleException(key, null); > > -Chris. From chris.hegarty at oracle.com Wed Mar 21 20:33:28 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 21 Mar 2018 20:33:28 +0000 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> Message-ID: <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Hi, This is a request for review for the HTTP Client implementation - JEP 321. The javadoc link has been refreshed, based on feedback received from the CSR [1] ( and a follow on supplementary CSR [2] ): http://cr.openjdk.java.net/~chegar/httpclient/02/javadoc/api/java.net.http/java/net/http/package-summary.html The webrev link has been refreshed to include implementation changes to support the latest API, additional test code coverage and scenarios: http://cr.openjdk.java.net/~chegar/httpclient/02/webrev/ > On 9 Feb 2018, at 14:33, Chris Hegarty wrote: > Development of the JDK HTTP Client, incubated in JDK 9 and re-incubated > in JDK 10, has continued. It will soon be in a position to be proposed > for inclusion in Java 11, through JEP 321. This will proceed, as usual, > through the JEP 2.0 process. > > JEP 321 has recently been updated with the proposed package and module > name, `java.net.http`. The code in the sandbox has been updated to > reflect this. > > Javadoc based on the latest code in the sandbox: > http://cr.openjdk.java.net/~chegar/httpclient/00/javadoc/api/java.net.http-summary.html > > Webrev based on the latest code in the sandbox: > http://cr.openjdk.java.net/~chegar/httpclient/00/webrev/ > > ( much of the changes in the webrev are mechanical due to renaming, but > there are stability fixes and the API documentation has been > reorganized to improve readability ) > > More information about the JDK HTTP Client can be found on the > networking groups page. > http://openjdk.java.net/groups/net/httpclient/ -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8197565 [2] https://bugs.openjdk.java.net/browse/JDK-8199938 From james at lightbend.com Thu Mar 22 00:28:14 2018 From: james at lightbend.com (James Roper) Date: Thu, 22 Mar 2018 11:28:14 +1100 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: Hi Chris, I'm still really keen to see an option provided out of the box for using Reactive Streams with the WebSocket API. Note that I'm not proposing that anything be changed about the existing API, I understand fully that you want to provide a full powered API that offers the full range of WebSocket features with very high performance, and mapping that to the Reactive Streams API goes against that. Rather, I think that there should be a built in helper that allows Reactive Streams to be added on top, with the understanding that such a helper will limit the capabilities and performance when its used - eg, the range of options for closing, range of options for how errors are handled, and perhaps introduce some small performance penalties such as additional allocations and/or buffer copies. The use case here is how well this API will interact with other APIs. WebSockets are rarely used in isolation from other technologies. Most non trivial usages of WebSockets will involve plumbing the WebSocket API to some other source or sink of streaming data - for example, a message broker, a database, perhaps a gRPC stream to another service. The question is, how difficult will it be to connect those two APIs? Will their backpressure mechanisms map to each other easily? Will their failure handling and error handling semantics map to each other easily? How many lines of boilerplate will it take to hook them up? If the WebSocket provides an option to use Reactive Streams, then for any other sources/sink of streamed data that support Reactive Streams, application developers will have an option where the answers are guaranteed to be yes without the developer having to do any mapping themselves, and the lines of code to do that will be one. Regards, James On 22 March 2018 at 07:33, Chris Hegarty wrote: > Hi, > > This is a request for review for the HTTP Client implementation - JEP 321. > > The javadoc link has been refreshed, based on feedback received from > the CSR [1] ( and a follow on supplementary CSR [2] ): > http://cr.openjdk.java.net/~chegar/httpclient/02/javadoc/ > api/java.net.http/java/net/http/package-summary.html > > The webrev link has been refreshed to include implementation changes > to support the latest API, additional test code coverage and scenarios: > http://cr.openjdk.java.net/~chegar/httpclient/02/webrev/ > > > On 9 Feb 2018, at 14:33, Chris Hegarty wrote: > > Development of the JDK HTTP Client, incubated in JDK 9 and re-incubated > > in JDK 10, has continued. It will soon be in a position to be proposed > > for inclusion in Java 11, through JEP 321. This will proceed, as usual, > > through the JEP 2.0 process. > > > > JEP 321 has recently been updated with the proposed package and module > > name, `java.net.http`. The code in the sandbox has been updated to > > reflect this. > > > > Javadoc based on the latest code in the sandbox: > > http://cr.openjdk.java.net/~chegar/httpclient/00/javadoc/ > api/java.net.http-summary.html > > > > Webrev based on the latest code in the sandbox: > > http://cr.openjdk.java.net/~chegar/httpclient/00/webrev/ > > > > ( much of the changes in the webrev are mechanical due to renaming, but > > there are stability fixes and the API documentation has been > > reorganized to improve readability ) > > > > More information about the JDK HTTP Client can be found on the > > networking groups page. > > http://openjdk.java.net/groups/net/httpclient/ > > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8197565 > [2] https://bugs.openjdk.java.net/browse/JDK-8199938 -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Mar 22 12:57:14 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 22 Mar 2018 12:57:14 +0000 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: James, Thanks for your reply. > On 22 Mar 2018, at 00:28, James Roper wrote: > > Hi Chris, > > I'm still really keen to see an option provided out of the box for using Reactive Streams with the WebSocket API. What we are delivering in JEP 321 is mainly an HTTP Client API ( you get a small footprint, high performant, low-level, WebSocket API for free ). It has taken a long time to get to this point in the project, where we are close to requesting that it be Proposed To Target for Java SE 11. > Note that I'm not proposing that anything be changed about the existing API, Ok, good. > I understand fully that you want to provide a full powered API that offers the full range of WebSocket features with very high performance, and mapping that to the Reactive Streams API goes against that. Yes, it does. > Rather, I think that there should be a built in helper that allows Reactive Streams to be added on top, I fully agree that a Reactive Streams WebSocket abstraction should be built on top. Where I disagree is that it needs to be done in Java SE, let alone in JEP 321. > with the understanding that such a helper will limit the capabilities and performance when its used - eg, the range of options for closing, range of options for how errors are handled, and perhaps introduce some small performance penalties such as additional allocations and/or buffer copies. That is a whole set of comprises and decisions that would need to be considered. > The use case here is how well this API will interact with other APIs. WebSockets are rarely used in isolation from other technologies. Most non trivial usages of WebSockets will involve plumbing the WebSocket API to some other source or sink of streaming data - for example, a message broker, a database, perhaps a gRPC stream to another service. I could well image a set of vertical-market specific libraries being built on top of the WebSocket API, and that is how it should be. In this specific area, providing the lowest level API is an enabler for others. > The question is, how difficult will it be to connect those two APIs? I imagine some work may be required, but certainly much less than without the WebSocket API. > Will their backpressure mechanisms map to each other easily? I see no reason that it should not. Are you aware of any? > Will their failure handling and error handling semantics map to each other easily? I see no reason that they should not. In fact, most higher-level APIs provide more coarse grain error handling. > How many lines of boilerplate will it take to hook them up? I guess that depends on the particular library's functionality, will it support/handle/expose partial message delivery, buffering, etc. I don?t see this as boilerplate though. > If the WebSocket provides an option to use Reactive Streams, then for any other sources/sink of streamed data that support Reactive Streams, application developers will have an option where the answers are guaranteed to be yes without the developer having to do any mapping themselves, and the lines of code to do that will be one. Sure, if one has a higher-level library that works with Reactive Streams, and one wants to use WebSocket as a transport, then it would be little work if Java SE provided a Reactive Streams interface to WebSocket. There are also vertical-markets using WebSocket, but not using Reactive Streams. At this point in JEP 321, I do not want to increase the scope of the project to include a Reactive Streams interface for WebSocket. This is something that can, and should, be considered separate to JEP 321. It could follow in a future release if deemed appropriate, or not, but adding it at this point will add risk to JEP 321, which I am not willing to do. -Chris. From wenboz at google.com Thu Mar 22 17:36:51 2018 From: wenboz at google.com (Wenbo Zhu) Date: Thu, 22 Mar 2018 10:36:51 -0700 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: On Thu, Mar 22, 2018 at 5:57 AM, Chris Hegarty wrote: > James, > > Thanks for your reply. > > > On 22 Mar 2018, at 00:28, James Roper wrote: > > > > Hi Chris, > > > > I'm still really keen to see an option provided out of the box for using > Reactive Streams with the WebSocket API. > > What we are delivering in JEP 321 is mainly an HTTP Client API > ( you get a small footprint, high performant, low-level, WebSocket > API for free ). It has taken a long time to get to this point in the > project, where we are close to requesting that it be Proposed > To Target for Java SE 11. > > > Note that I'm not proposing that anything be changed about the existing > API, > > Ok, good. > > > I understand fully that you want to provide a full powered API that > offers the full range of WebSocket features with very high performance, and > mapping that to the Reactive Streams API goes against that. > > Yes, it does. > > > Rather, I think that there should be a built in helper that allows > Reactive Streams to be added on top, > > I fully agree that a Reactive Streams WebSocket abstraction should be > built on top. Where I disagree is that it needs to be done in Java SE, let > alone in JEP 321. > +1 Message-based flow-control is unspecified by the websocket protocol; and exposing byte-based flow-control (via RS) as part of the WS API seems a rather questionable approach. Any useful RS support will have to involve some application-level protocols. > > with the understanding that such a helper will limit the capabilities > and performance when its used - eg, the range of options for closing, range > of options for how errors are handled, and perhaps introduce some small > performance penalties such as additional allocations and/or buffer copies. > > That is a whole set of comprises and decisions that would need to > be considered. > > > The use case here is how well this API will interact with other APIs. > WebSockets are rarely used in isolation from other technologies. Most non > trivial usages of WebSockets will involve plumbing the WebSocket API to > some other source or sink of streaming data - for example, a message > broker, a database, perhaps a gRPC stream to another service. > > I could well image a set of vertical-market specific libraries being built > on top of the WebSocket API, and that is how it should be. In this > specific area, providing the lowest level API is an enabler for others. > > > The question is, how difficult will it be to connect those two APIs? > > I imagine some work may be required, but certainly much less than > without the WebSocket API. > > > Will their backpressure mechanisms map to each other easily? > > I see no reason that it should not. Are you aware of any? > > > Will their failure handling and error handling semantics map to each > other easily? > > I see no reason that they should not. In fact, most higher-level APIs > provide more coarse grain error handling. > > > How many lines of boilerplate will it take to hook them up? > > I guess that depends on the particular library's functionality, will it > support/handle/expose partial message delivery, buffering, etc. I > don?t see this as boilerplate though. > > > If the WebSocket provides an option to use Reactive Streams, then for > any other sources/sink of streamed data that support Reactive Streams, > application developers will have an option where the answers are guaranteed > to be yes without the developer having to do any mapping themselves, and > the lines of code to do that will be one. > > Sure, if one has a higher-level library that works with Reactive Streams, > and one wants to use WebSocket as a transport, then it would be little > work if Java SE provided a Reactive Streams interface to WebSocket. > There are also vertical-markets using WebSocket, but not using Reactive > Streams. > > At this point in JEP 321, I do not want to increase the scope of the > project > to include a Reactive Streams interface for WebSocket. This is something > that can, and should, be considered separate to JEP 321. It could follow > in a future release if deemed appropriate, or not, but adding it at this > point > will add risk to JEP 321, which I am not willing to do. > > -Chris. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simone.bordet at gmail.com Fri Mar 23 08:28:20 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 23 Mar 2018 09:28:20 +0100 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: Hi, On Thu, Mar 22, 2018 at 1:57 PM, Chris Hegarty wrote: >> Rather, I think that there should be a built in helper that allows Reactive Streams to be added on top, > > I fully agree that a Reactive Streams WebSocket abstraction should be > built on top. Where I disagree is that it needs to be done in Java SE, let > alone in JEP 321. Why is that ? There have been no problems in having Flow-based APIs for the HTTP client, so can you expand on why it would be different for WebSocket ? Not that I would like to have Flow-based APIs in WebSocket (it's indifferent for me, as long as the low-level API is there), I am just wondering why the difference. >> with the understanding that such a helper will limit the capabilities and performance when its used - eg, the range of options for closing, range of options for how errors are handled, and perhaps introduce some small performance penalties such as additional allocations and/or buffer copies. > > That is a whole set of comprises and decisions that would need to > be considered. Sure. However, such decisions have been made for the HTTP client. For example, the decision to use Flow-based APIs makes the HTTP client unsuitable to write an efficient HTTP proxy, as it will require data copies. What I see incongruent in JEP 321 is that it provides Flow based APIs for HTTP but not WebSocket, and low-level efficient APIs for WebSocket, but not for HTTP. If the focus is on the HTTP client, I'd rather see efficient APIs for the HTTP client since the HTTP proxy use case is quite common. The design decisions taken for WebSocket (low-level APIs; easily wrappable in Flow APIs; efficient with respect to data copy and allocation; based on CompletableFuture) should have been taken for HTTP with even more importance, given how more popular is HTTP over WebSocket. I would like this revision to be taken into consideration, offering an efficient low-level API for HTTP as well. Thanks ! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From chris.hegarty at oracle.com Fri Mar 23 10:40:37 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 23 Mar 2018 10:40:37 +0000 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: <661bda04-640d-f725-bf92-c7442292d6e5@oracle.com> Simone, Thank you for you comments. On 23/03/18 08:28, Simone Bordet wrote: > Hi, > > On Thu, Mar 22, 2018 at 1:57 PM, Chris Hegarty wrote: >>> Rather, I think that there should be a built in helper that allows Reactive Streams to be added on top, >> >> I fully agree that a Reactive Streams WebSocket abstraction should be >> built on top. Where I disagree is that it needs to be done in Java SE, let >> alone in JEP 321. > > Why is that ? > There have been no problems in having Flow-based APIs for the HTTP > client, so can you expand on why it would be different for WebSocket ? > Not that I would like to have Flow-based APIs in WebSocket (it's > indifferent for me, as long as the low-level API is there), I am just > wondering why the difference. You have been directly involved in the design of the WebSocket API as it stands today ( not based on reactive streams Flow ). Are you now suggesting now that Java SE should provide two programming models for WebSocket? I think one is more than sufficient. >>> with the understanding that such a helper will limit the capabilities and performance when its used - eg, the range of options for closing, range of options for how errors are handled, and perhaps introduce some small performance penalties such as additional allocations and/or buffer copies. >> >> That is a whole set of comprises and decisions that would need to >> be considered. > > Sure. However, such decisions have been made for the HTTP client. > For example, the decision to use Flow-based APIs makes the HTTP client > unsuitable to write an efficient HTTP proxy, as it will require data > copies. Writing an efficient HTTP proxy is not the primary use-case for this API. It is being positioned as a replacement for the legacy HttpURLConnection API. It needs to be approachable by the average ( whatever that is ) Java developer. > What I see incongruent in JEP 321 is that it provides Flow based APIs > for HTTP but not WebSocket, and low-level efficient APIs for > WebSocket, but not for HTTP. I disagree. WebSocket and HTTP are two distinct protocols, whose use will likely be very different. I don't see why it is incongruent to model them in different ways. > If the focus is on the HTTP client, I'd rather see efficient APIs for > the HTTP client since the HTTP proxy use case is quite common. Answered above. > The design decisions taken for WebSocket (low-level APIs; easily > wrappable in Flow APIs; efficient with respect to data copy and > allocation; based on CompletableFuture) should have been taken for > HTTP with even more importance, given how more popular is HTTP over > WebSocket. I disagree. The WebSocket API is less approachable by the average Java developer. It's low-level semantics and use of CompletableFuture, while appropriate for libraries building on top or using it as a transport, is acceptable. It is not something that I would expect the large majority of Java developers ( likely to be retrieving HTTP resources ) to be as comfortable with. The substantive point is that the HTTP Client API and the WebSocket API are targeting two different audiences. > I would like this revision to be taken into consideration, offering an > efficient low-level API for HTTP as well. The HTTP Client API been based on Flow for for a long time now. Incubated in JDK 9, and re-incubated in JDK 10. I note, that you have been following and contributing to JEP 110 and now JEP 321 too ( your contributions have been very valuable, thank you ). Offering a low-level HTTP API ( akin to that of WebSocket ) will make the API less approachable to the majority of Java developers. This is not something that I am comfortable with. -Chris. From chris.hegarty at oracle.com Fri Mar 23 17:18:08 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 23 Mar 2018 17:18:08 +0000 Subject: RFR 8200181 [11] Remove superflous non-IPv4 code from Java_java_net_TwoStacksPlainSocketImpl_socketListen Message-ID: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> Given that JDK-8058965 removed the IPv6 support from TwoStacksPlainSocketImpl, the native socketListen method no longer needs to deal with the non-IPv4 code path. This simplifies the code, brings it in line with the unix variants, and removes a possible problematic code path that could close the socket without notifying the socket cleaner. http://cr.openjdk.java.net/~chegar/8200181.00/ https://bugs.openjdk.java.net/browse/JDK-8200181 -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8058965 From Alan.Bateman at oracle.com Fri Mar 23 17:21:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Mar 2018 17:21:38 +0000 Subject: RFR 8200181 [11] Remove superflous non-IPv4 code from Java_java_net_TwoStacksPlainSocketImpl_socketListen In-Reply-To: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> References: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> Message-ID: On 23/03/2018 17:18, Chris Hegarty wrote: > Given that JDK-8058965 removed the IPv6 support from > TwoStacksPlainSocketImpl, the native socketListen method no longer > needs to deal with the non-IPv4 code path. This simplifies the code, > brings it in line with the unix variants, and removes a possible > problematic code path that could close the socket without notifying > the socket cleaner. > > http://cr.openjdk.java.net/~chegar/8200181.00/ > https://bugs.openjdk.java.net/browse/JDK-8200181 Looks good. From ivan.gerasimov at oracle.com Fri Mar 23 18:38:50 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 23 Mar 2018 11:38:50 -0700 Subject: RFR 8200181 [11] Remove superflous non-IPv4 code from Java_java_net_TwoStacksPlainSocketImpl_socketListen In-Reply-To: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> References: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> Message-ID: Hi Chris! I was about to submit a patch for 8198358 that heavily modifies TwoStacksPlainSocketImpl to make it aligned with DualStackPlainSocketImpl. That patch will also remove this problematic code. I can rebase my patch, but wouldn't it be easier to proceed with that fix? I'm going send the webrev set shortly. With kind regards, Ivan On 3/23/18 10:18 AM, Chris Hegarty wrote: > Given that JDK-8058965 removed the IPv6 support from > TwoStacksPlainSocketImpl, the native socketListen method no longer > needs to deal with the non-IPv4 code path. This simplifies the code, > brings it in line with the unix variants, and removes a possible > problematic code path that could close the socket without notifying > the socket cleaner. > > http://cr.openjdk.java.net/~chegar/8200181.00/ > https://bugs.openjdk.java.net/browse/JDK-8200181 > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8058965 > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Fri Mar 23 18:48:36 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 23 Mar 2018 18:48:36 +0000 Subject: RFR 8200181 [11] Remove superflous non-IPv4 code from Java_java_net_TwoStacksPlainSocketImpl_socketListen In-Reply-To: References: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> Message-ID: Ivan, On 23/03/18 18:38, Ivan Gerasimov wrote: > Hi Chris! > > I was about to submit a patch for 8198358 that heavily modifies > TwoStacksPlainSocketImpl to make it aligned with DualStackPlainSocketImpl. > > That patch will also remove this problematic code. > > I can rebase my patch, but wouldn't it be easier to proceed with that fix? There is an immediate urgency to have this possibly problematic code removed. We're seeing strangle rare intermittent failures that may be caused by it, and possibly the socket cleaner. It is extremely difficult to reproduce, so we want to eliminate this code from the equation. Regarding 8198358, I've been thinking again about this, and I think a better approach may be to just remove TwoStackPlainSocketImpl completely. The preferIPv4Stack can go through DualStackPlainSocketImpl with a indicator to create an AF_INET socket, just like on unix. I think this could be a more straight forward path forward, unless testing shows otherwise. Maybe 8198358 could be amended to do this? Is this something that you would be interested in doing? Otherwise, I can try some experiments to see how it looks. -Chris. > I'm going send the webrev set shortly. > > With kind regards, > > Ivan > > > On 3/23/18 10:18 AM, Chris Hegarty wrote: >> Given that JDK-8058965 removed the IPv6 support from >> TwoStacksPlainSocketImpl, the native socketListen method no longer >> needs to deal with the non-IPv4 code path. This simplifies the code, >> brings it in line with the unix variants, and removes a possible >> problematic code path that could close the socket without notifying >> the socket cleaner. >> >> http://cr.openjdk.java.net/~chegar/8200181.00/ >> https://bugs.openjdk.java.net/browse/JDK-8200181 >> >> -Chris. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8058965 >> > From ivan.gerasimov at oracle.com Fri Mar 23 19:09:49 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 23 Mar 2018 12:09:49 -0700 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: Hello! I reverted the direction of the patch, and now TwoStacks implementation is changed to mimic DualStack. As part of this fix, I also added another @run line to several tests to execute them with the option -Djava.net.preferIPv4Stack=true to make sure both implementations work as expected. After this patch the diff shows difference in only hundred lines of the two implementations, so it will be easy to merge them together on the next step. Here's the set of webrev: WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/02/000/webrev/ remove overridden TwoStacksPlainSocketImpl.close(). The only difference from the base AbstractPlainSocketImpl was that the base first calls socketPreClose() and then socketClose(), and there's no difference is these two on Windows. WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/02/001/webrev/ rename initProto to initIDs WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/02/002/webrev/ changing socketBind WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/02/003/webrev/ changing socketCreate and socketListen WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/02/004/webrev/ changing socketAvailable WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/02/005/webrev/ changing socketClose0 WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/02/006/webrev/ changing socketShutdown WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/02/007/webrev/ changing socketSendUrgentData WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/02/008/webrev/ changing socketAccept WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/02/009/webrev/ changing socketConnect WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/02/010/webrev/ delegating to super.getOption(), overriding socketGetOption WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/02/011/webrev/ changing socketSetOption WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/02/012/webrev/ changing socketGetOption WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/02/013/webrev/ removing unnecessary global variables WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/02/014/webrev/ network tests to run with -Djava.net.preferIPv4Stack=true For reference, here's the combined webrev: http://cr.openjdk.java.net/~igerasim/8198358/03/webrev/ With kind regards, Ivan On 3/9/18 3:50 AM, Chris Hegarty wrote: > Ivan, > > Thanks for breaking up the changes, it makes it easier to review. > > I disagree with changing DualStackPlainSocketImp to conform with > TwoStacks (and unix/PlainSocketImpl). I would prefer to move the > latter to the former. Specifically, around the use of fdAccess. > > -Chris. > > > On 06/03/18 20:31, Ivan Gerasimov wrote: >> In order to make is easier to review the fix, I made the webrev.ksh >> to generate a series of incremental webrevs from the mq patch stack. >> >> Here's the list of the incremental changes with a brief comments: >> >> WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/00/000/webrev/ >> Only changing the order of methods declaration >> >> WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/00/001/webrev/ >> Renaming initIDs to initProto >> >> WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/00/002/webrev/ >> Changing socketBind >> >> WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/00/003/webrev/ >> Changing socketCreate >> >> WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/00/004/webrev/ >> Changing socketListen >> >> WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/00/005/webrev/ >> Changin socketAvailable >> >> WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/00/006/webrev/ >> Changing socketClose0 >> >> WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/00/007/webrev/ >> Changing socketShutdown >> >> WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/00/008/webrev/ >> Changing socketSendUrgentData >> >> WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/00/009/webrev/ >> Changing socketAccept >> >> WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/00/010/webrev/ >> Changing socketConnect >> >> WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/00/011/webrev/ >> Minor editing, comments, moving code >> >> WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/00/012/webrev/ >> Changing socketSetOption >> >> WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/00/013/webrev/ >> Changing socketGetOption >> >> WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/00/014/webrev/ >> Moving a few methods one more time >> >> >> Accumulative webrev with all the changes above is available here: >> http://cr.openjdk.java.net/~igerasim/8198358/01/webrev/ >> >> >> Thanks in advance! >> >> Ivan >> >> On 3/1/18 8:43 PM, Ivan Gerasimov wrote: >>> Hello! >>> >>> I'd like to do the next step toward removing the TwoStacks socket >>> implementation on Windows. >>> >>> It would be aligning the two implementations (DualStack and >>> TwoStacks), so they can be easier merged together during the next step. >>> >>> There are three PlainSocketImpl implementations in JDK: >>> java.base/windows/classes/java/net/DualStackPlainSocketImpl.java >>> java.base/windows/classes/java/net/TwoStacksPlainSocketImpl.java >>> java.base/unix/classes/java/net/PlainSocketImpl.java >>> >>> While two later have very similar organization (in particular, set >>> of native methods), the former is organized slightly differently. >>> In order to merge the two Windows implementation together, they >>> first need to be organized in a similar way. >>> For consistency, DualStack implementation will be reorganized to be >>> aligned with TwoStacks and unix/PlainSocketImpl. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8198358 >>> Webrev: http://cr.openjdk.java.net/~igerasim/8198358/00/webrev/ >>> >>> The change looks somewhat messy, but in fact it was a series of >>> incremental changes, which I still keep in the mercurial 'mq'. >>> >>> (I wish the webrev could be made incremental based on the mq >>> patches, to make it easier to review.) >>> >>> The patched JDK builds fine and all the regression tests pass Okay. >>> >>> >>> Thanks in advance! >> > -- With kind regards, Ivan Gerasimov From ivan.gerasimov at oracle.com Fri Mar 23 19:44:43 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 23 Mar 2018 12:44:43 -0700 Subject: RFR 8200181 [11] Remove superflous non-IPv4 code from Java_java_net_TwoStacksPlainSocketImpl_socketListen In-Reply-To: References: <7884e0b1-2d30-f6e2-9f82-a6c59bba47c0@oracle.com> Message-ID: <40e708c9-b6b0-4da2-d41a-5d638b28e56c@oracle.com> Hi Chris! On 3/23/18 11:48 AM, Chris Hegarty wrote: > Ivan, > > On 23/03/18 18:38, Ivan Gerasimov wrote: >> Hi Chris! >> >> I was about to submit a patch for 8198358 that heavily modifies >> TwoStacksPlainSocketImpl to make it aligned with >> DualStackPlainSocketImpl. >> >> That patch will also remove this problematic code. >> >> I can rebase my patch, but wouldn't it be easier to proceed with that >> fix? > > There is an immediate urgency to have this possibly problematic > code removed. We're seeing strangle rare intermittent failures > that may be caused by it, and possibly the socket cleaner. It > is extremely difficult to reproduce, so we want to eliminate > this code from the equation. > Okay, got it. I can rebase my patch after you push your fix. > Regarding 8198358, I've been thinking again about this, and I > think a better approach may be to just remove > TwoStackPlainSocketImpl completely. The preferIPv4Stack can go > through DualStackPlainSocketImpl with a indicator to create an > AF_INET socket, just like on unix. I think this could be a more > straight forward path forward, unless testing shows otherwise. > Right. That's what I wanted to approach, but in a step-by-step way. Here's the list of other differences that remained after aligning the implementations: - in waitForConnect() I had to add throwing ConnectException if the last error was WSAEADDRNOTAVAIL (there was a regression test failure, when I tried to remove this special case), - in accept(), had to check the new file descriptor for == -2 (in the DualStacks the last error code is checked, which may be wrong), - had to add handling SO_TIMEOUT (in DualStack it's just ignored, but there was a test failure if I tried the same with TwoStacks). I agree that it looks too much work, given we're going to remove TwoStacks anyways, but I found it easier to do it this way to avoid the regressions. With kind regards, Ivan > Maybe 8198358 could be amended to do this? Is this something that > you would be interested in doing? Otherwise, I can try some > experiments to see how it looks. > > -Chris. > >> I'm going send the webrev set shortly. >> >> With kind regards, >> >> Ivan >> >> >> On 3/23/18 10:18 AM, Chris Hegarty wrote: >>> Given that JDK-8058965 removed the IPv6 support from >>> TwoStacksPlainSocketImpl, the native socketListen method no longer >>> needs to deal with the non-IPv4 code path. This simplifies the code, >>> brings it in line with the unix variants, and removes a possible >>> problematic code path that could close the socket without notifying >>> the socket cleaner. >>> >>> http://cr.openjdk.java.net/~chegar/8200181.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8200181 >>> >>> -Chris. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8058965 >>> >> > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Fri Mar 23 21:55:31 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 23 Mar 2018 21:55:31 +0000 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> Ivan, Thank you for persisting with this. > On 23 Mar 2018, at 19:09, Ivan Gerasimov wrote: > > Hello! > > I reverted the direction of the patch, and now TwoStacks implementation is changed to mimic DualStack. > As part of this fix, I also added another @run line to several tests to execute them with the option -Djava.net.preferIPv4Stack=true to make sure both implementations work as expected. > > After this patch the diff shows difference in only hundred lines of the two implementations, so it will be easy to merge them together on the next step. > > For reference, here's the combined webrev: > http://cr.openjdk.java.net/~igerasim/8198358/03/webrev/ I think this is good. And thanks for the additional test coverage, this is great. You asked previously about maybe creating a branch in the sandbox for this work. If you could that would facilitate working on the next step, the changes could then be folded and pushed into the mainline. This has the added benefit of reducing merge costs for the loom project. -Chris. > With kind regards, > Ivan > > On 3/9/18 3:50 AM, Chris Hegarty wrote: >> Ivan, >> >> Thanks for breaking up the changes, it makes it easier to review. >> >> I disagree with changing DualStackPlainSocketImp to conform with >> TwoStacks (and unix/PlainSocketImpl). I would prefer to move the >> latter to the former. Specifically, around the use of fdAccess. >> >> -Chris. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Sun Mar 25 13:37:12 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 25 Mar 2018 14:37:12 +0100 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> Message-ID: <2EABD07F-0E10-4599-9370-9BF3865E9ECB@oracle.com> > On 23 Mar 2018, at 21:55, Chris Hegarty wrote: > >> For reference, here's the combined webrev: >> http://cr.openjdk.java.net/~igerasim/8198358/03/webrev/ > > I think this is good. And thanks for the additional test coverage, this is great. Actually, there is a couple of places in the new native code that has the following pattern: if (sa.sa.sa_family != AF_INET) { JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", "Protocol family not supported"); NET_SocketClose(newfd); return -1; } I would like to remove the NET_SocketClose, and allow the exception propagating at the Java-level be responsible for closing the socket, which will then do so in cooperation with the socket cleaner ( it will effectively cancel the cleaner ). -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.gerasimov at oracle.com Sun Mar 25 18:13:18 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 25 Mar 2018 11:13:18 -0700 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <2EABD07F-0E10-4599-9370-9BF3865E9ECB@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> <2EABD07F-0E10-4599-9370-9BF3865E9ECB@oracle.com> Message-ID: <74adedcd-5b5e-a662-89cb-442e1f692a54@oracle.com> Hi Chris! On 3/25/18 6:37 AM, Chris Hegarty wrote: > >> On 23 Mar 2018, at 21:55, Chris Hegarty > > wrote: >> >>> For reference, here's the combined webrev: >>> http://cr.openjdk.java.net/~igerasim/8198358/03/webrev/ >>> >> >> I think this is good. And thanks for the additional test coverage, >> this is great. > > Actually, there is a couple of places in the new native code that > has the following pattern: > > if (sa.sa.sa_family != AF_INET) { > JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", > "Protocol family not supported"); > NET_SocketClose(newfd); > return -1; > } > > I would like to remove the NET_SocketClose, and allow the > exception propagating at the Java-level be responsible for > closing the socket, which will then do so in cooperation > with the socket cleaner ( it will effectively cancel the cleaner ). > > -Chris. > In the code above, newfd was obtained from a call to accept() a few lines before this check. So, the Java code has no way of being aware of this socket, and it will never be closed unless we do it right here, before returning -1. I checked the other places, where family is checked to be AF_INET and can confirm there's no other places where a socket gets closed. With kind regards, Ivan -- With kind regards, Ivan Gerasimov -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sun Mar 25 19:11:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 25 Mar 2018 20:11:14 +0100 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <74adedcd-5b5e-a662-89cb-442e1f692a54@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> <2EABD07F-0E10-4599-9370-9BF3865E9ECB@oracle.com> <74adedcd-5b5e-a662-89cb-442e1f692a54@oracle.com> Message-ID: <1705b4da-c5c2-ee91-e73a-68777e52740e@oracle.com> On 25/03/2018 19:13, Ivan Gerasimov wrote: > : > > In the code above, newfd was obtained from a call to accept() a few > lines before this check. > So, the Java code has no way of being aware of this socket, and it > will never be closed unless we do it right here, before returning -1. > The SocketImpl's fd is set with this code: ? (*env)->SetIntField(env, socketFdObj, IO_fd_fdID, fd); If there is any possibility of returning without a pending exception it will be registered with the cleaner. At this point, I think we have to treat all code calling NET_SocketClose as a suspect until the regression is tracked down. -Alan From ivan.gerasimov at oracle.com Mon Mar 26 22:08:15 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 26 Mar 2018 15:08:15 -0700 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> Message-ID: <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> Hello everyone! A few modifications were done to the patch. Below is the list of updated parts of the patch with comments about what has changed, comparing to the previous version of the patch. The patched JDK builds fine and all the tests pass well. WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/04/000/webrev/ WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/04/001/webrev/ WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/04/002/webrev/ WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/04/003/webrev/ - Rebased to reflect recent changes to socketListen() WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/04/004/webrev/ WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/04/005/webrev/ WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/04/006/webrev/ WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/04/007/webrev/ WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/04/008/webrev/ - The check the line 400 was changed to test if (newfd < 0) and then if (newfd == -2). I don't think the later is quite accurate (cannot find in the documentation that accept can return -2), but it is how it was done in TwoStacks originally, so no regression is expected. WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/04/009/webrev/ - Added missing 'return -1;' at the line 188. WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/04/010/webrev/ WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/04/011/webrev/ WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/04/012/webrev/ WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/04/013/webrev/ WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/04/014/webrev/ With kind regards, Ivan From ivan.gerasimov at oracle.com Mon Mar 26 22:06:07 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 26 Mar 2018 15:06:07 -0700 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <1705b4da-c5c2-ee91-e73a-68777e52740e@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <440AE865-44F0-4E20-A4A9-C7AE909624DA@oracle.com> <2EABD07F-0E10-4599-9370-9BF3865E9ECB@oracle.com> <74adedcd-5b5e-a662-89cb-442e1f692a54@oracle.com> <1705b4da-c5c2-ee91-e73a-68777e52740e@oracle.com> Message-ID: <97e53de1-7f0f-c7fb-9a8b-d1554180977e@oracle.com> On 3/25/18 12:11 PM, Alan Bateman wrote: > > > On 25/03/2018 19:13, Ivan Gerasimov wrote: >> : >> >> In the code above, newfd was obtained from a call to accept() a few >> lines before this check. >> So, the Java code has no way of being aware of this socket, and it >> will never be closed unless we do it right here, before returning -1. >> > The SocketImpl's fd is set with this code: > (*env)->SetIntField(env, socketFdObj, IO_fd_fdID, fd); > > If there is any possibility of returning without a pending exception > it will be registered with the cleaner. > Okay. In the patched TwoStacksPlainSocketImp.c there is no setting FileDescriptor.fd from JNI left, so that the code will be safer in this aspect. > At this point, I think we have to treat all code calling > NET_SocketClose as a suspect until the regression is tracked down. > > -Alan > -- With kind regards, Ivan Gerasimov From christoph.langer at sap.com Tue Mar 27 08:11:04 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 27 Mar 2018 08:11:04 +0000 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> Message-ID: Hi Ivan, I went through your changes and I really like this work. Nothing to complain about - especially if there are no regressions with the additional test coverage. Best regards Christoph > -----Original Message----- > From: net-dev [mailto:net-dev-bounces at openjdk.java.net] On Behalf Of > Ivan Gerasimov > Sent: Dienstag, 27. M?rz 2018 00:08 > To: Chris Hegarty ; net-dev at openjdk.java.net; > Alan Bateman > Subject: Re: RFR [11] 8198358 : Align organization of > DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] > > Hello everyone! > > A few modifications were done to the patch. > Below is the list of updated parts of the patch with comments about what > has changed, comparing to the previous version of the patch. > > The patched JDK builds fine and all the tests pass well. > > WEBREV-000: > http://cr.openjdk.java.net/~igerasim/8198358/04/000/webrev/ > WEBREV-001: > http://cr.openjdk.java.net/~igerasim/8198358/04/001/webrev/ > WEBREV-002: > http://cr.openjdk.java.net/~igerasim/8198358/04/002/webrev/ > WEBREV-003: > http://cr.openjdk.java.net/~igerasim/8198358/04/003/webrev/ > - Rebased to reflect recent changes to socketListen() > > WEBREV-004: > http://cr.openjdk.java.net/~igerasim/8198358/04/004/webrev/ > WEBREV-005: > http://cr.openjdk.java.net/~igerasim/8198358/04/005/webrev/ > WEBREV-006: > http://cr.openjdk.java.net/~igerasim/8198358/04/006/webrev/ > WEBREV-007: > http://cr.openjdk.java.net/~igerasim/8198358/04/007/webrev/ > WEBREV-008: > http://cr.openjdk.java.net/~igerasim/8198358/04/008/webrev/ > - The check the line 400 was changed to test if (newfd < 0) and then if > (newfd == -2). > I don't think the later is quite accurate (cannot find in the > documentation that accept can return -2), but it is how it was done in > TwoStacks originally, so no regression is expected. > > WEBREV-009: > http://cr.openjdk.java.net/~igerasim/8198358/04/009/webrev/ > - Added missing 'return -1;' at the line 188. > > WEBREV-010: > http://cr.openjdk.java.net/~igerasim/8198358/04/010/webrev/ > WEBREV-011: > http://cr.openjdk.java.net/~igerasim/8198358/04/011/webrev/ > WEBREV-012: > http://cr.openjdk.java.net/~igerasim/8198358/04/012/webrev/ > WEBREV-013: > http://cr.openjdk.java.net/~igerasim/8198358/04/013/webrev/ > WEBREV-014: > http://cr.openjdk.java.net/~igerasim/8198358/04/014/webrev/ > > With kind regards, > Ivan From peter.firmstone at zeus.net.au Tue Mar 27 12:56:39 2018 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Tue, 27 Mar 2018 22:56:39 +1000 Subject: -Djava.security.manager=problems for service providers Message-ID: <5ABA3F87.50804@zeus.net.au> Not sure if this is the right place to mention this. Anyone notice that specifying a custom security manager at jvm start up causes issues with service providers loading? If using the sun PolicyFile implementation, the policy doesn't load due to the provider failure, I have a custom policy implementation that will allow the jvm to run in this state, and other providers are also not loading, such as the logger and JCE. Note that it doesn't occur if the security manager is set programmatically in the main method at start up, only if it's set via command line option. Examples of providers not loading: [java] java.lang.NullPointerException [java] Can't load log handler "java.util.logging.ConsoleHandler" [java] java.lang.NullPointerException [java] java.lang.NullPointerException [java] at java.util.logging.LogManager$5.run(LogManager.java:965) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:958) [java] at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:1578) [java] at java.util.logging.LogManager.access$1500(LogManager.java:145) [java] at java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1667) [java] at java.util.logging.Logger.getHandlers(Logger.java:1777) [java] at java.util.logging.Logger.log(Logger.java:735) [java] at java.util.logging.Logger.doLog(Logger.java:765) [java] at java.util.logging.Logger.log(Logger.java:788) [java] at org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:496) [java] at org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:469) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at org.apache.river.api.security.ConcurrentPolicyFile.readPoliciesNoCheckGuard(ConcurrentPolicyFile.java:468) [java] at org.apache.river.api.security.ConcurrentPolicyFile.readPolicyPermissionGrants(ConcurrentPolicyFile.java:243) [java] at org.apache.river.api.security.ConcurrentPolicyFile.(ConcurrentPolicyFile.java:253) [java] at org.apache.river.api.security.ConcurrentPolicyFile.(ConcurrentPolicyFile.java:226) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:154) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:133) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:137) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:162) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [java] at java.lang.Class.newInstance(Class.java:442) [java] at sun.misc.Launcher.(Launcher.java:93) [java] at sun.misc.Launcher.(Launcher.java:54) [java] at java.lang.ClassLoader.initSystemClassLoader(ClassLoader.java:1451) [java] at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:1436) [java] Error occurred during initialization of VM [java] java.lang.ExceptionInInitializerError [java] at java.util.ResourceBundle.getLoader(ResourceBundle.java:482) [java] at java.util.ResourceBundle.getBundle(ResourceBundle.java:783) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:47) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:44) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at sun.security.util.ResourcesMgr.getString(ResourcesMgr.java:43) [java] at sun.security.provider.PolicyFile.addGrantEntry(PolicyFile.java:888) [java] at sun.security.provider.PolicyFile.init(PolicyFile.java:626) [java] at sun.security.provider.PolicyFile.access$400(PolicyFile.java:258) [java] at sun.security.provider.PolicyFile$3.run(PolicyFile.java:521) [java] at sun.security.provider.PolicyFile$3.run(PolicyFile.java:495) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at sun.security.provider.PolicyFile.initPolicyFile(PolicyFile.java:495) [java] at sun.security.provider.PolicyFile.initPolicyFile(PolicyFile.java:480) [java] at sun.security.provider.PolicyFile.init(PolicyFile.java:439) [java] at sun.security.provider.PolicyFile.(PolicyFile.java:297) [java] at java.security.Policy.getPolicyNoCheck(Policy.java:196) [java] at java.security.Policy.getPolicy(Policy.java:154) [java] at net.jini.security.Security$7.run(Security.java:1054) [java] at net.jini.security.Security$7.run(Security.java:1052) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at net.jini.security.Security.getPolicy(Security.java:1052) [java] at net.jini.security.Security.getContext(Security.java:506) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:140) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:132) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:137) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:160) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [java] at java.lang.Class.newInstance(Class.java:442) [java] at sun.misc.Launcher.(Launcher.java:93) [java] at sun.misc.Launcher.(Launcher.java:54) [java] at java.lang.ClassLoader.initSystemClassLoader(ClassLoader.java:1451) [java] at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:1436) [java] Caused by: java.lang.NullPointerException [java] at java.util.ResourceBundle$RBClassLoader.(ResourceBundle.java:502) [java] at java.util.ResourceBundle.getLoader(ResourceBundle.java:482) [java] at java.util.ResourceBundle.getBundle(ResourceBundle.java:783) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:47) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:44) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at sun.security.util.ResourcesMgr.getString(ResourcesMgr.java:43) [java] at sun.security.provider.PolicyFile.addGrantEntry(PolicyFile.java:888) [java] at sun.security.provider.PolicyFile.init(PolicyFile.java:626) [java] at sun.security.provider.PolicyFile.access$400(PolicyFile.java:258) [java] at sun.security.provider.PolicyFile$3.run(PolicyFile.java:521) [java] at sun.security.provider.PolicyFile$3.run(PolicyFile.java:495) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at sun.security.provider.PolicyFile.initPolicyFile(PolicyFile.java:495) [java] at sun.security.provider.PolicyFile.initPolicyFile(PolicyFile.java:480) [java] at sun.security.provider.PolicyFile.init(PolicyFile.java:439) [java] at sun.security.provider.PolicyFile.(PolicyFile.java:297) [java] at java.security.Policy.getPolicyNoCheck(Policy.java:196) [java] at java.security.Policy.getPolicy(Policy.java:154) [java] at net.jini.security.Security$7.run(Security.java:1054) [java] at net.jini.security.Security$7.run(Security.java:1052) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at net.jini.security.Security.getPolicy(Security.java:1052) [java] at net.jini.security.Security.getContext(Security.java:506) [java] Unexpected exception: [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:140) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:132) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:137) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:160) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [java] at java.lang.Class.newInstance(Class.java:442) [java] at sun.misc.Launcher.(Launcher.java:93) [java] at sun.misc.Launcher.(Launcher.java:54) [java] at java.lang.ClassLoader.initSystemClassLoader(ClassLoader.java:1451) [java] at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:1436) [java] java.lang.ExceptionInInitializerError [java] at javax.crypto.JceSecurityManager.(JceSecurityManager.java:65) [java] at javax.crypto.Cipher.getConfiguredPermission(Cipher.java:2586) [java] at javax.crypto.Cipher.getMaxAllowedKeyLength(Cipher.java:2610) [java] at sun.security.ssl.CipherSuite$BulkCipher.isUnlimited(CipherSuite.java:535) [java] at sun.security.ssl.CipherSuite$BulkCipher.(CipherSuite.java:507) [java] at sun.security.ssl.CipherSuite.(CipherSuite.java:614) [java] at sun.security.ssl.SSLContextImpl.getApplicableCipherSuiteList(SSLContextImpl.java:294) [java] at sun.security.ssl.SSLContextImpl.access$100(SSLContextImpl.java:42) [java] at sun.security.ssl.SSLContextImpl$AbstractTLSContext.(SSLContextImpl.java:425) [java] at java.lang.Class.forName0(Native Method) [java] at java.lang.Class.forName(Class.java:264) [java] at java.security.Provider$Service.getImplClass(Provider.java:1634) [java] at java.security.Provider$Service.newInstance(Provider.java:1592) [java] at sun.security.jca.GetInstance.getInstance(GetInstance.java:236) [java] at sun.security.jca.GetInstance.getInstance(GetInstance.java:164) [java] at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156) [java] at net.jini.jeri.ssl.Utilities.getServerSSLContextInfo(Utilities.java:712) [java] at net.jini.jeri.ssl.Utilities.getSupportedCipherSuites(Utilities.java:284) [java] at net.jini.jeri.ssl.SslEndpointImpl.getConnectionContexts(SslEndpointImpl.java:750) [java] at net.jini.jeri.ssl.SslEndpointImpl.getCallContext(SslEndpointImpl.java:326) [java] at net.jini.jeri.ssl.SslEndpointImpl.newRequest(SslEndpointImpl.java:185) [java] at net.jini.jeri.ssl.SslEndpoint.newRequest(SslEndpoint.java:550) [java] at net.jini.jeri.BasicObjectEndpoint.newCall(BasicObjectEndpoint.java:421) [java] at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:688) [java] at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:571) [java] at com.sun.proxy.$Proxy2.registerGroup(Unknown Source) [java] at org.apache.river.start.SharedActivationGroupDescriptor.create(SharedActivationGroupDescriptor.java:370) [java] at org.apache.river.qa.harness.SharedGroupAdmin.start(SharedGroupAdmin.java:204) [java] at org.apache.river.qa.harness.AdminManager.startService(AdminManager.java:639) [java] at org.apache.river.qa.harness.AdminManager.startService(AdminManager.java:660) [java] at org.apache.river.qa.harness.ActivatableServiceStarterAdmin.getServiceSharedLogDir(ActivatableServiceStarterAdmin.java:388) [java] at org.apache.river.qa.harness.ActivatableServiceStarterAdmin.start(ActivatableServiceStarterAdmin.java:224) [java] at org.apache.river.qa.harness.AdminManager.startService(AdminManager.java:639) [java] at org.apache.river.qa.harness.AdminManager.startService(AdminManager.java:660) [java] at org.apache.river.qa.harness.AdminManager.startLookupService(AdminManager.java:679) [java] at org.apache.river.test.spec.lookupservice.QATestRegistrar.construct(QATestRegistrar.java:458) [java] at org.apache.river.test.spec.lookupservice.test_set00.EvntLeaseExpiration.construct(EvntLeaseExpiration.java:88) [java] at org.apache.river.qa.harness.MasterTest.doTest(MasterTest.java:228) [java] at org.apache.river.qa.harness.MasterTest.access$000(MasterTest.java:48) [java] at org.apache.river.qa.harness.MasterTest$1.run(MasterTest.java:174) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at javax.security.auth.Subject.doAsPrivileged(Subject.java:483) [java] at org.apache.river.qa.harness.MasterTest.doTestWithLogin(MasterTest.java:171) [java] at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:150) [java] Caused by: java.lang.SecurityException: Can not initialize cryptographic mechanism [java] at javax.crypto.JceSecurity.(JceSecurity.java:93) [java] ... 44 more [java] Caused by: java.lang.SecurityException: Cannot locate policy or framework files! [java] at javax.crypto.JceSecurity.setupJurisdictionPolicies(JceSecurity.java:316) [java] at javax.crypto.JceSecurity.access$000(JceSecurity.java:50) [java] at javax.crypto.JceSecurity$1.run(JceSecurity.java:85) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at javax.crypto.JceSecurity.(JceSecurity.java:82) From chris.hegarty at oracle.com Tue Mar 27 13:47:26 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 27 Mar 2018 14:47:26 +0100 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> Message-ID: Ivan, > On 26 Mar 2018, at 23:08, Ivan Gerasimov wrote: > > Hello everyone! > > A few modifications were done to the patch. > Below is the list of updated parts of the patch with comments about what has changed, comparing to the previous version of the patch. > > The patched JDK builds fine and all the tests pass well. > > WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/04/000/webrev/ Ok. > WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/04/001/webrev/ Ok. > WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/04/002/webrev/ Can we make fdAccess final ( and in DualStack too ) static *final* JavaIOFileDescriptorAccess fdAccess JNIEXPORT void JNICALL Java_java_net_TwoStacksPlainSocketImpl_bind0 ... 142 if (IS_NULL(iaObj)) { 143 JNU_ThrowNullPointerException(env, "inet address argument"); 144 return; 145 } 146 147 family = getInetAddress_family(env, iaObj); 148 if (family != java_net_InetAddress_IPv4) { 149 JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", 150 "Protocol family not supported"); 151 return; 152 } The address is already null checked at the java level, so can be removed here, no? Can the family check be hoisted up into Java? I think this will help when it comes to the eventual merge with DualStack. The native code will then align with DualStack?s. I prefer to reduce and align the native layers as much as possible. It?s easier to deal with differences at the Java level. > WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/04/003/webrev/ > - Rebased to reflect recent changes to socketListen() Ok. > WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/04/004/webrev/ Ok. > WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/04/005/webrev/ Ok. > WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/04/006/webrev/ Ok. > WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/04/007/webrev/ Ok. > WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/04/008/webrev/ > - The check the line 400 was changed to test if (newfd < 0) and then if (newfd == -2). > I don't think the later is quite accurate (cannot find in the documentation that accept can return -2), but it is how it was done in TwoStacks originally, so no regression is expected. Ok. 413 if (sa.sa.sa_family != AF_INET) { 414 JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", 415 "Protocol family not supported"); 416 NET_SocketClose(newfd); 417 return -1; 418 } This check could be hoisted up to the Java-level too, no? Again, I think it would be easier to deal with this kinda differences in Java. > WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/04/009/webrev/ > - Added missing 'return -1;' at the line 188. Can the family be checked at the java level. > WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/04/010/webrev/ Ok. > WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/04/011/webrev/ Ok. > WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/04/012/webrev/ Ok. > WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/04/013/webrev/ Ok. > WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/04/014/webrev/ Good. -Chris. From ivan.gerasimov at oracle.com Tue Mar 27 17:30:45 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 27 Mar 2018 10:30:45 -0700 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> Message-ID: Thank you Christoph and Chris for the review! I made the following suggested changes: - fdAccess made private static final in both TwoStacks and DualStack. - removed redundant check IS_NULL(iaObj) at bind0 (indeed, the address was already checked at Java level). For now, I would prefer to keep the checks for family == IPv4 as they are. I totally agree they should be moved to Java, but I'm planning to do this during the TwoStacks-DualStack merge step. I'm also planning to add a regtest to make sure that Inet6Address is rejected when fed to a IPv4-only socked. Are we good to push now? Please let me know, if you'd like me to send the updated webrev set. With kind regards, Ivan On 3/27/18 6:47 AM, Chris Hegarty wrote: > Ivan, > >> On 26 Mar 2018, at 23:08, Ivan Gerasimov wrote: >> >> Hello everyone! >> >> A few modifications were done to the patch. >> Below is the list of updated parts of the patch with comments about what has changed, comparing to the previous version of the patch. >> >> The patched JDK builds fine and all the tests pass well. >> >> WEBREV-000: http://cr.openjdk.java.net/~igerasim/8198358/04/000/webrev/ > Ok. > >> WEBREV-001: http://cr.openjdk.java.net/~igerasim/8198358/04/001/webrev/ > Ok. > >> WEBREV-002: http://cr.openjdk.java.net/~igerasim/8198358/04/002/webrev/ > Can we make fdAccess final ( and in DualStack too ) > static *final* JavaIOFileDescriptorAccess fdAccess > > JNIEXPORT void JNICALL Java_java_net_TwoStacksPlainSocketImpl_bind0 ... > > 142 if (IS_NULL(iaObj)) { > 143 JNU_ThrowNullPointerException(env, "inet address argument"); > 144 return; > 145 } > 146 > 147 family = getInetAddress_family(env, iaObj); > 148 if (family != java_net_InetAddress_IPv4) { > 149 JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", > 150 "Protocol family not supported"); > 151 return; > 152 } > > The address is already null checked at the java level, so can be removed here, no? > > Can the family check be hoisted up into Java? I think this will help when it comes > to the eventual merge with DualStack. > > The native code will then align with DualStack?s. I prefer to reduce and align > the native layers as much as possible. It?s easier to deal with differences at > the Java level. > >> WEBREV-003: http://cr.openjdk.java.net/~igerasim/8198358/04/003/webrev/ >> - Rebased to reflect recent changes to socketListen() > Ok. > >> WEBREV-004: http://cr.openjdk.java.net/~igerasim/8198358/04/004/webrev/ > Ok. > >> WEBREV-005: http://cr.openjdk.java.net/~igerasim/8198358/04/005/webrev/ > Ok. > >> WEBREV-006: http://cr.openjdk.java.net/~igerasim/8198358/04/006/webrev/ > Ok. > >> WEBREV-007: http://cr.openjdk.java.net/~igerasim/8198358/04/007/webrev/ > Ok. > >> WEBREV-008: http://cr.openjdk.java.net/~igerasim/8198358/04/008/webrev/ >> - The check the line 400 was changed to test if (newfd < 0) and then if (newfd == -2). >> I don't think the later is quite accurate (cannot find in the documentation that accept can return -2), but it is how it was done in TwoStacks originally, so no regression is expected. > Ok. > > 413 if (sa.sa.sa_family != AF_INET) { > 414 JNU_ThrowByName(env, JNU_JAVANETPKG "SocketException", > 415 "Protocol family not supported"); > 416 NET_SocketClose(newfd); > 417 return -1; > 418 } > > This check could be hoisted up to the Java-level too, no? Again, I think it > would be easier to deal with this kinda differences in Java. > >> WEBREV-009: http://cr.openjdk.java.net/~igerasim/8198358/04/009/webrev/ >> - Added missing 'return -1;' at the line 188. > Can the family be checked at the java level. > >> WEBREV-010: http://cr.openjdk.java.net/~igerasim/8198358/04/010/webrev/ > Ok. > >> WEBREV-011: http://cr.openjdk.java.net/~igerasim/8198358/04/011/webrev/ > Ok. > >> WEBREV-012: http://cr.openjdk.java.net/~igerasim/8198358/04/012/webrev/ > Ok. > >> WEBREV-013: http://cr.openjdk.java.net/~igerasim/8198358/04/013/webrev/ > Ok. > >> WEBREV-014: http://cr.openjdk.java.net/~igerasim/8198358/04/014/webrev/ > Good. > > -Chris. > > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Tue Mar 27 17:32:02 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 27 Mar 2018 18:32:02 +0100 Subject: RFR [11] 8198358 : Align organization of DualStackPlainSocketImpl with TwoStacksPlainSocketImp [win] In-Reply-To: References: <5AD6F958-9E05-4BF1-BFB3-0567DFFD758C@oracle.com> <5ad8cbb3-94f5-7e15-4b1a-9f450632b309@oracle.com> <988504e4-7d35-678e-cb85-ced1c4fe9cfd@oracle.com> Message-ID: <24B13381-1C6A-45EF-B786-906D7217A4ED@oracle.com> > On 27 Mar 2018, at 18:30, Ivan Gerasimov wrote: > > Thank you Christoph and Chris for the review! > > I made the following suggested changes: > - fdAccess made private static final in both TwoStacks and DualStack. > - removed redundant check IS_NULL(iaObj) at bind0 (indeed, the address was already checked at Java level). > > For now, I would prefer to keep the checks for family == IPv4 as they are. > I totally agree they should be moved to Java, but I'm planning to do this during the TwoStacks-DualStack merge step. > > I'm also planning to add a regtest to make sure that Inet6Address is rejected when fed to a IPv4-only socked. > > Are we good to push now? Yes. -Chris. From james at lightbend.com Wed Mar 28 02:14:41 2018 From: james at lightbend.com (James Roper) Date: Wed, 28 Mar 2018 13:14:41 +1100 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: Hi Chris, Requiring a third party adapter for using Reactive Streams for WebSockets is not the end of the world, so I don't want to push this too hard, we can just agree to disagree. But there were a few small questions in your reply that I'll answer. > Will their backpressure mechanisms map to each other easily? > > I see no reason that it should not. Are you aware of any? > It would take actually implementing an adapter to be fully aware. Note that the Reactive Streams request() mechanism is very strictly specified, particularly around the edge cases. As an example of just one case, what happens if you invoke: request(16); request(Long.MAX_VALUE); This is explicitly allowed by the reactive streams spec, and the expected behavior is as as soon as demand reaches Long.MAX_VALUE, the demand is considered to be infinite. This is also an implementation detail that many implementations get wrong, but that's ok because the Reactive Streams TCK verifies this behavior to ensure they get it right. But it's unspecified in the HTTP client spec (or in the javadocs at least), so will it map nicely? I don't know. Also, the above scenario is very possible - for example, an implementation might consume a certain number of messages, and then decide that it wants to ignore the rest (ignore, as opposed to push back and block, or cancel which would close the socket), and so they just request Long.MAX_VALUE to indicate that. > Will their failure handling and error handling semantics map to each > other easily? > > I see no reason that they should not. In fact, most higher-level APIs > provide more coarse grain error handling. > So for a start, Reactive Streams prohibits any method from throwing an exception - for example, if you invoke request(-1), you're not allowed to throw an IllegalArgumentException, you have to instead pass the exception to onError. But a lot of other methods in the WebSocket spec are allowed to throw IOException. How a Reactive Streams implementation handles these is unspecified, so in order to safely ensure that errors are propagated (and the important thing here is to make sure that the right errors appear in the right place at the right time), an adapter will have to wrap all these methods, and implement the necessary handling to pass the errors to onError, invoke the upstream cancel, and then ensure that after that, no further callbacks are delegated through in either direction, and given that this is a concurrent API with callbacks being invoked by different threads in both directions, that's going to require things like AtomicReference.compareAndSet, etc. Doing it correctly, ensuring there are no race conditions, ensuring that and end developer can 100% predictably know that all errors will be handled in a predictable way, is not at all trivial. -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From james at lightbend.com Wed Mar 28 02:34:16 2018 From: james at lightbend.com (James Roper) Date: Wed, 28 Mar 2018 13:34:16 +1100 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: On 23 March 2018 at 04:36, Wenbo Zhu wrote: > > +1 > > Message-based flow-control is unspecified by the websocket protocol; and > exposing byte-based flow-control (via RS) as part of the WS API seems a > rather questionable approach. > No one is suggesting a byte based flow control mechanism, RS flow control is never byte based (unless you use Publisher/Subscriber, but no one ever does that because it's incredibly inefficient). Not even TCP supports byte based flow control, it's packet based. I'm not sure why you're raising byte based flow control here. The WebSocket protocol does have implied flow control semantics by virtue of the fact that it is layered on top of TCP. The current WebSocket API in JEP 321 offers frame based flow control using an API very similar to RS, which for all intents and purposes, is message based flow control, since I've never actually seen, in the wild, fragmented WebSocket messages be used. Of course, due to the mismatch of TCP packets, OS buffers and WebSocket frames, there is lag in the propagation of flow control while buffers fill up, but it is none the less there. Any useful RS support will have to involve some application-level protocols. > I completely disagree with this statement. As someone who has been involved in/led the implementation of multiple WebSocket clients and servers (including Play Framework, Lagom Framework and Akka HTTP), I can tell you from my experience that after the initial WebSocket handshake, WebSockets can be offered to application developers as just streams of messages using Reactive Streams, without any additional WebSocket protocol exposure, and still be useful for 99% of use cases. It certainly does not have to involve exposing the protocol to the application, most applications care nothing about message fragmentation, they care nothing about close codes or reasons beyond whether the connection was terminated with a success or error, and they care nothing about pings/pongs. Everything that they do care about, Reactive Streams gives a way of expressing. > with the understanding that such a helper will limit the capabilities and >> performance when its used - eg, the range of options for closing, range of >> options for how errors are handled, and perhaps introduce some small >> performance penalties such as additional allocations and/or buffer copies. >> >> That is a whole set of comprises and decisions that would need to >> be considered. >> >> > The use case here is how well this API will interact with other APIs. >> WebSockets are rarely used in isolation from other technologies. Most non >> trivial usages of WebSockets will involve plumbing the WebSocket API to >> some other source or sink of streaming data - for example, a message >> broker, a database, perhaps a gRPC stream to another service. >> >> I could well image a set of vertical-market specific libraries being built >> on top of the WebSocket API, and that is how it should be. In this >> specific area, providing the lowest level API is an enabler for others. >> >> > The question is, how difficult will it be to connect those two APIs? >> >> I imagine some work may be required, but certainly much less than >> without the WebSocket API. >> >> > Will their backpressure mechanisms map to each other easily? >> >> I see no reason that it should not. Are you aware of any? >> >> > Will their failure handling and error handling semantics map to each >> other easily? >> >> I see no reason that they should not. In fact, most higher-level APIs >> provide more coarse grain error handling. >> >> > How many lines of boilerplate will it take to hook them up? >> >> I guess that depends on the particular library's functionality, will it >> support/handle/expose partial message delivery, buffering, etc. I >> don?t see this as boilerplate though. >> >> > If the WebSocket provides an option to use Reactive Streams, then for >> any other sources/sink of streamed data that support Reactive Streams, >> application developers will have an option where the answers are guaranteed >> to be yes without the developer having to do any mapping themselves, and >> the lines of code to do that will be one. >> >> Sure, if one has a higher-level library that works with Reactive Streams, >> and one wants to use WebSocket as a transport, then it would be little >> work if Java SE provided a Reactive Streams interface to WebSocket. >> There are also vertical-markets using WebSocket, but not using Reactive >> Streams. >> >> At this point in JEP 321, I do not want to increase the scope of the >> project >> to include a Reactive Streams interface for WebSocket. This is something >> that can, and should, be considered separate to JEP 321. It could follow >> in a future release if deemed appropriate, or not, but adding it at this >> point >> will add risk to JEP 321, which I am not willing to do. >> >> -Chris. >> >> > -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From wenboz at google.com Wed Mar 28 07:50:46 2018 From: wenboz at google.com (Wenbo Zhu) Date: Wed, 28 Mar 2018 00:50:46 -0700 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: On Tue, Mar 27, 2018 at 7:34 PM, James Roper wrote: > On 23 March 2018 at 04:36, Wenbo Zhu wrote: >> >> +1 >> >> Message-based flow-control is unspecified by the websocket protocol; and >> exposing byte-based flow-control (via RS) as part of the WS API seems a >> rather questionable approach. >> > > No one is suggesting a byte based flow control mechanism, RS flow control > is never byte based (unless you use Publisher/Subscriber, but > no one ever does that because it's incredibly inefficient). Not even TCP > supports byte based flow control, it's packet based. I'm not sure why > you're raising byte based flow control here. > Not sure this is the case - TCP does packet based flow-control. Maybe we are talking about different things. WS protocol as a L7 protocol has no concept of flow-control (back-pressure) and this is why I mentioned it has to be byte-based (or packet-based if such a thing exists). > The WebSocket protocol does have implied flow control semantics by virtue > of the fact that it is layered on top of TCP. The current WebSocket API in > JEP 321 offers frame based flow control using an API very similar to RS, > which for all intents and purposes, is message based flow control, since > I've never actually seen, in the wild, fragmented WebSocket messages be > used. Of course, due to the mismatch of TCP packets, OS buffers and > WebSocket frames, there is lag in the propagation of flow control while > buffers fill up, but it is none the less there. > > Any useful RS support will have to involve some application-level >> protocols. >> > > I completely disagree with this statement. As someone who has been > involved in/led the implementation of multiple WebSocket clients and > servers (including Play Framework, Lagom Framework and Akka HTTP), I can > tell you from my experience that after the initial WebSocket handshake, > WebSockets can be offered to application developers as just streams of > messages using Reactive Streams, without any additional WebSocket protocol > exposure, and still be useful for 99% of use cases. It certainly does not > have to involve exposing the protocol to the application, most applications > care nothing about message fragmentation, they care nothing about close > codes or reasons beyond whether the connection was terminated with a > success or error, and they care nothing about pings/pongs. Everything that > they do care about, Reactive Streams gives a way of expressing. > I don't see the role of RS if the WS wire protocol doesn't support any signaling for the receiver to notify the sender to "slow down", except to respect the basic TCP flow-control. I am reasonably familiar with the IETF WS spec ( https://tools.ietf.org/html/rfc6455) ... and it would be useful if you could point out how exactly flow-control is supposed to work with RS on top of Websockets. Thanks. > > with the understanding that such a helper will limit the capabilities >>> and performance when its used - eg, the range of options for closing, range >>> of options for how errors are handled, and perhaps introduce some small >>> performance penalties such as additional allocations and/or buffer copies. >>> >>> That is a whole set of comprises and decisions that would need to >>> be considered. >>> >>> > The use case here is how well this API will interact with other APIs. >>> WebSockets are rarely used in isolation from other technologies. Most non >>> trivial usages of WebSockets will involve plumbing the WebSocket API to >>> some other source or sink of streaming data - for example, a message >>> broker, a database, perhaps a gRPC stream to another service. >>> >>> I could well image a set of vertical-market specific libraries being >>> built >>> on top of the WebSocket API, and that is how it should be. In this >>> specific area, providing the lowest level API is an enabler for others. >>> >>> > The question is, how difficult will it be to connect those two APIs? >>> >>> I imagine some work may be required, but certainly much less than >>> without the WebSocket API. >>> >>> > Will their backpressure mechanisms map to each other easily? >>> >>> I see no reason that it should not. Are you aware of any? >>> >>> > Will their failure handling and error handling semantics map to each >>> other easily? >>> >>> I see no reason that they should not. In fact, most higher-level APIs >>> provide more coarse grain error handling. >>> >>> > How many lines of boilerplate will it take to hook them up? >>> >>> I guess that depends on the particular library's functionality, will it >>> support/handle/expose partial message delivery, buffering, etc. I >>> don?t see this as boilerplate though. >>> >>> > If the WebSocket provides an option to use Reactive Streams, then for >>> any other sources/sink of streamed data that support Reactive Streams, >>> application developers will have an option where the answers are guaranteed >>> to be yes without the developer having to do any mapping themselves, and >>> the lines of code to do that will be one. >>> >>> Sure, if one has a higher-level library that works with Reactive Streams, >>> and one wants to use WebSocket as a transport, then it would be little >>> work if Java SE provided a Reactive Streams interface to WebSocket. >>> There are also vertical-markets using WebSocket, but not using Reactive >>> Streams. >>> >>> At this point in JEP 321, I do not want to increase the scope of the >>> project >>> to include a Reactive Streams interface for WebSocket. This is something >>> that can, and should, be considered separate to JEP 321. It could follow >>> in a future release if deemed appropriate, or not, but adding it at this >>> point >>> will add risk to JEP 321, which I am not willing to do. >>> >>> -Chris. >>> >>> >> > > > -- > *James Roper* > *Senior Octonaut* > > Lightbend ? Build reactive apps! > Twitter: @jroper > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Wed Mar 28 09:08:38 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 28 Mar 2018 10:08:38 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner Message-ID: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> 8195059 [1] updated the socket implementation to use a cleaner, rather than the previous finalizer implementation. Unfortunately the TwoStacksPlainDatagramSocketImpl on Windows, that uses two native file descriptors, may close one or other when binding, or it may in fact close both sockets and create two completely new ones. This is problematic since the cleaner has been registered with the native file descriptors of the original `fd` and `fd1` when the impl was created. As such, the cleaner may interfere with other resources that happen to re-use the native file descriptor. This area of code is ultimately destined for replacement with the dual stack implementation, but unfortunately we?re not quite at that point yet, so I would like to resolve this issue with a small localised fix that checks the file descriptors before and after the actual bind call, unregistering and re-registering the cleaner as necessary. http://cr.openjdk.java.net/~chegar/8200304/00/ Before this change many net and nio tests have been observed to fail with "java.io.IOException: An operation was attempted on something that is not a socket ?, or ?java.io.IOException: Socket operation on nonsocket?. With this change no such failures are observed in several hundreds of tier2 test runs. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8195059 From Alan.Bateman at oracle.com Wed Mar 28 17:22:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 28 Mar 2018 18:22:59 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> Message-ID: <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> On 28/03/2018 10:08, Chris Hegarty wrote: > 8195059 [1] updated the socket implementation to use a cleaner, > rather than the previous finalizer implementation. Unfortunately the > TwoStacksPlainDatagramSocketImpl on Windows, that uses two > native file descriptors, may close one or other when binding, or it > may in fact close both sockets and create two completely new > ones. This is problematic since the cleaner has been registered > with the native file descriptors of the original `fd` and `fd1` when > the impl was created. As such, the cleaner may interfere with > other resources that happen to re-use the native file descriptor. > > This area of code is ultimately destined for replacement with the > dual stack implementation, but unfortunately we?re not quite at > that point yet, so I would like to resolve this issue with a small > localised fix that checks the file descriptors before and after the > actual bind call, unregistering and re-registering the cleaner as > necessary. > > http://cr.openjdk.java.net/~chegar/8200304/00/ > It's great that this issue has been tracked down. bind0 can try an exception so shouldn't there be a try-finally around this? Also would it be saner to unregister before bind0 and register afresh after it is called so that there is no point where a closed socket is registered with the cleaner? -Alan From Roger.Riggs at Oracle.com Wed Mar 28 18:47:00 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 28 Mar 2018 14:47:00 -0400 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> Message-ID: Hi Chris, TwoStacksPlainDatagramSocketImpl.java:136 could be explicit about the native bind0 changing the state. And I agree with Alan that the calls to updateCleaner (:144) should occur whether or not the native bind0 throws an exception. Thanks, Roger On 3/28/2018 1:22 PM, Alan Bateman wrote: > On 28/03/2018 10:08, Chris Hegarty wrote: >> 8195059 [1] updated the socket implementation to use a cleaner, >> rather than the previous finalizer implementation. Unfortunately the >> TwoStacksPlainDatagramSocketImpl on Windows, that uses two >> native file descriptors, may close one or other when binding, or it >> may in fact close both sockets and create two completely new >> ones. This is problematic since the cleaner has been registered >> with the native file descriptors of the original `fd` and `fd1` when >> the impl was created. As such, the cleaner may interfere with >> other resources that happen to re-use the native file descriptor. >> >> This area of code is ultimately destined for replacement with the >> dual stack implementation, but unfortunately we?re not quite at >> that point yet, so I would like to resolve this issue with a small >> localised fix that checks the file descriptors before and after the >> actual bind call, unregistering and re-registering the cleaner as >> necessary. >> >> http://cr.openjdk.java.net/~chegar/8200304/00/ >> > It's great that this issue has been tracked down. > > bind0 can try an exception so shouldn't there be a try-finally around > this? Also would it be saner to unregister before bind0 and register > afresh after it is called so that there is no point where a closed > socket is registered with the cleaner? > > -Alan From chris.hegarty at oracle.com Wed Mar 28 19:47:53 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 28 Mar 2018 20:47:53 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> Message-ID: <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> > On 28 Mar 2018, at 19:47, Roger Riggs wrote: > > Hi Chris, > > TwoStacksPlainDatagramSocketImpl.java:136 could be explicit about the native bind0 > changing the state. Ok. > And I agree with Alan that the calls to updateCleaner (:144) should occur whether or not > the native bind0 throws an exception. > ... >> >> bind0 can try an exception so shouldn't there be a try-finally around this? Also would it be saner to unregister before bind0 and register afresh after it is called so that there is no point where a closed socket is registered with the cleaner? Right, unconditionally unregistering and re-registering would be easier to follow, and then try-finally would not be required either. I?ll try this, test it, and report back. -Chris. From james at lightbend.com Thu Mar 29 04:04:28 2018 From: james at lightbend.com (James Roper) Date: Thu, 29 Mar 2018 15:04:28 +1100 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: On 28 March 2018 at 18:50, Wenbo Zhu wrote: > >>> Any useful RS support will have to involve some application-level >>> protocols. >>> >> >> I completely disagree with this statement. As someone who has been >> involved in/led the implementation of multiple WebSocket clients and >> servers (including Play Framework, Lagom Framework and Akka HTTP), I can >> tell you from my experience that after the initial WebSocket handshake, >> WebSockets can be offered to application developers as just streams of >> messages using Reactive Streams, without any additional WebSocket protocol >> exposure, and still be useful for 99% of use cases. It certainly does not >> have to involve exposing the protocol to the application, most applications >> care nothing about message fragmentation, they care nothing about close >> codes or reasons beyond whether the connection was terminated with a >> success or error, and they care nothing about pings/pongs. Everything that >> they do care about, Reactive Streams gives a way of expressing. >> > > I don't see the role of RS if the WS wire protocol doesn't support any > signaling for the receiver to notify the sender to "slow down", except to > respect the basic TCP flow-control. > > I am reasonably familiar with the IETF WS spec ( > https://tools.ietf.org/html/rfc6455) ... and it would be useful if you > could point out how exactly flow-control is supposed to work with RS on top > of Websockets. > I'm not sure how familiar you are with Reactive Streams, so let me first start by explaining its backpressure mechanism. RS backpressure works by a Subscriber having a Subscription object. Subscription provides a method request(n: Long). A Publisher may only invoke a Subscribers onNext method as many times as the number that has been supplied to request. So, if request(8) has been invoked, then the Publisher is allowed to invoke onNext 8 times with 8 elements. The outstanding demand is cumulative, so if request(8) is invoked, then after that request(4) has been invoked, then altogether, the publisher is allowed to invoke onNext 12 times. So backpressure is implemented in RS by virtue of a Subscriber letting the outstanding demand reach 0, and not invoking request again until it's ready to receive more elements. When a Publisher reaches this state, where outstanding demand is zero, it will stop consuming data from its source of data. In the case of WebSockets, this will be a TCP socket. Assuming an NIO based implementation, this will in practice mean that it will cancel the selection key for that channels read interest. Now at this point it's worth pointing out, there will be buffers involved along the way, there may be partially decoded WebSocket frames sitting in a buffer, there may be more fully decoded WebSocket frames ready to be emitted but can't because demand is zero. So, there is buffering done here, deregistering interest in reading doesn't result in immediate backpressure propagation. But eventually, the OS receive buffer will fill up, and the OS will then send an ack with a receive window of zero. This will prevent the remote system from sending any more data. Now, how that then impacts the WebSocket endpoint on the remote system depends on that endpoints implementation. If it's using blocking IO, then, once the remote OS's send buffer is full, any calls to send new messages will block. Thus, back pressure will have been propagated to the remote WebSocket endpoint. If it's using non blocking IO with a backpressure mechanism, for example, if it's using Reactive Streams, then the subscriber is going to stop requesting demand. Demand will reach zero, causing the publisher to be unable to send more data. The publisher will then translate this to whatever backpressure mechanism its source of data represents, perhaps its receiving messages from a message queue, it's going to stop consuming those messages. Thus, backpressure will have been propagated. Another case is that it might be using non blocking IO with no backpressure mechanism - this is unfortunately the case for many WebSocket implementations, including web browsers, and the Java EE WebSocket API. In that case, the WebSocket implementation there has two options, buffer, drop, or fail, or a combination of two of those. If it buffers, it will eventually run out of memory. Probably the most common implementation is to buffer up to a point, and then fail. In such a case - back pressure has been propagated, though not gracefully, the result is a failure. But the important thing to realise here is that the side that is receiving messages has been able to use backpressure to protect itself from running out of memory - without a backpressure mechanism, there would have been no tcp backpressure, and it would have had to have accepted data at the rate produced by the remote end, potentially buffering and running out of memory, or having to fail. So, as you can see, WebSockets don't need an explicit backpressure mechanism in the protocol, they can piggy back off TCP. And Reactive Streams can use this to ensure a consumer of a WebSocket stream on one end can push back on the producer at the other end, and have that push back propagated all the way through, albeit with a certain amount of buffering delaying the push back from happening immediately. > Thanks. > > >> > with the understanding that such a helper will limit the capabilities >>>> and performance when its used - eg, the range of options for closing, range >>>> of options for how errors are handled, and perhaps introduce some small >>>> performance penalties such as additional allocations and/or buffer copies. >>>> >>>> That is a whole set of comprises and decisions that would need to >>>> be considered. >>>> >>>> > The use case here is how well this API will interact with other APIs. >>>> WebSockets are rarely used in isolation from other technologies. Most non >>>> trivial usages of WebSockets will involve plumbing the WebSocket API to >>>> some other source or sink of streaming data - for example, a message >>>> broker, a database, perhaps a gRPC stream to another service. >>>> >>>> I could well image a set of vertical-market specific libraries being >>>> built >>>> on top of the WebSocket API, and that is how it should be. In this >>>> specific area, providing the lowest level API is an enabler for others. >>>> >>>> > The question is, how difficult will it be to connect those two APIs? >>>> >>>> I imagine some work may be required, but certainly much less than >>>> without the WebSocket API. >>>> >>>> > Will their backpressure mechanisms map to each other easily? >>>> >>>> I see no reason that it should not. Are you aware of any? >>>> >>>> > Will their failure handling and error handling semantics map to each >>>> other easily? >>>> >>>> I see no reason that they should not. In fact, most higher-level APIs >>>> provide more coarse grain error handling. >>>> >>>> > How many lines of boilerplate will it take to hook them up? >>>> >>>> I guess that depends on the particular library's functionality, will it >>>> support/handle/expose partial message delivery, buffering, etc. I >>>> don?t see this as boilerplate though. >>>> >>>> > If the WebSocket provides an option to use Reactive Streams, then for >>>> any other sources/sink of streamed data that support Reactive Streams, >>>> application developers will have an option where the answers are guaranteed >>>> to be yes without the developer having to do any mapping themselves, and >>>> the lines of code to do that will be one. >>>> >>>> Sure, if one has a higher-level library that works with Reactive >>>> Streams, >>>> and one wants to use WebSocket as a transport, then it would be little >>>> work if Java SE provided a Reactive Streams interface to WebSocket. >>>> There are also vertical-markets using WebSocket, but not using Reactive >>>> Streams. >>>> >>>> At this point in JEP 321, I do not want to increase the scope of the >>>> project >>>> to include a Reactive Streams interface for WebSocket. This is something >>>> that can, and should, be considered separate to JEP 321. It could follow >>>> in a future release if deemed appropriate, or not, but adding it at >>>> this point >>>> will add risk to JEP 321, which I am not willing to do. >>>> >>>> -Chris. >>>> >>>> >>> >> >> >> -- >> *James Roper* >> *Senior Octonaut* >> >> Lightbend ? Build reactive apps! >> Twitter: @jroper >> > > -- *James Roper* *Senior Octonaut* Lightbend ? Build reactive apps! Twitter: @jroper -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Mar 29 10:10:02 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 29 Mar 2018 11:10:02 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> Message-ID: <928D03C8-2DC5-48FC-B920-152AF9761B22@oracle.com> > On 28 Mar 2018, at 20:47, Chris Hegarty wrote: > ... > Right, unconditionally unregistering and re-registering would be easier > to follow, and then try-finally would not be required either. I?ll try this, > test it, and report back. This is much simpler. @Override protected synchronized void bind0(int lport, InetAddress laddr) throws SocketException { + // The native bind0 may close one or both of the underlying file + // descriptors, and even create new sockets, so the safest course of + // action is to unregister the socket cleaners, and register afterwards. + SocketCleanable.unregister(fd); + SocketCleanable.unregister(fd1); + bind0(lport, laddr, exclusiveBind); + + SocketCleanable.register(fd); + SocketCleanable.register(fd1); } protected synchronized void receive(DatagramPacket p) throws IOException { try { webrev: http://cr.openjdk.java.net/~chegar/8200304/01/ -Chris. From Alan.Bateman at oracle.com Thu Mar 29 10:43:00 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 29 Mar 2018 11:43:00 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: <928D03C8-2DC5-48FC-B920-152AF9761B22@oracle.com> References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> <928D03C8-2DC5-48FC-B920-152AF9761B22@oracle.com> Message-ID: On 29/03/2018 11:10, Chris Hegarty wrote: > : > This is much simpler. > > @Override > protected synchronized void bind0(int lport, InetAddress laddr) > throws SocketException > { > + // The native bind0 may close one or both of the underlying file > + // descriptors, and even create new sockets, so the safest course of > + // action is to unregister the socket cleaners, and register afterwards. > + SocketCleanable.unregister(fd); > + SocketCleanable.unregister(fd1); > + > bind0(lport, laddr, exclusiveBind); > + > + SocketCleanable.register(fd); > + SocketCleanable.register(fd1); > } > This is much simpler but are there any cases where bind0 will throw an exception with open sockets? It looks like the !ipv6_supported case might do that when the native bind fails (the ipv6_supported case seems to clean up). -Alan From chris.hegarty at oracle.com Thu Mar 29 14:05:41 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 29 Mar 2018 15:05:41 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> <928D03C8-2DC5-48FC-B920-152AF9761B22@oracle.com> Message-ID: > On 29 Mar 2018, at 11:47, Daniel Fuchs wrote: > ... > I wonder if the class could be marked final? > All these bind0 method are protected, which > raises the question of what would happen if they are > overridden in a subclass. I will make it final. > On 29 Mar 2018, at 11:43, Alan Bateman wrote: > ... > This is much simpler but are there any cases where bind0 will throw an exception with open sockets? It looks like the !ipv6_supported case might do that when the native bind fails (the ipv6_supported case seems to clean up). Good catch. The !ipv6_supported case has been updated to close the socket and set the fd to null. Given that the cleaner is unregistered, and the fd is null ( close will effectively be a no-op ), it should be fine. http://cr.openjdk.java.net/~chegar/8200304/02/ -Chris. From Alan.Bateman at oracle.com Thu Mar 29 14:13:27 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 29 Mar 2018 15:13:27 +0100 Subject: RFR [11] 8200304: TwoStacksPlainDatagramSocketImpl and socket cleaner In-Reply-To: References: <398CED49-4617-44C7-A308-C5E5E91F2AAB@oracle.com> <0450d1c2-6826-40ee-ee3c-6f0eea2f0ef5@oracle.com> <5481D0B5-27EB-4000-BC73-611BA0C53DFA@oracle.com> <928D03C8-2DC5-48FC-B920-152AF9761B22@oracle.com> Message-ID: On 29/03/2018 15:05, Chris Hegarty wrote: > : > Good catch. The !ipv6_supported case has been updated to close the > socket and set the fd to null. Given that the cleaner is unregistered, and > the fd is null ( close will effectively be a no-op ), it should be fine. > > http://cr.openjdk.java.net/~chegar/8200304/02/ > This version looks okay. -Alan From simone.bordet at gmail.com Fri Mar 30 13:34:20 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 30 Mar 2018 15:34:20 +0200 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: Hi, On Wed, Mar 21, 2018 at 9:33 PM, Chris Hegarty wrote: > Hi, > > This is a request for review for the HTTP Client implementation - JEP 321. It is my understanding, and that of the ReactiveStreams (RS) people I spoke with, that both the RS API and the higher level API based on RS (such as those offered by Reactor and RxJava2) are supposed to expose to users only Publishers. This may be counterintuitive at beginning: the typical case of InputStream to read content and OutputStream to write content are both represented with Publishers in the RS experience. The reason behind only offering Publishers is that Subscribers are more difficult to implement by regular developers; Processors even more so, so it is better that this task is left to library implementers such as the JDK, Reactor and RxJava2. It is therefore a surprise that the the HTTP client uses instead Subscriber for the response content. BodyHandler must return a BodySubscriber, which is-a Flow.Subscriber. This will force users that need to write non-trivial response content handling to implement a Subscriber, or most of the times a Processor (to convert the Subscriber back into a Publisher for consumption by other more "regular" RS APIs). It also creates an asymmetry between read and write side, so that for example it is not possible, without writing a processor, to echo a response content as the next request content. Another difficult example is piping the response content to a WebSocket RS API (that would take a Publisher), and so on for basically all RS libraries out there. I would like to suggest to review this: it is definitely possible to have BodyHandler refactored in this way: public Publisher apply(int statusCode, HttpHeaders responseHeaders, Publisher responseContent); Returning a Publisher that performs transformations on the responseContent Publisher is the heart of libraries such as Reactor and RxJava2, so this will be super simple for users of the API. Utility Publishers that currently discard, convert to string, save to file, etc. can be equally trivially implemented as they are now in the JDK. With the occasion, I think that the statusCode parameter and the responseHeaders parameter can be replaced by a single HttpResponse parameter, which would be better for future maintenance in case other information needs to be provided. The simple case that comes to mind is the HTTP version of the response, which may be different from that of the request and indicates server capabilities: this is currently not provided in the BodyHandler.apply() signature. So perhaps: public Publisher apply(HttpResponse response, Publisher responseContent); This change would also simplify the maintenance since BodySubscriber will be removed. My concern is that driving users of the JDK APIs outside of the familiar RS APIs offered by Reactor and RxJava2 where only Publishers are relevant, by forcing them to implement Subscribers and Processors (heck, I even have an IntelliJ warning telling me that I should not do that!), will cause confusion and be potential source of bugs - we all know non-blocking/async programming is hard. Thanks ! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From wenboz at google.com Sat Mar 31 20:05:35 2018 From: wenboz at google.com (Wenbo Zhu) Date: Sat, 31 Mar 2018 13:05:35 -0700 Subject: RFR [11] 8197564: HTTP Client implementation - JEP 321 In-Reply-To: References: <6fc14dd6-8dc1-6073-5a51-872e71321810@oracle.com> <23EEF0A3-99EB-4CD6-93B3-5F0B14C87FA2@oracle.com> Message-ID: Thanks for all the details. I haven't actually authored anything that implements or uses ReactiveStreams but am very interested in this subject [1]. >From my perspective, the HTML5 JS API doesn't expose any notion of flow-control either (although there has been discussion to expose whatwg streams as they are being finalized). I see the value to keep the Java WS API as a low-level API. I have been traveling and won't have time to provide any detailed feedback to this topic, but am happy to review any final conclusion. Thanks! [1] https://github.com/bidiweb/bidiweb-semantics/blob/master/FlowControlApiStyles.md On Wed, Mar 28, 2018 at 9:04 PM, James Roper wrote: > On 28 March 2018 at 18:50, Wenbo Zhu wrote: > >> >>>> Any useful RS support will have to involve some application-level >>>> protocols. >>>> >>> >>> I completely disagree with this statement. As someone who has been >>> involved in/led the implementation of multiple WebSocket clients and >>> servers (including Play Framework, Lagom Framework and Akka HTTP), I can >>> tell you from my experience that after the initial WebSocket handshake, >>> WebSockets can be offered to application developers as just streams of >>> messages using Reactive Streams, without any additional WebSocket protocol >>> exposure, and still be useful for 99% of use cases. It certainly does not >>> have to involve exposing the protocol to the application, most applications >>> care nothing about message fragmentation, they care nothing about close >>> codes or reasons beyond whether the connection was terminated with a >>> success or error, and they care nothing about pings/pongs. Everything that >>> they do care about, Reactive Streams gives a way of expressing. >>> >> >> I don't see the role of RS if the WS wire protocol doesn't support any >> signaling for the receiver to notify the sender to "slow down", except to >> respect the basic TCP flow-control. >> >> I am reasonably familiar with the IETF WS spec ( >> https://tools.ietf.org/html/rfc6455) ... and it would be useful if you >> could point out how exactly flow-control is supposed to work with RS on top >> of Websockets. >> > > I'm not sure how familiar you are with Reactive Streams, so let me first > start by explaining its backpressure mechanism. > > RS backpressure works by a Subscriber having a Subscription object. > Subscription provides a method request(n: Long). A Publisher may only > invoke a Subscribers onNext method as many times as the number that has > been supplied to request. So, if request(8) has been invoked, then the > Publisher is allowed to invoke onNext 8 times with 8 elements. The > outstanding demand is cumulative, so if request(8) is invoked, then after > that request(4) has been invoked, then altogether, the publisher is allowed > to invoke onNext 12 times. So backpressure is implemented in RS by virtue > of a Subscriber letting the outstanding demand reach 0, and not invoking > request again until it's ready to receive more elements. > > When a Publisher reaches this state, where outstanding demand is zero, it > will stop consuming data from its source of data. In the case of > WebSockets, this will be a TCP socket. Assuming an NIO based > implementation, this will in practice mean that it will cancel the > selection key for that channels read interest. Now at this point it's worth > pointing out, there will be buffers involved along the way, there may be > partially decoded WebSocket frames sitting in a buffer, there may be more > fully decoded WebSocket frames ready to be emitted but can't because demand > is zero. So, there is buffering done here, deregistering interest in > reading doesn't result in immediate backpressure propagation. But > eventually, the OS receive buffer will fill up, and the OS will then send > an ack with a receive window of zero. This will prevent the remote system > from sending any more data. > > Now, how that then impacts the WebSocket endpoint on the remote system > depends on that endpoints implementation. If it's using blocking IO, then, > once the remote OS's send buffer is full, any calls to send new messages > will block. Thus, back pressure will have been propagated to the remote > WebSocket endpoint. If it's using non blocking IO with a backpressure > mechanism, for example, if it's using Reactive Streams, then the subscriber > is going to stop requesting demand. Demand will reach zero, causing the > publisher to be unable to send more data. The publisher will then translate > this to whatever backpressure mechanism its source of data represents, > perhaps its receiving messages from a message queue, it's going to stop > consuming those messages. Thus, backpressure will have been propagated. > Another case is that it might be using non blocking IO with no backpressure > mechanism - this is unfortunately the case for many WebSocket > implementations, including web browsers, and the Java EE WebSocket API. In > that case, the WebSocket implementation there has two options, buffer, > drop, or fail, or a combination of two of those. If it buffers, it will > eventually run out of memory. Probably the most common implementation is to > buffer up to a point, and then fail. In such a case - back pressure has > been propagated, though not gracefully, the result is a failure. But the > important thing to realise here is that the side that is receiving messages > has been able to use backpressure to protect itself from running out of > memory - without a backpressure mechanism, there would have been no tcp > backpressure, and it would have had to have accepted data at the rate > produced by the remote end, potentially buffering and running out of > memory, or having to fail. > > So, as you can see, WebSockets don't need an explicit backpressure > mechanism in the protocol, they can piggy back off TCP. And Reactive > Streams can use this to ensure a consumer of a WebSocket stream on one end > can push back on the producer at the other end, and have that push back > propagated all the way through, albeit with a certain amount of buffering > delaying the push back from happening immediately. > > >> Thanks. >> >> >>> > with the understanding that such a helper will limit the capabilities >>>>> and performance when its used - eg, the range of options for closing, range >>>>> of options for how errors are handled, and perhaps introduce some small >>>>> performance penalties such as additional allocations and/or buffer copies. >>>>> >>>>> That is a whole set of comprises and decisions that would need to >>>>> be considered. >>>>> >>>>> > The use case here is how well this API will interact with other >>>>> APIs. WebSockets are rarely used in isolation from other technologies. Most >>>>> non trivial usages of WebSockets will involve plumbing the WebSocket API to >>>>> some other source or sink of streaming data - for example, a message >>>>> broker, a database, perhaps a gRPC stream to another service. >>>>> >>>>> I could well image a set of vertical-market specific libraries being >>>>> built >>>>> on top of the WebSocket API, and that is how it should be. In this >>>>> specific area, providing the lowest level API is an enabler for others. >>>>> >>>>> > The question is, how difficult will it be to connect those two APIs? >>>>> >>>>> I imagine some work may be required, but certainly much less than >>>>> without the WebSocket API. >>>>> >>>>> > Will their backpressure mechanisms map to each other easily? >>>>> >>>>> I see no reason that it should not. Are you aware of any? >>>>> >>>>> > Will their failure handling and error handling semantics map to >>>>> each other easily? >>>>> >>>>> I see no reason that they should not. In fact, most higher-level APIs >>>>> provide more coarse grain error handling. >>>>> >>>>> > How many lines of boilerplate will it take to hook them up? >>>>> >>>>> I guess that depends on the particular library's functionality, will it >>>>> support/handle/expose partial message delivery, buffering, etc. I >>>>> don?t see this as boilerplate though. >>>>> >>>>> > If the WebSocket provides an option to use Reactive Streams, then >>>>> for any other sources/sink of streamed data that support Reactive Streams, >>>>> application developers will have an option where the answers are guaranteed >>>>> to be yes without the developer having to do any mapping themselves, and >>>>> the lines of code to do that will be one. >>>>> >>>>> Sure, if one has a higher-level library that works with Reactive >>>>> Streams, >>>>> and one wants to use WebSocket as a transport, then it would be little >>>>> work if Java SE provided a Reactive Streams interface to WebSocket. >>>>> There are also vertical-markets using WebSocket, but not using Reactive >>>>> Streams. >>>>> >>>>> At this point in JEP 321, I do not want to increase the scope of the >>>>> project >>>>> to include a Reactive Streams interface for WebSocket. This is >>>>> something >>>>> that can, and should, be considered separate to JEP 321. It could >>>>> follow >>>>> in a future release if deemed appropriate, or not, but adding it at >>>>> this point >>>>> will add risk to JEP 321, which I am not willing to do. >>>>> >>>>> -Chris. >>>>> >>>>> >>>> >>> >>> >>> -- >>> *James Roper* >>> *Senior Octonaut* >>> >>> Lightbend ? Build reactive apps! >>> Twitter: @jroper >>> >> >> > > > -- > *James Roper* > *Senior Octonaut* > > Lightbend ? Build reactive apps! > Twitter: @jroper > -------------- next part -------------- An HTML attachment was scrubbed... URL: From turbanoff at gmail.com Sat Mar 31 12:16:34 2018 From: turbanoff at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCi0YPRgNCx0LDQvdC+0LI=?=) Date: Sat, 31 Mar 2018 12:16:34 -0000 Subject: NullPointerException in jdk.incubator.http.internal.hpack.HeaderTable.Table Message-ID: Hello I'm trying to new use new shiny httpclient from JDK10. Code is pretty simple. I use synchronous HTTP call form multiple threads: private final HttpClient httpClient = HttpClient.newBuilder() .executor(Utils.newFixedThreadPoolExecutor(1, "HttpClient")) .build(); private JsonObject useHttpClient(URL url, String params) throws Exception { HttpRequest req = HttpRequest.newBuilder() .uri(url.toURI()) .setHeader("Connection", "keep-alive") .setHeader("Accept-Encoding", "gzip") .timeout(timeout) .POST(HttpRequest.BodyPublisher.fromString(params)) .build(); HttpResponse response = httpClient.send(req, HttpResponse.BodyHandler.asInputStream()); if (response.statusCode() != 200) { throw new IOException("Server returned " + response.statusCode()); } String encoding = response.headers().firstValue("content-encoding").orElse(""); return parseResponseStream(encoding, response.body());} Sometimes I get NPEs: java.lang.NullPointerException: null at jdk.incubator.http.internal.hpack.HeaderTable$Table.remove(HeaderTable.java:455) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.HeaderTable.evictEntry(HeaderTable.java:264) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.HeaderTable.put(HeaderTable.java:233) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.HeaderTable.put(HeaderTable.java:215) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.Decoder.resumeLiteralWithIndexing(Decoder.java:464) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.Decoder.proceed(Decoder.java:268) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.hpack.Decoder.decode(Decoder.java:246) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.Http2Connection.decodeHeaders(Http2Connection.java:471) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.Http2Connection.processFrame(Http2Connection.java:635) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.frame.FramesDecoder.decode(FramesDecoder.java:156) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.Http2Connection$FramesController.processReceivedData(Http2Connection.java:195) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.Http2Connection.asyncReceive(Http2Connection.java:528) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.Http2Connection$Http2TubeSubscriber.processQueue(Http2Connection.java:1054) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147) ~[jdk.incubator.httpclient:?] at jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198) ~[jdk.incubator.httpclient:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?] Are there any problem in my code? What can be reason for this exception? C:\Program Files\Java\jdk-10\bin>java -version java version "10" 2018-03-20Java(TM) SE Runtime Environment 18.3 (build 10+46)Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode) Andrey Turbanov -------------- next part -------------- An HTML attachment was scrubbed... URL: