From nezihyigitbasi at gmail.com Wed Aug 1 09:56:08 2018 From: nezihyigitbasi at gmail.com (nezih yigitbasi) Date: Wed, 1 Aug 2018 02:56:08 -0700 Subject: Priority can be set for HttpClient with HTTP/1.1 Message-ID: Hi, Priority is only defined for version HTTP_2 (and that's clear from the javadoc). However, it's possible to construct an instance of HttpClient with a priority value and with version HTTP_1_1 (tested with Java "9.0.1"). I feel that it will be less error prone for the users if conflicting combinations of arguments weren't allowed. It's also confusing when I read the below code as it doesn't make sense to set the priority for HTTP/1.1. What do you think? HttpClient client = HttpClient.newBuilder() .executor(httpClientExecutor) .priority(32) .version(HTTP_1_1) .build(); Thanks, Nezih -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.x.mcmahon at oracle.com Wed Aug 1 15:01:25 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 01 Aug 2018 16:01:25 +0100 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> Message-ID: <5B61CB45.50504@oracle.com> Simone, Just to clarify a couple of points below: On 31/07/2018, 22:07, Simone Bordet wrote: > Hi, > > On Tue, Jul 31, 2018 at 9:58 PM Chris Hegarty wrote: >> It is unfortunate that this has come up so late the JDK 11 lifecycle, >> but since this has API impact, and is the right thing to do, it is >> certainly worth addressing now ( rather than waiting to do something in >> a future release). >> >> TL;DR provide API support for setting connection and response specific >> timeouts, while continuing to support a single overall request timeout. >> >> The following issue has been file to track this: >> https://bugs.openjdk.java.net/browse/JDK-8208391 >> >> Propose >> ------- >> >> In the `HttpRequest.Builder` class remove the single `timeout` method >> and replace it with a pair of `connectTimeout` and `responseTimeout`. >> The `connectTimeout` method can be used to set a timeout duration that >> is specific to the connection phase. The `responseTimeout` method can be >> used to set a timeout for the actual of the HTTP response. For example, >> >> HttpRequest request = HttpRequest.newBuilder() >> .uri(URI.create("https://foo.com/")) >> .connectTimeout(Duration.ofSeconds(20)) >> .responseTimeout(Duration.ofMinutes(1)) >> .header("Content-Type", "application/json") >> .POST(BodyPublishers.ofFile(Paths.get("file.json"))) >> .build(); >> >> We consider the ability to be able to set a single timeout for the >> overall response to be important, so the `responseTimeout` method can be >> used to set an overall response timeout, in the absence of a more >> specific `connectTimeout`. This means that the newly named >> `responseTimeout` is semantically equivalent to the old `timeout` when >> used without the new `connectTimeout`. > I don't think this is precisely specified. > > AFAIU, connectTimeout is the time to perform DNS lookup plus the time > to establish the TCP connection (SYN+SYN/ACK+ACK), so you may want to > clarify that. What about TLS? Should connectTimeout not include the handshake as well? The definition above seems over specified and might preclude that. > But responseTimeout also is not precisely defined, and assumes that a > response starts after a request, but this is not always the case. > Imagine the case of uploading a huge file to the server on a slow network. > The server may reply with a 500 immediately so the response side is > finished almost immediately, but the request content upload may > continue for minutes after the response is arrived, depending on the > server implementation. > The good case, where the huge upload continues normally and completes > before the response is sent by the server, is difficult to quantify > beforehand and therefore to configure on the request object (as it > depends on the network). > Point being that "responseTimeout" does not convey the fact that in > certain cases 99% of the time can be spent in the request, not in the > response. I can understand that a server might respond early with an error code like 500 before the client has finished sending a very large request body, but I don't see how a server would likely respond early with 200. Surely, the server has to wait for the whole request body to be received successfully, before sending a 200? If that is the case, then I don't see how timing the entire request, separate from the response, adds much value. - Michael From chris.hegarty at oracle.com Wed Aug 1 15:10:38 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 1 Aug 2018 16:10:38 +0100 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> Message-ID: <82C46441-7672-4FB3-8937-6E3553A571D3@oracle.com> Simone, Thanks for you prompt and detailed reply. Comments inline. > On 31 Jul 2018, at 22:07, Simone Bordet wrote: > > ... > I would argue that you would put this method in HttpClient and so it > is HttpClient that needs to be configured with the connectTimeout. Good suggestion. I agree, and this leaves the `timeout` method name space for the request, as you suggest below. Which is more preferable. > I propose to keep "timeout" as a total request+response timeout (and > possibly conversation timeout, your call) on the request builder, and The current HttpRequest `timeout` has gotten us this far quite well. It?s specification is, deliberately, a little vague so as to leave some of the lower level details to an implementation. I think the HttpRequest `timeout`, as specified in JDK 11 today, is pretty good, but could benefit from some clarification. From the point of view of an ?average? developer the request "starts" once send[Async] is invoked, and ends when either send[Async] returns/completes. It?s not quite that simple for streaming subscribers, but that?s close enough for general discussion. These timeouts need to make sense from the perspective of the API, as well as adhering to the preciseness of the terms relating to the underlying protocols. Before trying to draft the javadoc, the semantic we want for this overall timeout is that it should approximately related to the observed time spent in send[Async]. Ignoring connect for now, this means that the timer starts from the initial DNS lookup/TCP connection and ends when the complete request has been sent and the complete response has been received. This seems like the most useful overall timeout semantic. There is a question about whether or not the response body is part of the timeout. It currently is not since the body handler API has the ability to either stream the response body or return it as a higher-level Java Object, e.g String, etc. > to move "connectTimeout" to the HttpClient builder, specifying that it > means DNS lookup plus TCP connection establishment. I agree with adding `connectTimeout` to HttpClient. The low-level details of what constitutes a new underlying connection is somewhat known. I agree that it should take into account such things as DNS lookup, TCP connect, and SSL handshake. These low-level details could be part of javadoc, but not necessarily part of the normative specification. The javadoc can mention such when describing the connect phase. > In case a request needs to open a connection, "timeout" will still be > the total time from when the request is sent to when both request and > response are finished, including the connectTimeout. Agreed. This seems like the most intuitive semantic of these timeouts. Summary of modes: 1) HttpRequest `timeout` only - times both connect ( if any ) and until both the request and the response are fully send and received, respectively. 2) HttpClient `connectTimeout` only - times only the connect phase, if any. Timer stops after connection has been established. 3) HttpClient `connectTimeout` and HttpRequest `timeout` - start connect phase timer at DNS lookup, stop at TCP connect finished or SSL handshake complete. Start request specific timer shortly after connect timer stops, i.e. just before the HTTP request is sent. Stop request specific timer when both the request and the response are fully send and received, respectively. > By having a typically shorter connectTimeout (say 5 seconds) and a > longer total timeout (say 2 minutes), a request will fail in 5 seconds > if HttpClient fails to open a connection to the origin server, and > fail in 2 minutes if it takes too much. Yes, this the exact scenario that the API should support. > Finally, I would consider TLS handshake time as part of the request > data: the cost will be paid by the first request and capped by "total" > timeout (not connectTimeout). > With TLS 1.3 optimizations, it may be just few TLS bytes before the > encrypted request bytes. I don?t have a strong opinion about which timeout bucket TLS handshake falls into, so long as it falls into one or the other. -Chris. > Thanks! > > -- > Simone Bordet > --- > Finally, no matter how good the architecture and design are, > to deliver bug-free software with optimal performance and reliability, > the implementation technique must be flawless. Victoria Livschitz From simone.bordet at gmail.com Wed Aug 1 15:11:07 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 1 Aug 2018 17:11:07 +0200 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: <5B61CB45.50504@oracle.com> References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> <5B61CB45.50504@oracle.com> Message-ID: Hi, On Wed, Aug 1, 2018 at 5:01 PM Michael McMahon wrote: > I can understand that a server might respond early with an error code > like 500 before the client has finished sending a very large request body, > but I don't see how a server would likely respond early with 200. > Surely, the server has to wait for the whole request body to be received > successfully, before sending a 200? Nope. Not one HTTP RFC forbids to reply with an early 200. That may actually be a common case for large uploads paired with polling the server for the request status. And you would be surprised at how many bad server exist out there, and how creative people can be in using HTTP. I would not base the behavior of the JDK HttpClient on false assumptions, especialy that HTTP responses must finish _after_ HTTP requests. It has never been the case since the inception of HTTP. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From simone.bordet at gmail.com Wed Aug 1 19:20:55 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Wed, 1 Aug 2018 21:20:55 +0200 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: <82C46441-7672-4FB3-8937-6E3553A571D3@oracle.com> References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> <82C46441-7672-4FB3-8937-6E3553A571D3@oracle.com> Message-ID: Hi, On Wed, Aug 1, 2018 at 5:10 PM Chris Hegarty wrote: > Before trying to draft the javadoc, the semantic we want for this > overall timeout is that it should approximately related to the observed > time spent in send[Async]. I'm not sure I agree, especially for sendAsync(), which should setup things for sending and then immediately return. > Ignoring connect for now, this means that > the timer starts from the initial DNS lookup/TCP connection Confused. You are not ignoring the connect if you take into account DNS and TCP connection establishment. > and ends > when the complete request has been sent and the complete response > has been received. This seems like the most useful overall timeout > semantic. Indeed. Note that it's indeed request *and* response completion, including data for both. That means if the request takes longer than the response, it'll possible trigger _after_ the complete response has arrived to the client. > There is a question about whether or not the response body is part > of the timeout. It currently is not since the body handler API has the > ability to either stream the response body or return it as a higher-level > Java Object, e.g String, etc. IMHO it should be. Response content is where things may go really slow, as the HTTP response headers are typically delivered in just one TCP frame (for both HTTP1 and HTTP2). > I agree with adding `connectTimeout` to HttpClient. Great. > The low-level details > of what constitutes a new underlying connection is somewhat known. I > agree that it should take into account such things as DNS lookup, TCP > connect, and SSL handshake. As I said, I'm not sure about the latter, but either works for me. > Summary of modes: > > 1) HttpRequest `timeout` only - times both connect ( if any ) and > until both the request and the response are fully send and received, > respectively. Yes, including - for both - their respective content. > 2) HttpClient `connectTimeout` only - times only the connect phase, > if any. Timer stops after connection has been established. Yes. > 3) HttpClient `connectTimeout` and HttpRequest `timeout` - start > connect phase timer at DNS lookup, stop at TCP connect finished > or SSL handshake complete. Start request specific timer shortly > after connect timer stops, i.e. just before the HTTP request is sent. > Stop request specific timer when both the request and the response > are fully send and received, respectively. Seems to me that this contradicts what you said before (total timeout includes the connectTimeout). Given exchange = request+response, then: 1. send request 2. start "total" timer 3a. need to open connection, start "connectTimeout" timer and try to open connection; if connectTimeout elapses, "total" timeout is cancelled, and exchange is failed; if connection is opened, cancel connectTimeout. 3b. connection available, send request. 3c. connections exist, but none is available to send (all busy with other exchanges and cannot open a new connection due to capping of number of connections to same origin), request is queued. 4. "total" timeout elapses, exchange can be: queued, sending or receiving; in all cases fail the exchange. 5. exchange completes in time, cancel "total" timeout. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From chris.hegarty at oracle.com Wed Aug 1 19:49:34 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 1 Aug 2018 20:49:34 +0100 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> <82C46441-7672-4FB3-8937-6E3553A571D3@oracle.com> Message-ID: Simone, > On 1 Aug 2018, at 20:20, Simone Bordet wrote: > > Hi, > > On Wed, Aug 1, 2018 at 5:10 PM Chris Hegarty wrote: >> Before trying to draft the javadoc, the semantic we want for this >> overall timeout is that it should approximately related to the observed >> time spent in send[Async]. > > I'm not sure I agree, especially for sendAsync(), which should setup > things for sending and then immediately return. The CompletableFuture returned by sendAsync must complete exceptionally with an HttpTimeoutException, say if the response is not received, before the timeout elapses. I would expect that if one was to observe the amount of time for the CF to complete it should be approximately equal to the overall timeout. >> Ignoring connect for now, this means that >> the timer starts from the initial DNS lookup/TCP connection > > Confused. You are not ignoring the connect if you take into account > DNS and TCP connection establishment. Sorry, I meant ignoring an explicit `connectTimeout` method for now, then the amount of time spent setting up the connection should be considered part of the overall time spent for the timer. More on this below. >> and ends >> when the complete request has been sent and the complete response >> has been received. This seems like the most useful overall timeout >> semantic. > > Indeed. > Note that it's indeed request *and* response completion, including > data for both. Right. I just wanted to confirm this explicitly ( since the current timeout implementation stops the timer after the response headers are received ). Changing this will take time. > That means if the request takes longer than the response, it'll > possible trigger _after_ the complete response has arrived to the > client. True, at which point there may not be a mechanism available to report the error, beyond cancelling the request body?s publisher, which may be enough in itself to indicate that the whole request body was not sent. >> There is a question about whether or not the response body is part >> of the timeout. It currently is not since the body handler API has the >> ability to either stream the response body or return it as a higher-level >> Java Object, e.g String, etc. > > IMHO it should be. I think so too. I?m concerned though that attempting to change this now could destabilise things. I?m working on a prototype, but we may need a backup evolutionary plan that allows this to be adjusted post JDK 11. > Response content is where things may go really > slow, as the HTTP response headers are typically delivered in just one > TCP frame (for both HTTP1 and HTTP2). Right. >> I agree with adding `connectTimeout` to HttpClient. > > Great. Ok. We have agreement here. >> The low-level details >> of what constitutes a new underlying connection is somewhat known. I >> agree that it should take into account such things as DNS lookup, TCP >> connect, and SSL handshake. > > As I said, I'm not sure about the latter, but either works for me. Ok. >> Summary of modes: >> >> 1) HttpRequest `timeout` only - times both connect ( if any ) and >> until both the request and the response are fully send and received, >> respectively. > > Yes, including - for both - their respective content. Yes. >> 2) HttpClient `connectTimeout` only - times only the connect phase, >> if any. Timer stops after connection has been established. > > Yes. > >> 3) HttpClient `connectTimeout` and HttpRequest `timeout` - start >> connect phase timer at DNS lookup, stop at TCP connect finished >> or SSL handshake complete. Start request specific timer shortly >> after connect timer stops, i.e. just before the HTTP request is sent. >> Stop request specific timer when both the request and the response >> are fully send and received, respectively. > > Seems to me that this contradicts what you said before (total timeout > includes the connectTimeout). What I meant was: Total timeout includes the time to connect if an explicit connect timeout is not specified. If an explicit connect timeout is specified then the connect part is timed by it, and the total timeout only starts after the connection is setup. Anyway, what you outlined below may be more straight forward. > Given exchange = request+response, then: > 1. send request > 2. start "total" timer > 3a. need to open connection, start "connectTimeout" timer and try to > open connection; if connectTimeout elapses, "total" timeout is > cancelled, and exchange is failed; if connection is opened, cancel > connectTimeout. > 3b. connection available, send request. > 3c. connections exist, but none is available to send (all busy with > other exchanges and cannot open a new connection due to capping of > number of connections to same origin), request is queued. > 4. "total" timeout elapses, exchange can be: queued, sending or > receiving; in all cases fail the exchange. > 5. exchange completes in time, cancel "total" timeout. Implementing the connect timeout part of this seems reasonably straight forward, and I have a few prototypes that I hacked on over the weekend. They are stable and well tested ( some work remains on intermediate responses/requests, like authentication and redirection ). What this discussion has uncovered is the, justifiable, desire to time the complete response body. The current timeout mechanism does not do this, and has never done this ( the timer stops once the response headers are received ). I want to prototype changing this to cover the complete response body and see how it goes. -Chris. From xuelei.fan at oracle.com Thu Aug 2 02:27:24 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 1 Aug 2018 19:27:24 -0700 Subject: Code Review Request, JDK-8207009 SSLEngine#closeInbound mentions SSLException when no close_notify is received In-Reply-To: References: <99d6eeaf-f77b-1967-10be-92347273b794@oracle.com> Message-ID: Update: http://cr.openjdk.java.net/~xuelei/8207009/webrev.01/ Integrated the fix for JDK-8208642, "Server initiated TLSv1.2 renegotiation fails if Java client allows TLSv1.3". SSLHandshake.java is updated to use negotiated version so that TLS 1.2 HelloRequest is acceptable in TLS 1.3 client side. Thanks, Xuelei On 7/30/2018 10:24 AM, Xuelei Fan wrote: > > Please let me know your concerns by the end of August 1st, 2018. > > Thanks, > Xuelei > > > On 7/30/2018 9:59 AM, Xuelei Fan wrote: >> Hi, >> >> Please review the update for the TLS 1.3 half-close and >> synchronization implementation: >> ??? http://cr.openjdk.java.net/~xuelei/8207009/webrev.00/ >> >> Unlike TLS 1.2 and prior versions, for TLS 1.3, the close_notify is >> use to close the local write side and peer read side only.? After the >> close_notify get handles, the local read side and peer write side may >> still be open. >> >> In this update, if an application calls >> SSLEngine.closeInbound/Outbound() or SSLSocket.shutdownInput/Output(), >> half-close will be used.? For compatibility, if SSLSocket.close() get >> called, a duplex close will be tried.? In order to support duplex >> close, JDK will use the user_canceled warning alert even the handshake >> complete. >> >> In practice, an application may only close outbound even it is >> intended to close the inbound as well, or close the connection >> completely.? It works for TLS 1.2 and prior versions.? But no more for >> TLS 1.3 because of the close_notify behavior change in the TLS 1.3 >> specification.? The application may be hung and dead-waiting for >> read/close.? It could be solved by closing the inbound explicitly.? In >> order to mitigate the impact, a new System Property is introduced, >> "jdk.tls.acknowledgeCloseNotify" if source code update is not >> available. ??If the System Property is set to "true", if receiving the >> close_notify, a close_notify alert will be responded.? It is a >> countermeasure of the TLS 1.3 half-close issues. >> >> Thanks, >> Xuelei >> >> >> From michael.x.mcmahon at oracle.com Thu Aug 2 08:44:05 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Thu, 02 Aug 2018 09:44:05 +0100 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> Message-ID: <5B62C455.5010109@oracle.com> On 31/07/2018, 22:07, Simone Bordet wrote: .. > Finally, I would consider TLS handshake time as part of the request > data: the cost will be paid by the first request and capped by "total" > timeout (not connectTimeout). With TLS 1.3 optimizations, it may be > just few TLS bytes before the encrypted request bytes. Thanks! On this point, why not time the handshake as part of the connect timeout? Then you would have consistency for all response times regardless of whether a new TLS connection is being set up, or an existing one reused. Granted TLS 1.3 improves this, but there is still overhead, and significantly more for new TLS sessions pre 1.3. Michael. From simone.bordet at gmail.com Thu Aug 2 09:06:52 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Thu, 2 Aug 2018 11:06:52 +0200 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: <5B62C455.5010109@oracle.com> References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> <5B62C455.5010109@oracle.com> Message-ID: Hi, On Thu, Aug 2, 2018 at 10:44 AM Michael McMahon wrote: > On 31/07/2018, 22:07, Simone Bordet wrote: > .. > > Finally, I would consider TLS handshake time as part of the request > > data: the cost will be paid by the first request and capped by "total" > > timeout (not connectTimeout). With TLS 1.3 optimizations, it may be > > just few TLS bytes before the encrypted request bytes. Thanks! > > On this point, why not time the handshake as part of the connect > timeout? Then you would have consistency for all response times > regardless of whether a new TLS connection is being set up, or an > existing one reused. Granted TLS 1.3 improves this, but there is still > overhead, and significantly more for new TLS sessions pre 1.3. Having the TLS handshake as part of the connectTimeout is certainly fine. I guess it'll just be more complicated from the implementation point of view: if you consider TLS handshake as part of the "connect" operation, then you will need to consider part of the "connect" also the HTTP upgrade for WebSocket, and also the preface for HTTP/2, etc. The "connect" operation to me is DNS+TCP, and it's typically easier to design and code the "connect" separately from the layers that run application protocols such as TLS, HTTP, etc. which may be combined in various way. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From chris.hegarty at oracle.com Fri Aug 3 17:33:12 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 3 Aug 2018 18:33:12 +0100 Subject: Connection timeouts in JEP 321 (HTTP Client) In-Reply-To: References: <4CA92584-6723-4887-A73D-D5741558A0A7@oracle.com> <445271EB-E465-420A-A8BA-6CD319F547F9@oracle.com> <82C46441-7672-4FB3-8937-6E3553A571D3@oracle.com> Message-ID: We?ve almost come to agreement on this ( at least as much as is possible without further prototyping ). The point where we reached, which I think is a good solution, leaves connect timeout and response timeout as orthogonal mechanisms. I would like to separate the issues. 1) 8208391 [1] will be used to implement and support a connect timeout as discussed below. I?ll send a separate review request for this. 2) 8208693 [2] will track the issue around the existing timeout and the possibility of extending the current implementation to cover the request+body AND response+body -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8208391 [2]https://bugs.openjdk.java.net/browse/JDK-8208693 > On 1 Aug 2018, at 20:49, Chris Hegarty wrote: > > Simone, > >> On 1 Aug 2018, at 20:20, Simone Bordet wrote: >> >> Hi, >> >> On Wed, Aug 1, 2018 at 5:10 PM Chris Hegarty wrote: >>> Before trying to draft the javadoc, the semantic we want for this >>> overall timeout is that it should approximately related to the observed >>> time spent in send[Async]. >> >> I'm not sure I agree, especially for sendAsync(), which should setup >> things for sending and then immediately return. > > The CompletableFuture returned by sendAsync must complete > exceptionally with an HttpTimeoutException, say if the response is > not received, before the timeout elapses. I would expect that if one > was to observe the amount of time for the CF to complete it should > be approximately equal to the overall timeout. > >>> Ignoring connect for now, this means that >>> the timer starts from the initial DNS lookup/TCP connection >> >> Confused. You are not ignoring the connect if you take into account >> DNS and TCP connection establishment. > > Sorry, I meant ignoring an explicit `connectTimeout` method for now, > then the amount of time spent setting up the connection should be > considered part of the overall time spent for the timer. More on this > below. > >>> and ends >>> when the complete request has been sent and the complete response >>> has been received. This seems like the most useful overall timeout >>> semantic. >> >> Indeed. >> Note that it's indeed request *and* response completion, including >> data for both. > > Right. I just wanted to confirm this explicitly ( since the current timeout > implementation stops the timer after the response headers are > received ). Changing this will take time. > >> That means if the request takes longer than the response, it'll >> possible trigger _after_ the complete response has arrived to the >> client. > > True, at which point there may not be a mechanism available to > report the error, beyond cancelling the request body?s publisher, > which may be enough in itself to indicate that the whole request > body was not sent. > >>> There is a question about whether or not the response body is part >>> of the timeout. It currently is not since the body handler API has the >>> ability to either stream the response body or return it as a higher-level >>> Java Object, e.g String, etc. >> >> IMHO it should be. > > I think so too. I?m concerned though that attempting to change this > now could destabilise things. I?m working on a prototype, but we may > need a backup evolutionary plan that allows this to be adjusted post > JDK 11. > >> Response content is where things may go really >> slow, as the HTTP response headers are typically delivered in just one >> TCP frame (for both HTTP1 and HTTP2). > > Right. > >>> I agree with adding `connectTimeout` to HttpClient. >> >> Great. > > Ok. We have agreement here. > >>> The low-level details >>> of what constitutes a new underlying connection is somewhat known. I >>> agree that it should take into account such things as DNS lookup, TCP >>> connect, and SSL handshake. >> >> As I said, I'm not sure about the latter, but either works for me. > > Ok. > >>> Summary of modes: >>> >>> 1) HttpRequest `timeout` only - times both connect ( if any ) and >>> until both the request and the response are fully send and received, >>> respectively. >> >> Yes, including - for both - their respective content. > > Yes. > >>> 2) HttpClient `connectTimeout` only - times only the connect phase, >>> if any. Timer stops after connection has been established. >> >> Yes. >> >>> 3) HttpClient `connectTimeout` and HttpRequest `timeout` - start >>> connect phase timer at DNS lookup, stop at TCP connect finished >>> or SSL handshake complete. Start request specific timer shortly >>> after connect timer stops, i.e. just before the HTTP request is sent. >>> Stop request specific timer when both the request and the response >>> are fully send and received, respectively. >> >> Seems to me that this contradicts what you said before (total timeout >> includes the connectTimeout). > > What I meant was: Total timeout includes the time to connect if an explicit > connect timeout is not specified. If an explicit connect timeout is specified > then the connect part is timed by it, and the total timeout only starts after > the connection is setup. Anyway, what you outlined below may be more > straight forward. > >> Given exchange = request+response, then: >> 1. send request >> 2. start "total" timer >> 3a. need to open connection, start "connectTimeout" timer and try to >> open connection; if connectTimeout elapses, "total" timeout is >> cancelled, and exchange is failed; if connection is opened, cancel >> connectTimeout. >> 3b. connection available, send request. >> 3c. connections exist, but none is available to send (all busy with >> other exchanges and cannot open a new connection due to capping of >> number of connections to same origin), request is queued. >> 4. "total" timeout elapses, exchange can be: queued, sending or >> receiving; in all cases fail the exchange. >> 5. exchange completes in time, cancel "total" timeout. > > Implementing the connect timeout part of this seems reasonably straight > forward, and I have a few prototypes that I hacked on over the weekend. > They are stable and well tested ( some work remains on intermediate > responses/requests, like authentication and redirection ). > > What this discussion has uncovered is the, justifiable, desire to time the > complete response body. The current timeout mechanism does not do > this, and has never done this ( the timer stops once the response headers > are received ). I want to prototype changing this to cover the complete > response body and see how it goes. > > -Chris. > From xuelei.fan at oracle.com Fri Aug 3 20:55:10 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 3 Aug 2018 13:55:10 -0700 Subject: Code Review Request, JDK-8207009 SSLEngine#closeInbound mentions SSLException when no close_notify is received In-Reply-To: References: <99d6eeaf-f77b-1967-10be-92347273b794@oracle.com> Message-ID: Update: http://cr.openjdk.java.net/~xuelei/8207009/webrev.02/ In webrev.01, the socket close may be blocked by super class close synchronization. Updated the SSLSocketImpl.java to use handshake only lock in the startHandshake() implementation. Thanks, Xuelei On 8/1/2018 7:27 PM, Xuelei Fan wrote: > Update: http://cr.openjdk.java.net/~xuelei/8207009/webrev.01/ > > Integrated the fix for JDK-8208642, "Server initiated TLSv1.2 > renegotiation fails if Java client allows TLSv1.3".? SSLHandshake.java > is updated to use negotiated version so that TLS 1.2 HelloRequest is > acceptable in TLS 1.3 client side. > > Thanks, > Xuelei > > On 7/30/2018 10:24 AM, Xuelei Fan wrote: >> >> Please let me know your concerns by the end of August 1st, 2018. >> >> Thanks, >> Xuelei >> >> >> On 7/30/2018 9:59 AM, Xuelei Fan wrote: >>> Hi, >>> >>> Please review the update for the TLS 1.3 half-close and >>> synchronization implementation: >>> ??? http://cr.openjdk.java.net/~xuelei/8207009/webrev.00/ >>> >>> Unlike TLS 1.2 and prior versions, for TLS 1.3, the close_notify is >>> use to close the local write side and peer read side only.? After the >>> close_notify get handles, the local read side and peer write side may >>> still be open. >>> >>> In this update, if an application calls >>> SSLEngine.closeInbound/Outbound() or >>> SSLSocket.shutdownInput/Output(), half-close will be used.? For >>> compatibility, if SSLSocket.close() get called, a duplex close will >>> be tried.? In order to support duplex close, JDK will use the >>> user_canceled warning alert even the handshake complete. >>> >>> In practice, an application may only close outbound even it is >>> intended to close the inbound as well, or close the connection >>> completely.? It works for TLS 1.2 and prior versions.? But no more >>> for TLS 1.3 because of the close_notify behavior change in the TLS >>> 1.3 specification.? The application may be hung and dead-waiting for >>> read/close.? It could be solved by closing the inbound explicitly. >>> In order to mitigate the impact, a new System Property is introduced, >>> "jdk.tls.acknowledgeCloseNotify" if source code update is not >>> available. ??If the System Property is set to "true", if receiving >>> the close_notify, a close_notify alert will be responded.? It is a >>> countermeasure of the TLS 1.3 half-close issues. >>> >>> Thanks, >>> Xuelei >>> >>> >>> From chris.hegarty at oracle.com Sat Aug 4 11:45:04 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sat, 4 Aug 2018 12:45:04 +0100 Subject: RFR [11] 8208391: Need to differentiate response and connect timeouts in HTTP Client API Message-ID: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> Hi, This is a code review to add connection specific timeout support to the new HTTP Client, as discussed recently over on another thread [1]. The connection timeout duration is proposed to be added at the level of the client, and if specified applies to all requests sent by that client. The connect timeout mechanism is orthogonal to the HttpRequest `timeout`, but can be used to complement it. For example, HttpClient client = HttpClient.newBuilder() .connectTimeout(Duration.ofSeconds(20)) .build(); HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("https://foo.com/")) .timeout(Duration.ofMinutes(2)) .header("Content-Type", "application/json") .POST(BodyPublishers.ofFile(Paths.get("file.json"))) .build(); client.sendAsync(request, BodyHandlers.ofString()) sends the request with a 20 second connect timeout and a response timeout of 2 minutes. The connect timeout, as implemented in the webrev below, covers the DNS lookup, the TCP SYN handshake, and the TLS handshake. The proposed spec wording deliberately avoids adding such low-level details. Outline of the specification changes: To HttpClient.Builder add: /** * Sets the connect timeout duration for this client. * *

In the case where a new connection needs to be established, if * the connection cannot be established within the given {@code * duration}, then an {@link HttpConnectTimeoutException} is thrown * from {@link HttpClient#send(java.net.http.HttpRequest, * java.net.http.HttpResponse.BodyHandler) HttpClient::send}, or * {@link HttpClient#sendAsync(java.net.http.HttpRequest, * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} * completes exceptionally with an {@code HttpConnectTimeoutException}. * If a new connection does not need to be established, for example * if a connection can be reused from a previous request, then this * timeout duration has no effect. * * @param duration the duration to allow the underlying connection to be * established * @return this builder * @throws IllegalArgumentException if the duration is non-positive */ public Builder connectTimeout(Duration duration); To HttpClient add an accessor: /** * Returns an {@code Optional} containing the connect timeout duration * for this client. If the {@linkplain Builder#connectTimeout(Duration) * connect timeout duration} was not set in the client's builder, then the * {@code Optional} is empty. * * @return an {@code Optional} containing this client's connect timeout * duration */ public abstract Optional connectTimeout(); Add new Exception type: /** * Thrown when a connection, over which an {@code HttpRequest} is intended to be * sent, is not successfully established within a specified time period. */ public class HttpConnectTimeoutException extends HttpTimeoutException { ... } Webrev: http://cr.openjdk.java.net/~chegar/8208391/webrev.00/ -Chris [1] http://mail.openjdk.java.net/pipermail/net-dev/2018-August/011683.html From jai.forums2013 at gmail.com Sat Aug 4 13:08:57 2018 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Sat, 4 Aug 2018 18:38:57 +0530 Subject: RFR [11] 8208391: Need to differentiate response and connect timeouts in HTTP Client API In-Reply-To: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> References: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> Message-ID: <92a69903-683b-1685-eb57-41d679a11aa1@gmail.com> Hi Chris, src/java.net.http/share/classes/java/net/http/HttpClient.java +???????? *

In the case where a new connection needs to be established, if +???????? * the connection cannot be established within the given {@code +???????? * duration}, then an {@link HttpConnectTimeoutException} is thrown +???????? * from {@link HttpClient#send(java.net.http.HttpRequest, +???????? * java.net.http.HttpResponse.BodyHandler) HttpClient::send}, or +???????? * {@link HttpClient#sendAsync(java.net.http.HttpRequest, +???????? * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} +???????? * completes exceptionally with an {@code HttpConnectTimeoutException}. Do you think this can be reworded a bit? Although I understand what's being said here, the wording doesn't seem right. Maybe something like: ???????? ???????? *

In the case where a new connection needs to be established, if ???????? * the connection cannot be established within the given {@code ???????? * duration}, then {@link HttpClient#send(java.net.http.HttpRequest, ???????? * java.net.http.HttpResponse.BodyHandler) HttpClient::send} throws a ???????? * {@link HttpConnectTimeoutException}, or ???????? * {@link HttpClient#sendAsync(java.net.http.HttpRequest, ???????? * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} ???????? * completes exceptionally with an {@code HttpConnectTimeoutException}. src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java -??????? this.timeout = null; +??????? this.timeout = Duration.ofSeconds(30); Is this an intentional change of default value for the timeout? Is that something that needs to be documented? One other thing - maybe not directly related to this single patch, but as you are aware, recently as part of [1], a new system property (and a security property) was introduced to optionally include host info in the exception messages thrown for socket exceptions. Does the HttpClient honour that system property in the exceptions it throws? I don't see it being used in this patch for the timeout exceptions. I haven't checked the code, outside of this patch, to see if it's dealt with in other parts of this API. [1] https://bugs.openjdk.java.net/browse/JDK-8204233 -Jaikiran On 04/08/18 5:15 PM, Chris Hegarty wrote: > Hi, This is a code review to add connection specific timeout support > to the new HTTP Client, as discussed recently over on another thread > [1]. The connection timeout duration is proposed to be added at the > level of the client, and if specified applies to all requests sent by > that client. The connect timeout mechanism is orthogonal to the > HttpRequest `timeout`, but can be used to complement it. For example, > HttpClient client = HttpClient.newBuilder() > .connectTimeout(Duration.ofSeconds(20)) .build(); HttpRequest request > = HttpRequest.newBuilder() .uri(URI.create("https://foo.com/")) > .timeout(Duration.ofMinutes(2)) .header("Content-Type", > "application/json") > .POST(BodyPublishers.ofFile(Paths.get("file.json"))) .build(); > client.sendAsync(request, BodyHandlers.ofString()) sends the request > with a 20 second connect timeout and a response timeout of 2 minutes. > The connect timeout, as implemented in the webrev below, covers the > DNS lookup, the TCP SYN handshake, and the TLS handshake. The proposed > spec wording deliberately avoids adding such low-level details. > Outline of the specification changes: To HttpClient.Builder add: /** * > Sets the connect timeout duration for this client. * *

In the case > where a new connection needs to be established, if * the connection > cannot be established within the given {@code * duration}, then an > {@link HttpConnectTimeoutException} is thrown * from {@link > HttpClient#send(java.net.http.HttpRequest, * > java.net.http.HttpResponse.BodyHandler) HttpClient::send}, or * {@link > HttpClient#sendAsync(java.net.http.HttpRequest, * > java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} * > completes exceptionally with an {@code HttpConnectTimeoutException}. * > If a new connection does not need to be established, for example * if > a connection can be reused from a previous request, then this * > timeout duration has no effect. * * @param duration the duration to > allow the underlying connection to be * established * @return this > builder * @throws IllegalArgumentException if the duration is > non-positive */ public Builder connectTimeout(Duration duration); To > HttpClient add an accessor: /** * Returns an {@code Optional} > containing the connect timeout duration * for this client. If > the {@linkplain Builder#connectTimeout(Duration) * connect timeout > duration} was not set in the client's builder, then the * {@code > Optional} is empty. * * @return an {@code Optional} containing this > client's connect timeout * duration */ public abstract > Optional connectTimeout(); Add new Exception type: /** * > Thrown when a connection, over which an {@code HttpRequest} is > intended to be * sent, is not successfully established within a > specified time period. */ public class HttpConnectTimeoutException > extends HttpTimeoutException { ... } Webrev: > http://cr.openjdk.java.net/~chegar/8208391/webrev.00/ -Chris [1] > http://mail.openjdk.java.net/pipermail/net-dev/2018-August/011683.html From albi646 at gmx.de Tue Aug 7 09:25:36 2018 From: albi646 at gmx.de (Albert Schimpf) Date: Tue, 7 Aug 2018 11:25:36 +0200 Subject: [httpclient] HTTP2: Memory Leak with Proxy Message-ID: Hi, I stumbled upon some strange behavior when using the new Java httpclient. The issue is very simple to reproduce. Send a GET request via a known bad proxy: HttpClient client = HttpClient.newBuilder() ??????????????????? .proxy(ProxySelector.of(BAD_PROXY)) ??????????????????? .build(); HttpRequest req = HttpRequest ??????????????????? // target is not relevant ??????????????????? .newBuilder(...) ??????????????????? .GET() ??????????????????? .build(); // body handler is not relevant HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); // happens with both async and sync send client.sendAsync(req, t).get(30, TimeUnit.SECONDS); The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? Best, Albert -------------- next part -------------- A non-text attachment was scrubbed... Name: mon.png Type: image/png Size: 64534 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Main.java Type: text/x-java Size: 1866 bytes Desc: not available URL: From chris.hegarty at oracle.com Tue Aug 7 09:52:07 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 7 Aug 2018 10:52:07 +0100 Subject: RFR [11] 8208391: Need to differentiate response and connect timeouts in HTTP Client API In-Reply-To: <92a69903-683b-1685-eb57-41d679a11aa1@gmail.com> References: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> <92a69903-683b-1685-eb57-41d679a11aa1@gmail.com> Message-ID: > On 4 Aug 2018, at 14:08, Jaikiran Pai wrote: > > ... > > Do you think this can be reworded a bit? Although I understand what's > being said here, the wording doesn't seem right. Maybe something like: > > *

In the case where a new connection needs to be > established, if > * the connection cannot be established within the given {@code > * duration}, then {@link HttpClient#send(java.net.http.HttpRequest, > * java.net.http.HttpResponse.BodyHandler) HttpClient::send} > throws a > * {@link HttpConnectTimeoutException}, or > * {@link HttpClient#sendAsync(java.net.http.HttpRequest, > * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} > * completes exceptionally with an {@code > HttpConnectTimeoutException}. Agreed. This wording avoids the previous awkwardness. > src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java > > - this.timeout = null; > + this.timeout = Duration.ofSeconds(30); > > Is this an intentional change of default value for the timeout? Is that > something that needs to be documented? Accidental, left over from a previous hacking session. Removed. Updated webrev: http://cr.openjdk.java.net/~chegar/8208391/webrev.01/ > One other thing - maybe not directly related to this single patch, but > as you are aware, recently as part of [1], a new system property (and a > security property) was introduced to optionally include host info in the > exception messages thrown for socket exceptions I am aware of this. > . Does the HttpClient > honour that system property in the exceptions it throws? The HTPT Client does not have any special handing for this property. It may not be necessary, at least not for low-level NIO exceptions, once the exception, or its cause, is preserved. > I don't see it > being used in this patch for the timeout exceptions. I haven't checked > the code, outside of this patch, to see if it's dealt with in other > parts of this API. Separately, I will look into the possibility of where such an extension can be used. -Chris. From chris.hegarty at oracle.com Tue Aug 7 10:05:16 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 7 Aug 2018 11:05:16 +0100 Subject: [httpclient] HTTP2: Memory Leak with Proxy In-Reply-To: References: Message-ID: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> Hi Albert, Very strange indeed. Thanks for reporting it, I?ll investigate. What do your mean by ?bad proxy?. What is bad about it, and how does it behave? -Chris. > On 7 Aug 2018, at 10:25, Albert Schimpf wrote: > > Hi, > > I stumbled upon some strange behavior when using the new Java httpclient. > > The issue is very simple to reproduce. Send a GET request via a known bad proxy: > > HttpClient client = HttpClient.newBuilder() > .proxy(ProxySelector.of(BAD_PROXY)) > .build(); > > HttpRequest req = HttpRequest > // target is not relevant > .newBuilder(...) > .GET() > .build(); > > // body handler is not relevant > HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); > > // happens with both async and sync send > client.sendAsync(req, t).get(30, TimeUnit.SECONDS); > > The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. > > In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. > > I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: > > https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip > > I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. > > Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? > > > Best, > Albert > From xuelei.fan at oracle.com Tue Aug 7 14:46:28 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Tue, 7 Aug 2018 07:46:28 -0700 Subject: Code Review Request, JDK-8207009 SSLEngine#closeInbound mentions SSLException when no close_notify is received In-Reply-To: References: <99d6eeaf-f77b-1967-10be-92347273b794@oracle.com> Message-ID: <1eb8223a-f355-3778-d355-f217f28cdb75@oracle.com> New webrev: http://cr.openjdk.java.net/~xuelei/8207009/webrev.03/ Thanks for a find of Tim Brooks, that the SSLEngine inbound/outbound status is incorrect if closing during handshake. The above webrev is trying to fix the problems. See more in the OpenJDK thread: http://mail.openjdk.java.net/pipermail/security-dev/2018-August/017778.html Please let me know your concerns before this Wednesday. Thanks, Xuelei On 8/3/2018 1:55 PM, Xuelei Fan wrote: > Update: http://cr.openjdk.java.net/~xuelei/8207009/webrev.02/ > > In webrev.01, the socket close may be blocked by super class close > synchronization.? Updated the SSLSocketImpl.java to use handshake only > lock in the startHandshake() implementation. > > Thanks, > Xuelei > > On 8/1/2018 7:27 PM, Xuelei Fan wrote: >> Update: http://cr.openjdk.java.net/~xuelei/8207009/webrev.01/ >> >> Integrated the fix for JDK-8208642, "Server initiated TLSv1.2 >> renegotiation fails if Java client allows TLSv1.3".? SSLHandshake.java >> is updated to use negotiated version so that TLS 1.2 HelloRequest is >> acceptable in TLS 1.3 client side. >> >> Thanks, >> Xuelei >> >> On 7/30/2018 10:24 AM, Xuelei Fan wrote: >>> >>> Please let me know your concerns by the end of August 1st, 2018. >>> >>> Thanks, >>> Xuelei >>> >>> >>> On 7/30/2018 9:59 AM, Xuelei Fan wrote: >>>> Hi, >>>> >>>> Please review the update for the TLS 1.3 half-close and >>>> synchronization implementation: >>>> ??? http://cr.openjdk.java.net/~xuelei/8207009/webrev.00/ >>>> >>>> Unlike TLS 1.2 and prior versions, for TLS 1.3, the close_notify is >>>> use to close the local write side and peer read side only.? After >>>> the close_notify get handles, the local read side and peer write >>>> side may still be open. >>>> >>>> In this update, if an application calls >>>> SSLEngine.closeInbound/Outbound() or >>>> SSLSocket.shutdownInput/Output(), half-close will be used.? For >>>> compatibility, if SSLSocket.close() get called, a duplex close will >>>> be tried.? In order to support duplex close, JDK will use the >>>> user_canceled warning alert even the handshake complete. >>>> >>>> In practice, an application may only close outbound even it is >>>> intended to close the inbound as well, or close the connection >>>> completely.? It works for TLS 1.2 and prior versions.? But no more >>>> for TLS 1.3 because of the close_notify behavior change in the TLS >>>> 1.3 specification.? The application may be hung and dead-waiting for >>>> read/close.? It could be solved by closing the inbound explicitly. >>>> In order to mitigate the impact, a new System Property is >>>> introduced, "jdk.tls.acknowledgeCloseNotify" if source code update >>>> is not available. ??If the System Property is set to "true", if >>>> receiving the close_notify, a close_notify alert will be responded. >>>> It is a countermeasure of the TLS 1.3 half-close issues. >>>> >>>> Thanks, >>>> Xuelei >>>> >>>> >>>> From albi646 at gmx.de Tue Aug 7 15:55:40 2018 From: albi646 at gmx.de (Albert Schimpf) Date: Tue, 7 Aug 2018 17:55:40 +0200 Subject: [httpclient] HTTP2: Memory Leak with Proxy In-Reply-To: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> References: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> Message-ID: Hi, by bad I mean that this proxy is the only one out of ~1000 proxies which causes this behavior. It's also the only one which causes SSLExceptions (General SSL engine problem) and EOFExceptions. I don't think that particular proxy is a valid proxy. Using curl indicates that this is a SSL handshake problem (https): * Rebuilt URL to: www.google.de/ *?? Trying 165.165.248.90... * TCP_NODELAY set * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0) * successfully set certificate verify locations: *?? CAfile: /etc/ssl/certs/ca-certificates.crt ? CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS handshake, Client hello (1): * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 * Closing connection 0 curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 I don't know that much about the protocol itself,? so I don't think I can help very much. Thank you for looking into this! Best, Albert On 07.08.2018 12:05, Chris Hegarty wrote: > Hi Albert, > > Very strange indeed. Thanks for reporting it, I?ll investigate. > > What do your mean by ?bad proxy?. What is bad about it, and how does it behave? > > -Chris. > > >> On 7 Aug 2018, at 10:25, Albert Schimpf wrote: >> >> Hi, >> >> I stumbled upon some strange behavior when using the new Java httpclient. >> >> The issue is very simple to reproduce. Send a GET request via a known bad proxy: >> >> HttpClient client = HttpClient.newBuilder() >> .proxy(ProxySelector.of(BAD_PROXY)) >> .build(); >> >> HttpRequest req = HttpRequest >> // target is not relevant >> .newBuilder(...) >> .GET() >> .build(); >> >> // body handler is not relevant >> HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); >> >> // happens with both async and sync send >> client.sendAsync(req, t).get(30, TimeUnit.SECONDS); >> >> The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. >> >> In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. >> >> I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: >> >> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip >> >> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. >> >> Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? >> >> >> Best, >> Albert >> From chris.hegarty at oracle.com Tue Aug 7 16:05:43 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 7 Aug 2018 17:05:43 +0100 Subject: [httpclient] HTTP2: Memory Leak with Proxy In-Reply-To: References: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> Message-ID: > On 7 Aug 2018, at 16:55, Albert Schimpf wrote: > > Hi, > > by bad I mean that this proxy is the only one out of ~1000 proxies which causes this behavior. It's also the only one which causes SSLExceptions (General SSL engine problem) and EOFExceptions. I don't think that particular proxy is a valid proxy. Using curl indicates that this is a SSL handshake problem (https): > > * Rebuilt URL to: www.google.de/ > * Trying 165.165.248.90? Oh, you mean the proxy at that actual IP address. Ok got it. > * TCP_NODELAY set > * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0) > * successfully set certificate verify locations: > * CAfile: /etc/ssl/certs/ca-certificates.crt > CApath: /etc/ssl/certs > * TLSv1.2 (OUT), TLS handshake, Client hello (1): > * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 > * Closing connection 0 > curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 > > I don't know that much about the protocol itself, so I don't think I can help very much. > > Thank you for looking into this! -Chris. > Best, > Albert > > > On 07.08.2018 12:05, Chris Hegarty wrote: >> Hi Albert, >> >> Very strange indeed. Thanks for reporting it, I?ll investigate. >> >> What do your mean by ?bad proxy?. What is bad about it, and how does it behave? >> >> -Chris. >> >> >>> On 7 Aug 2018, at 10:25, Albert Schimpf wrote: >>> >>> Hi, >>> >>> I stumbled upon some strange behavior when using the new Java httpclient. >>> >>> The issue is very simple to reproduce. Send a GET request via a known bad proxy: >>> >>> HttpClient client = HttpClient.newBuilder() >>> .proxy(ProxySelector.of(BAD_PROXY)) >>> .build(); >>> >>> HttpRequest req = HttpRequest >>> // target is not relevant >>> .newBuilder(...) >>> .GET() >>> .build(); >>> >>> // body handler is not relevant >>> HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); >>> >>> // happens with both async and sync send >>> client.sendAsync(req, t).get(30, TimeUnit.SECONDS); >>> >>> The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. >>> >>> In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. >>> >>> I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: >>> >>> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip >>> >>> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. >>> >>> Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? >>> >>> >>> Best, >>> Albert >>> > From chris.hegarty at oracle.com Tue Aug 7 16:53:09 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 7 Aug 2018 17:53:09 +0100 Subject: [httpclient] HTTP2: Memory Leak with Proxy In-Reply-To: References: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> Message-ID: <22DA38CA-2568-4BE9-8889-2D756A3EC2A4@oracle.com> Albert, I haven?t yet looked at what happens in JDK 10, but just to say, since things have moved on a lot in JDK 11 EA, that the same test runs with a reasonable amount of memory and CPU with JDK 11 EA [1]. I updated the test a little, since some the names have been changed with the standardisation ( see javadoc for more details [2] ). -Chris. [1] http://jdk.java.net/11/ [2] https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/package-summary.html --- $ cat BadProxyLeak.java import java.net.InetSocketAddress; import java.net.ProxySelector; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.util.concurrent.TimeUnit; public class BadProxyLeak { public static void main(String[] args) throws InterruptedException { // 1st heap dump // Thread.sleep(10000); { HttpClient client = HttpClient.newBuilder() // malicious proxy? .proxy(ProxySelector.of(new InetSocketAddress("165.165.248.90", 8080))) .build(); HttpRequest req = HttpRequest // target is not relevant .newBuilder(URI.create("https://www.google.de")) // leak occurs only with HTTP_2 .version(HttpClient.Version.HTTP_2) // .version(HttpClient.Version.HTTP_1_1) .GET() .build(); // use same client for every request // most likely happens on the first retry, if not try again or increase i for (int i = 0; i < 10; i++) { System.out.println("Request " + i); // body handler is not relevant HttpResponse.BodyHandler t = HttpResponse.BodyHandlers.ofString(); try { // happens with both async and sync send client.sendAsync(req, t).get(15, TimeUnit.SECONDS); // client.send(req, handler); } catch (Exception e) { e.printStackTrace(); } } } // 3rd heap dump System.out.println("Generating heap dump manually"); // space is not released Thread.sleep(60000); } } > On 7 Aug 2018, at 17:05, Chris Hegarty wrote: > > >> On 7 Aug 2018, at 16:55, Albert Schimpf wrote: >> >> Hi, >> >> by bad I mean that this proxy is the only one out of ~1000 proxies which causes this behavior. It's also the only one which causes SSLExceptions (General SSL engine problem) and EOFExceptions. I don't think that particular proxy is a valid proxy. Using curl indicates that this is a SSL handshake problem (https): >> >> * Rebuilt URL to: www.google.de/ >> * Trying 165.165.248.90? > > Oh, you mean the proxy at that actual IP address. Ok got it. > > >> * TCP_NODELAY set >> * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0) >> * successfully set certificate verify locations: >> * CAfile: /etc/ssl/certs/ca-certificates.crt >> CApath: /etc/ssl/certs >> * TLSv1.2 (OUT), TLS handshake, Client hello (1): >> * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 >> * Closing connection 0 >> curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 >> >> I don't know that much about the protocol itself, so I don't think I can help very much. >> >> Thank you for looking into this! > > > -Chris. > > >> Best, >> Albert >> >> >> On 07.08.2018 12:05, Chris Hegarty wrote: >>> Hi Albert, >>> >>> Very strange indeed. Thanks for reporting it, I?ll investigate. >>> >>> What do your mean by ?bad proxy?. What is bad about it, and how does it behave? >>> >>> -Chris. >>> >>> >>>> On 7 Aug 2018, at 10:25, Albert Schimpf wrote: >>>> >>>> Hi, >>>> >>>> I stumbled upon some strange behavior when using the new Java httpclient. >>>> >>>> The issue is very simple to reproduce. Send a GET request via a known bad proxy: >>>> >>>> HttpClient client = HttpClient.newBuilder() >>>> .proxy(ProxySelector.of(BAD_PROXY)) >>>> .build(); >>>> >>>> HttpRequest req = HttpRequest >>>> // target is not relevant >>>> .newBuilder(...) >>>> .GET() >>>> .build(); >>>> >>>> // body handler is not relevant >>>> HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); >>>> >>>> // happens with both async and sync send >>>> client.sendAsync(req, t).get(30, TimeUnit.SECONDS); >>>> >>>> The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. >>>> >>>> In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. >>>> >>>> I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: >>>> >>>> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip >>>> >>>> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. >>>> >>>> Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? >>>> >>>> >>>> Best, >>>> Albert >>>> >> > From michael.x.mcmahon at oracle.com Wed Aug 8 11:55:46 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 08 Aug 2018 12:55:46 +0100 Subject: RFR [11] 8208391: Need to differentiate response and connect timeouts in HTTP Client API In-Reply-To: References: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> <92a69903-683b-1685-eb57-41d679a11aa1@gmail.com> Message-ID: <5B6ADA42.1050307@oracle.com> The updated webrev looks fine to me, Chris. Thanks, Michael On 07/08/2018, 10:52, Chris Hegarty wrote: >> On 4 Aug 2018, at 14:08, Jaikiran Pai wrote: >> >> ... >> >> Do you think this can be reworded a bit? Although I understand what's >> being said here, the wording doesn't seem right. Maybe something like: >> >> *

In the case where a new connection needs to be >> established, if >> * the connection cannot be established within the given {@code >> * duration}, then {@link HttpClient#send(java.net.http.HttpRequest, >> * java.net.http.HttpResponse.BodyHandler) HttpClient::send} >> throws a >> * {@link HttpConnectTimeoutException}, or >> * {@link HttpClient#sendAsync(java.net.http.HttpRequest, >> * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} >> * completes exceptionally with an {@code >> HttpConnectTimeoutException}. > Agreed. This wording avoids the previous awkwardness. > >> src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java >> >> - this.timeout = null; >> + this.timeout = Duration.ofSeconds(30); >> >> Is this an intentional change of default value for the timeout? Is that >> something that needs to be documented? > Accidental, left over from a previous hacking session. Removed. > > Updated webrev: > http://cr.openjdk.java.net/~chegar/8208391/webrev.01/ > >> One other thing - maybe not directly related to this single patch, but >> as you are aware, recently as part of [1], a new system property (and a >> security property) was introduced to optionally include host info in the >> exception messages thrown for socket exceptions > I am aware of this. > >> . Does the HttpClient >> honour that system property in the exceptions it throws? > The HTPT Client does not have any special handing for this property. > It may not be necessary, at least not for low-level NIO exceptions, once > the exception, or its cause, is preserved. > >> I don't see it >> being used in this patch for the timeout exceptions. I haven't checked >> the code, outside of this patch, to see if it's dealt with in other >> parts of this API. > Separately, I will look into the possibility of where such an extension > can be used. > > -Chris. > From albi646 at gmx.de Wed Aug 8 13:45:05 2018 From: albi646 at gmx.de (Albert Schimpf) Date: Wed, 8 Aug 2018 15:45:05 +0200 Subject: [httpclient] HTTP2: Memory Leak with Proxy In-Reply-To: <22DA38CA-2568-4BE9-8889-2D756A3EC2A4@oracle.com> References: <2CF05F02-ADAA-4238-A3D8-46967B0CF657@oracle.com> <22DA38CA-2568-4BE9-8889-2D756A3EC2A4@oracle.com> Message-ID: Chris, that's good to hear, I'll try it out. Best, Albert On 07.08.2018 18:53, Chris Hegarty wrote: > Albert, > > I haven?t yet looked at what happens in JDK 10, but just to say, since things have moved on a lot in JDK 11 EA, that the same test runs with a reasonable amount of memory and CPU with JDK 11 EA [1]. I updated the test a little, since some the names have been changed with the standardisation ( see javadoc for more details [2] ). > > -Chris. > > [1] http://jdk.java.net/11/ > [2] https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/package-summary.html > > --- > > $ cat BadProxyLeak.java > > import java.net.InetSocketAddress; > import java.net.ProxySelector; > import java.net.URI; > import java.net.http.HttpClient; > import java.net.http.HttpRequest; > import java.net.http.HttpResponse; > import java.util.concurrent.TimeUnit; > > public class BadProxyLeak { > public static void main(String[] args) throws InterruptedException { > // 1st heap dump > // Thread.sleep(10000); > > { > HttpClient client = HttpClient.newBuilder() > // malicious proxy? > .proxy(ProxySelector.of(new InetSocketAddress("165.165.248.90", 8080))) > .build(); > > HttpRequest req = HttpRequest > // target is not relevant > .newBuilder(URI.create("https://www.google.de")) > // leak occurs only with HTTP_2 > .version(HttpClient.Version.HTTP_2) > // .version(HttpClient.Version.HTTP_1_1) > .GET() > .build(); > > > // use same client for every request > // most likely happens on the first retry, if not try again or increase i > for (int i = 0; i < 10; i++) { > System.out.println("Request " + i); > // body handler is not relevant > HttpResponse.BodyHandler t = HttpResponse.BodyHandlers.ofString(); > > try { > // happens with both async and sync send > client.sendAsync(req, t).get(15, TimeUnit.SECONDS); > // client.send(req, handler); > } catch (Exception e) { > e.printStackTrace(); > } > } > } > > // 3rd heap dump > System.out.println("Generating heap dump manually"); > // space is not released > Thread.sleep(60000); > } > } > > >> On 7 Aug 2018, at 17:05, Chris Hegarty wrote: >> >> >>> On 7 Aug 2018, at 16:55, Albert Schimpf wrote: >>> >>> Hi, >>> >>> by bad I mean that this proxy is the only one out of ~1000 proxies which causes this behavior. It's also the only one which causes SSLExceptions (General SSL engine problem) and EOFExceptions. I don't think that particular proxy is a valid proxy. Using curl indicates that this is a SSL handshake problem (https): >>> >>> * Rebuilt URL to: www.google.de/ >>> * Trying 165.165.248.90? >> Oh, you mean the proxy at that actual IP address. Ok got it. >> >> >>> * TCP_NODELAY set >>> * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0) >>> * successfully set certificate verify locations: >>> * CAfile: /etc/ssl/certs/ca-certificates.crt >>> CApath: /etc/ssl/certs >>> * TLSv1.2 (OUT), TLS handshake, Client hello (1): >>> * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 >>> * Closing connection 0 >>> curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 >>> >>> I don't know that much about the protocol itself, so I don't think I can help very much. >>> >>> Thank you for looking into this! >> >> -Chris. >> >> >>> Best, >>> Albert >>> >>> >>> On 07.08.2018 12:05, Chris Hegarty wrote: >>>> Hi Albert, >>>> >>>> Very strange indeed. Thanks for reporting it, I?ll investigate. >>>> >>>> What do your mean by ?bad proxy?. What is bad about it, and how does it behave? >>>> >>>> -Chris. >>>> >>>> >>>>> On 7 Aug 2018, at 10:25, Albert Schimpf wrote: >>>>> >>>>> Hi, >>>>> >>>>> I stumbled upon some strange behavior when using the new Java httpclient. >>>>> >>>>> The issue is very simple to reproduce. Send a GET request via a known bad proxy: >>>>> >>>>> HttpClient client = HttpClient.newBuilder() >>>>> .proxy(ProxySelector.of(BAD_PROXY)) >>>>> .build(); >>>>> >>>>> HttpRequest req = HttpRequest >>>>> // target is not relevant >>>>> .newBuilder(...) >>>>> .GET() >>>>> .build(); >>>>> >>>>> // body handler is not relevant >>>>> HttpResponse.BodyHandler t = HttpResponse.BodyHandler.asString(); >>>>> >>>>> // happens with both async and sync send >>>>> client.sendAsync(req, t).get(30, TimeUnit.SECONDS); >>>>> >>>>> The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1. >>>>> >>>>> In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst. >>>>> >>>>> I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox: >>>>> >>>>> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip >>>>> >>>>> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times. >>>>> >>>>> Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior? >>>>> >>>>> >>>>> Best, >>>>> Albert >>>>> From chris.hegarty at oracle.com Wed Aug 8 13:31:00 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 8 Aug 2018 14:31:00 +0100 Subject: Priority can be set for HttpClient with HTTP/1.1 In-Reply-To: References: Message-ID: <826DEC39-015F-4556-B347-802BB85D8225@oracle.com> Hi Nezih, The client?s version is a default preferred version for requests that do not specify an explicit version. As such, it does not prevent requests, attempting to negotiate HTTP/2, from being sent, just that it is not the default. Given that, then the priority value is still be applicable to client?s build with a preferred HTTP/1.1 version. -Chris. > On 1 Aug 2018, at 10:56, nezih yigitbasi wrote: > > Hi, > Priority is only defined for version HTTP_2 (and that's clear from the javadoc). However, it's possible to construct an instance of HttpClient with a priority value and with version HTTP_1_1 (tested with Java "9.0.1"). I feel that it will be less error prone for the users if conflicting combinations of arguments weren't allowed. It's also confusing when I read the below code as it doesn't make sense to set the priority for HTTP/1.1. What do you think? > > HttpClient client = HttpClient.newBuilder() > .executor(httpClientExecutor) > .priority(32) > .version(HTTP_1_1) > .build(); > > Thanks, > Nezih From Klaus.Malorny at knipp.de Wed Aug 8 14:43:52 2018 From: Klaus.Malorny at knipp.de (Klaus Malorny) Date: Wed, 8 Aug 2018 16:43:52 +0200 Subject: Suggestion for the HTTP client: allow selection of a local address Message-ID: <6cafcfae-8563-ebe0-58e8-3f9df9c16087@knipp.de> Hi, I just discussed with a colleague that we dreadfully miss the ability in the old HttpURLConnection to set the address to which the client locally binds, which we require to get through the firewalls of our business partners when connecting from a multihomed system. This repeatedly caused us to use third-party libraries, which are powerful on the one hand, but cumbersome on the other. With the new HTTP2 client in mind, I wanted to tell him, wait, with Java 9+, it will be better. But I checked first and discovered that there is still no way to do so. So, when you eventually revise the API, it would be nice if you could consider the addition of such a feature. Regards, Klaus From michael.x.mcmahon at oracle.com Wed Aug 8 15:46:04 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 08 Aug 2018 16:46:04 +0100 Subject: Suggestion for the HTTP client: allow selection of a local address In-Reply-To: <6cafcfae-8563-ebe0-58e8-3f9df9c16087@knipp.de> References: <6cafcfae-8563-ebe0-58e8-3f9df9c16087@knipp.de> Message-ID: <5B6B103C.4020005@oracle.com> Klaus I've created an RFE at https://bugs.openjdk.java.net/browse/JDK-8209137 to track this. I imagine the simplest solution would be to be able to specify the local address when constructing a HttpClient, and then all connections created using that client would be bound to the chosen address. Michael. On 08/08/2018, 15:43, Klaus Malorny wrote: > > > Hi, > > I just discussed with a colleague that we dreadfully miss the ability > in the old HttpURLConnection to set the address to which the client > locally binds, which we require to get through the firewalls of our > business partners when connecting from a multihomed system. This > repeatedly caused us to use third-party libraries, which are powerful > on the one hand, but cumbersome on the other. > > With the new HTTP2 client in mind, I wanted to tell him, wait, with > Java 9+, it will be better. But I checked first and discovered that > there is still no way to do so. > > So, when you eventually revise the API, it would be nice if you could > consider the addition of such a feature. > > Regards, > > Klaus > From jai.forums2013 at gmail.com Thu Aug 9 09:09:18 2018 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Thu, 9 Aug 2018 14:39:18 +0530 Subject: RFR [11] 8208391: Need to differentiate response and connect timeouts in HTTP Client API In-Reply-To: References: <4255A920-468C-41E4-BC44-C309E5A32B65@oracle.com> <92a69903-683b-1685-eb57-41d679a11aa1@gmail.com> Message-ID: <34be2a6a-e0a9-8f12-c160-dff8ab99f183@gmail.com> Thanks for the updates, Chris. This looks fine to me (I'm not a reviewer). -Jaikiran On 07/08/18 3:22 PM, Chris Hegarty wrote: >> On 4 Aug 2018, at 14:08, Jaikiran Pai wrote: >> >> ... >> >> Do you think this can be reworded a bit? Although I understand what's >> being said here, the wording doesn't seem right. Maybe something like: >> >> *

In the case where a new connection needs to be >> established, if >> * the connection cannot be established within the given {@code >> * duration}, then {@link HttpClient#send(java.net.http.HttpRequest, >> * java.net.http.HttpResponse.BodyHandler) HttpClient::send} >> throws a >> * {@link HttpConnectTimeoutException}, or >> * {@link HttpClient#sendAsync(java.net.http.HttpRequest, >> * java.net.http.HttpResponse.BodyHandler) HttpClient::sendAsync} >> * completes exceptionally with an {@code >> HttpConnectTimeoutException}. > Agreed. This wording avoids the previous awkwardness. > >> src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java >> >> - this.timeout = null; >> + this.timeout = Duration.ofSeconds(30); >> >> Is this an intentional change of default value for the timeout? Is that >> something that needs to be documented? > Accidental, left over from a previous hacking session. Removed. > > Updated webrev: > http://cr.openjdk.java.net/~chegar/8208391/webrev.01/ > >> One other thing - maybe not directly related to this single patch, but >> as you are aware, recently as part of [1], a new system property (and a >> security property) was introduced to optionally include host info in the >> exception messages thrown for socket exceptions > I am aware of this. > >> . Does the HttpClient >> honour that system property in the exceptions it throws? > The HTPT Client does not have any special handing for this property. > It may not be necessary, at least not for low-level NIO exceptions, once > the exception, or its cause, is preserved. > >> I don't see it >> being used in this patch for the timeout exceptions. I haven't checked >> the code, outside of this patch, to see if it's dealt with in other >> parts of this API. > Separately, I will look into the possibility of where such an extension > can be used. > > -Chris. > From michael.x.mcmahon at oracle.com Tue Aug 14 13:59:51 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Tue, 14 Aug 2018 14:59:51 +0100 Subject: RFR [11]: 8207966 HttpClient response without content-length does not return body Message-ID: <5B72E057.3050900@oracle.com> Hi, This is an important fix for 11 which addresses the problem where HTTP/1.0 responses that do not include a content-length header are not handled correctly. http://cr.openjdk.java.net/~michaelm/8207966/webrev.2/index.html Thanks, Michael. From chris.hegarty at oracle.com Tue Aug 14 15:08:21 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 14 Aug 2018 16:08:21 +0100 Subject: RFR [11]: 8207966 HttpClient response without content-length does not return body In-Reply-To: <5B72E057.3050900@oracle.com> References: <5B72E057.3050900@oracle.com> Message-ID: <520853BC-F9CA-47DA-A138-5BB46D9D2DA8@oracle.com> > On 14 Aug 2018, at 14:59, Michael McMahon wrote: > > Hi, > > This is an important fix for 11 which addresses the problem where HTTP/1.0 > responses that do not include a content-length header are not handled correctly. > > http://cr.openjdk.java.net/~michaelm/8207966/webrev.2/index.html This looks good Michael. The explicit testng timeouts can be dropped from the @run lines in the test. -Chris. From openjdk at suche.org Tue Aug 14 18:37:36 2018 From: openjdk at suche.org (=?UTF-8?Q?Thomas_Lu=c3=9fnig?=) Date: Tue, 14 Aug 2018 20:37:36 +0200 Subject: RFR [11]: 8207966 HttpClient response without content-length does not return body In-Reply-To: <5B72E057.3050900@oracle.com> References: <5B72E057.3050900@oracle.com> Message-ID: <89af28b9-1a52-94cb-c8c2-23e64ea49057@suche.org> Hi, i see that only missing Content-length is allowed with http/1.0. I am not sure a) if this is not also allowed with http/1.1 b) even if it is not allowed that it is good to read the body but not make it available for the application. ? From integration scenarios i know that this can often cause trouble if one side say we send the message ? And the other say we received nothing. In these situation network analyses and traffic log tell the message ? was transmitted but the application tell 0 bytes received. If the message is dropped i think there should be an exception telling that there was an mandatory header missing. Gru? Thomas Lu?nig p.s. Great to see that an fix is available soon. On 14.08.2018 15:59:51, Michael McMahon wrote: > Hi, > > This is an important fix for 11 which addresses the problem where > HTTP/1.0 > responses that do not include a content-length header are not handled > correctly. > > http://cr.openjdk.java.net/~michaelm/8207966/webrev.2/index.html > > Thanks, > Michael. > From michael.x.mcmahon at oracle.com Wed Aug 15 08:31:14 2018 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Wed, 15 Aug 2018 09:31:14 +0100 Subject: RFR [11]: 8207966 HttpClient response without content-length does not return body In-Reply-To: <89af28b9-1a52-94cb-c8c2-23e64ea49057@suche.org> References: <5B72E057.3050900@oracle.com> <89af28b9-1a52-94cb-c8c2-23e64ea49057@suche.org> Message-ID: <5B73E4D2.3030407@oracle.com> Thomas, Yes, it appears that this has never been limited only to HTTP/1.0. Closing the connection has always been allowed as a way of terminating the response body. I will remove the restriction. Thanks, Michael On 14/08/2018, 19:37, Thomas Lu?nig wrote: > Hi, > > i see that only missing Content-length is allowed with http/1.0. > I am not sure > a) if this is not also allowed with http/1.1 > b) even if it is not allowed that it is good to read the body but not > make it available for the application. > From integration scenarios i know that this can often cause trouble > if one side say we send the message > And the other say we received nothing. In these situation network > analyses and traffic log tell the message > was transmitted but the application tell 0 bytes received. > If the message is dropped i think there should be an exception telling > that there was an mandatory header missing. > > Gru? Thomas Lu?nig > > p.s. Great to see that an fix is available soon. > > On 14.08.2018 15:59:51, Michael McMahon wrote: >> Hi, >> >> This is an important fix for 11 which addresses the problem where >> HTTP/1.0 >> responses that do not include a content-length header are not handled >> correctly. >> >> http://cr.openjdk.java.net/~michaelm/8207966/webrev.2/index.html >> >> Thanks, >> Michael. >> From chandra-shekar.s at hpe.com Tue Aug 21 17:21:11 2018 From: chandra-shekar.s at hpe.com (S, Chandra Shekar (CDI R&D)) Date: Tue, 21 Aug 2018 17:21:11 +0000 Subject: Regarding the Enhancement request for a new method (ex : getIPv6AddressByName) to give IPv6 address for the given host name. Message-ID: Hello Experts, This is regarding the Enhancement request for a new method to give IPv6 address for the given host name. Currently we have a method InetAddress.getByName(String host) to give the IPAddress for the given hostname. If the DNS server resolves to both IPv4 and IPv6 addresses for the given hostname, this method will return only the first IPv4 address. Actually this method queries the DNS server first for the IPv4 addresses (ie, with option -query=A) and then queries the DNS server for the IPv6 (ie, with option -query=AAAA) addresses. It consolidates the both IPv4 address and IPv6 address into the list, and returns the first address from the consolidate list. As the IPv4 address will be the first in the list , it always returns the IPv4 address only. If we need to get only IPv6 address for the given host name (especially in the IPv6Only environments), currently there is no such method/api for this. Currently we have method InetAddress.getAllByName(String host), which gives both IPv4 address and IPv6 address list, by iterating this we can get the IPv6 address resolution. But it is nice to have a separate method to get/resolve to IPv6address. Ex : getIPv6AddressByName(String host) or getByName(String host, AddressType addressType) - addressType will be passed by the call get IPv4/IPv6 address. This new method can query the DNS only for the IPv6 address (ie, with option -query=AAAA) which will have performance improvement over other. Thanks & Regards Shekar -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Aug 23 09:11:56 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 23 Aug 2018 10:11:56 +0100 Subject: Regarding the Enhancement request for a new method (ex : getIPv6AddressByName) to give IPv6 address for the given host name. In-Reply-To: References: Message-ID: <9DE86766-F463-4881-8E48-8B46BB81E253@oracle.com> Hi Shekar, Have you tried `java.net.preferIPv6Addresses` [1]? Setting that property to `true` should result in IPv6 addresses, if there are any, being returned before IPv4 addresses. -Chris. [1] https://docs.oracle.com/javase/10/docs/api/java/net/doc-files/net-properties.html#Ipv4IPv6 > On 21 Aug 2018, at 18:21, S, Chandra Shekar (CDI R&D) wrote: > > Hello Experts, > > This is regarding the Enhancement request for a new method to give IPv6 address for the given host name. > Currently we have a method InetAddress.getByName (String host) to give the IPAddress for the given hostname. > > If the DNS server resolves to both IPv4 and IPv6 addresses for the given hostname, this method will return only the first IPv4 address. > Actually this method queries the DNS server first for the IPv4 addresses (ie, with option -query=A) and then queries the DNS server for the IPv6 (ie, with option -query=AAAA) addresses. > It consolidates the both IPv4 address and IPv6 address into the list, and returns the first address from the consolidate list. As the IPv4 address will be the first in the list , it always returns the IPv4 address only. > > If we need to get only IPv6 address for the given host name (especially in the IPv6Only environments), currently there is no such method/api for this. > Currently we have method InetAddress.getAllByName (String host), which gives both IPv4 address and IPv6 address list, by iterating this we can get the IPv6 address resolution. > But it is nice to have a separate method to get/resolve to IPv6address. Ex : getIPv6AddressByName(String host) or getByName (String host, AddressType addressType) ? addressType will be passed by the call get IPv4/IPv6 address. > This new method can query the DNS only for the IPv6 address (ie, with option -query=AAAA) which will have performance improvement over other. > > Thanks & Regards > Shekar -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandra-shekar.s at hpe.com Fri Aug 24 14:46:39 2018 From: chandra-shekar.s at hpe.com (S, Chandra Shekar (CDI R&D)) Date: Fri, 24 Aug 2018 14:46:39 +0000 Subject: Regarding the Enhancement request for a new method (ex : getIPv6AddressByName) to give IPv6 address for the given host name. In-Reply-To: <9DE86766-F463-4881-8E48-8B46BB81E253@oracle.com> References: <9DE86766-F463-4881-8E48-8B46BB81E253@oracle.com> Message-ID: Hi Chris, This works. When this property is set in my application always , it will resolve to IPv6 address. Suppose if in application some code flow path needs IPv4 address resolution and some other code path needs IPv6 address resolution for different hostnames, then this is not possible. So, if there are different method to resolve differently for IPv4 and IPv4 address , that would be helpful. Thanks & Regards Shekar From: Chris Hegarty [mailto:chris.hegarty at oracle.com] Sent: Thursday, August 23, 2018 2:42 PM To: S, Chandra Shekar (CDI R&D) Cc: net-dev at openjdk.java.net; Pallavi Sonal Subject: Re: Regarding the Enhancement request for a new method (ex : getIPv6AddressByName) to give IPv6 address for the given host name. Hi Shekar, Have you tried `java.net.preferIPv6Addresses` [1]? Setting that property to `true` should result in IPv6 addresses, if there are any, being returned before IPv4 addresses. -Chris. [1] https://docs.oracle.com/javase/10/docs/api/java/net/doc-files/net-properties.html#Ipv4IPv6 On 21 Aug 2018, at 18:21, S, Chandra Shekar (CDI R&D) > wrote: Hello Experts, This is regarding the Enhancement request for a new method to give IPv6 address for the given host name. Currently we have a method InetAddress.getByName(String host) to give the IPAddress for the given hostname. If the DNS server resolves to both IPv4 and IPv6 addresses for the given hostname, this method will return only the first IPv4 address. Actually this method queries the DNS server first for the IPv4 addresses (ie, with option -query=A) and then queries the DNS server for the IPv6 (ie, with option -query=AAAA) addresses. It consolidates the both IPv4 address and IPv6 address into the list, and returns the first address from the consolidate list. As the IPv4 address will be the first in the list , it always returns the IPv4 address only. If we need to get only IPv6 address for the given host name (especially in the IPv6Only environments), currently there is no such method/api for this. Currently we have method InetAddress.getAllByName(String host), which gives both IPv4 address and IPv6 address list, by iterating this we can get the IPv6 address resolution. But it is nice to have a separate method to get/resolve to IPv6address. Ex : getIPv6AddressByName(String host) or getByName(String host, AddressType addressType) ? addressType will be passed by the call get IPv4/IPv6 address. This new method can query the DNS only for the IPv6 address (ie, with option -query=AAAA) which will have performance improvement over other. Thanks & Regards Shekar -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias.baesken at sap.com Mon Aug 27 15:12:57 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 27 Aug 2018 15:12:57 +0000 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list Message-ID: <805fae76c34642a1bb4a48c15064aa04@sap.com> Hello, please review this small fix ; When returning from Java_java_net_NetworkInterface_getAll (windows version), we have to free resources to avoid leaks. In some special cases this is not done . Bug : https://bugs.openjdk.java.net/browse/JDK-8209994 change : http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Mon Aug 27 15:16:04 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 27 Aug 2018 16:16:04 +0100 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <805fae76c34642a1bb4a48c15064aa04@sap.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: On 27 Aug 2018, at 16:12, Baesken, Matthias wrote: > > ... > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ Looks good. Thanks. -Chris. From volker.simonis at gmail.com Mon Aug 27 15:37:37 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 27 Aug 2018 17:37:37 +0200 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <805fae76c34642a1bb4a48c15064aa04@sap.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: Hi Matthias, the change looks good but I think you should also initialize 'ifList' in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its value is undefined and if 'enumInterfaces()' returns with an error without assigning 'ifList' you may end up calling 'free_netif()' with an undefined, non-NULL value. Best regards, Volker On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias wrote: > > Hello, please review this small fix ; > > > > When returning from Java_java_net_NetworkInterface_getAll (windows version), we have to free resources to avoid leaks. > > In some special cases this is not done . > > > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8209994 > > > > change : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ > > > > > > > > Thanks, Matthias From brian.burkhalter at oracle.com Mon Aug 27 23:48:25 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 27 Aug 2018 16:48:25 -0700 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <805fae76c34642a1bb4a48c15064aa04@sap.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: <73CBF13F-3A63-4E23-B81A-A881295E1707@oracle.com> Hi Matthias, This looks fine to me. Thanks, Brian On Aug 27, 2018, at 8:12 AM, Baesken, Matthias wrote: > Hello, please review this small fix ; > > When returning from Java_java_net_NetworkInterface_getAll (windows version), we have to free resources to avoid leaks. > In some special cases this is not done . > > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8209994 > > change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.burkhalter at oracle.com Mon Aug 27 23:52:01 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 27 Aug 2018 16:52:01 -0700 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <805fae76c34642a1bb4a48c15064aa04@sap.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: <1608B863-4DB1-4610-891B-B7D0A067E902@oracle.com> Hi Matthias, I neglected to mention that an appropriate noreg-* label [1], perhaps ?noreg-cleanup," will be needed on the issue as it seems likely untestable. Thanks, Brian [1] http://openjdk.java.net/guide/changePlanning.html#bug, section 6. On Aug 27, 2018, at 8:12 AM, Baesken, Matthias wrote: > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8209994 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias.baesken at sap.com Tue Aug 28 07:19:19 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 28 Aug 2018 07:19:19 +0000 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: > > the change looks good but I think you should also initialize 'ifList' > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > value is undefined and if 'enumInterfaces()' returns with an error > without assigning 'ifList' you may end up calling 'free_netif()' with > an undefined, non-NULL value. > Hi Volki, looking at the coding, your sceanario ***should*** not happen ; however to be on the safe side it is for sure better to do the initialization you propose. Looking a bit more at the coding, there is a Java_java_net_NetworkInterface_getAll_XP that has similar issues (missing free_netif calls in case of "early" returns ). I adjusted this as well in the second webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ Best regards , Matthias > -----Original Message----- > From: Volker Simonis > Sent: Montag, 27. August 2018 17:38 > To: Baesken, Matthias > Cc: net-dev > Subject: Re: [XS] RFR: 8209994: windows: > Java_java_net_NetworkInterface_getAll misses releasing interface-list > > Hi Matthias, > > the change looks good but I think you should also initialize 'ifList' > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > value is undefined and if 'enumInterfaces()' returns with an error > without assigning 'ifList' you may end up calling 'free_netif()' with > an undefined, non-NULL value. > > Best regards, > Volker > > On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias > wrote: > > > > Hello, please review this small fix ; > > > > > > > > When returning from Java_java_net_NetworkInterface_getAll (windows > version), we have to free resources to avoid leaks. > > > > In some special cases this is not done . > > > > > > > > > > > > Bug : > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8209994 > > > > > > > > change : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ > > > > > > > > > > > > > > > > Thanks, Matthias From volker.simonis at gmail.com Tue Aug 28 12:07:28 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 28 Aug 2018 14:07:28 +0200 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: Thanks for updating the change. Looks good now! Regards, Volker On Tue, Aug 28, 2018 at 9:19 AM Baesken, Matthias wrote: > > > > > the change looks good but I think you should also initialize 'ifList' > > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > > value is undefined and if 'enumInterfaces()' returns with an error > > without assigning 'ifList' you may end up calling 'free_netif()' with > > an undefined, non-NULL value. > > > > > Hi Volki, > looking at the coding, your sceanario ***should*** not happen ; however to be on the safe side it is for sure better to do the initialization > you propose. > > Looking a bit more at the coding, there is a Java_java_net_NetworkInterface_getAll_XP that has similar issues (missing free_netif calls in case of "early" returns ). > I adjusted this as well in the second webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ > > > Best regards , Matthias > > > > -----Original Message----- > > From: Volker Simonis > > Sent: Montag, 27. August 2018 17:38 > > To: Baesken, Matthias > > Cc: net-dev > > Subject: Re: [XS] RFR: 8209994: windows: > > Java_java_net_NetworkInterface_getAll misses releasing interface-list > > > > Hi Matthias, > > > > the change looks good but I think you should also initialize 'ifList' > > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > > value is undefined and if 'enumInterfaces()' returns with an error > > without assigning 'ifList' you may end up calling 'free_netif()' with > > an undefined, non-NULL value. > > > > Best regards, > > Volker > > > > On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias > > wrote: > > > > > > Hello, please review this small fix ; > > > > > > > > > > > > When returning from Java_java_net_NetworkInterface_getAll (windows > > version), we have to free resources to avoid leaks. > > > > > > In some special cases this is not done . > > > > > > > > > > > > > > > > > > Bug : > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8209994 > > > > > > > > > > > > change : > > > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ > > > > > > > > > > > > > > > > > > > > > > > > Thanks, Matthias From matthias.baesken at sap.com Tue Aug 28 12:13:24 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 28 Aug 2018 12:13:24 +0000 Subject: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c Message-ID: <92622c8a0eaa441f90469192d60ccf2d@sap.com> Hello, the MSDN docu about WSAGetLastError warns to get the error-code ***immediately*** after occurance. See : https://msdn.microsoft.com/de-de/library/windows/desktop/ms741580(v=vs.85).aspx " ... If a function call's return value indicates that error or other relevant data was returned in the error code, WSAGetLastError should be called immediately ..." However in windows SocketInputStream.c , this was not done; we noticed very seldom errors because of this (not reproducible however) so we had a fix for this in our code base for a long time. Should we change this as well in OpenJDK , for example from : jdk/src/java.base/windows/native/libnet/SocketInputStream.c 120 nread = recv(fd, bufP, len, 0); 121 if (nread > 0) { 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); 123 } else { 124 if (nread < 0) { 125 // Check if the socket has been closed since we last checked. 126 // This could be a reason for recv failing. 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { 128 JNU_ThrowByName(env, "java/net/SocketException", "Socket closed"); 129 } else { 130 switch (WSAGetLastError()) { to : 120 nread = recv(fd, bufP, len, 0); 121 if (nread > 0) { 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); 123 } else { 124 if (nread < 0) { 125 int err = WSAGetLastError(); ... switch (err) { Thanks and best regards, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Tue Aug 28 12:23:05 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 28 Aug 2018 14:23:05 +0200 Subject: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <92622c8a0eaa441f90469192d60ccf2d@sap.com> References: <92622c8a0eaa441f90469192d60ccf2d@sap.com> Message-ID: Hi Matthias, not strictly necessary, since WSAGetLastError() only refers to socket calls and GetIntField() is unlikely to call socket functions... However, I think your proposal is fine for code cleanliness sake. Best Regards, Thomas On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias wrote: > Hello, > > the MSDN docu about WSAGetLastError warns to get the error-code > ***immediately*** after occurance. > > See : > > > > https://msdn.microsoft.com/de-de/library/windows/desktop/ms741580(v=vs.85).aspx > > > > " ... If a function call's return value indicates that error or other > relevant data was returned in the error code, > > WSAGetLastError should be called immediately ..." > > > > However in windows SocketInputStream.c , this was not done; we noticed very > seldom errors because of this (not reproducible however) so > > we had a fix for this in our code base for a long time. > > > > Should we change this as well in OpenJDK , for example from : > > > > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > > > > > 120 nread = recv(fd, bufP, len, 0); > > 121 if (nread > 0) { > > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); > > 123 } else { > > 124 if (nread < 0) { > > 125 // Check if the socket has been closed since we last checked. > > 126 // This could be a reason for recv failing. > > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { > > 128 JNU_ThrowByName(env, "java/net/SocketException", "Socket > closed"); > > 129 } else { > > 130 switch (WSAGetLastError()) { > > > > to : > > > > 120 nread = recv(fd, bufP, len, 0); > > 121 if (nread > 0) { > > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); > > 123 } else { > > 124 if (nread < 0) { > > 125 int err = WSAGetLastError(); > > ... > > switch (err) { > > > > > > Thanks and best regards, Matthias From matthias.baesken at sap.com Tue Aug 28 14:25:05 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 28 Aug 2018 14:25:05 +0000 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: <4f3acead81ab421297f84c7127c51f8d@sap.com> Thanks Volki ! Can I have a second review please ? Best regards, Matthias > -----Original Message----- > From: Volker Simonis > Sent: Dienstag, 28. August 2018 14:07 > To: Baesken, Matthias > Cc: net-dev ; Chris Hegarty > ; Brian Burkhalter > > Subject: Re: [XS] RFR: 8209994: windows: > Java_java_net_NetworkInterface_getAll misses releasing interface-list > > Thanks for updating the change. > > Looks good now! > > Regards, > Volker > On Tue, Aug 28, 2018 at 9:19 AM Baesken, Matthias > wrote: > > > > > > > > the change looks good but I think you should also initialize 'ifList' > > > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > > > value is undefined and if 'enumInterfaces()' returns with an error > > > without assigning 'ifList' you may end up calling 'free_netif()' with > > > an undefined, non-NULL value. > > > > > > > > > Hi Volki, > > looking at the coding, your sceanario ***should*** not happen ; > however to be on the safe side it is for sure better to do the initialization > > you propose. > > > > Looking a bit more at the coding, there is a > Java_java_net_NetworkInterface_getAll_XP that has similar issues (missing > free_netif calls in case of "early" returns ). > > I adjusted this as well in the second webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ > > > > > > Best regards , Matthias > > > > > > > -----Original Message----- > > > From: Volker Simonis > > > Sent: Montag, 27. August 2018 17:38 > > > To: Baesken, Matthias > > > Cc: net-dev > > > Subject: Re: [XS] RFR: 8209994: windows: > > > Java_java_net_NetworkInterface_getAll misses releasing interface-list > > > > > > Hi Matthias, > > > > > > the change looks good but I think you should also initialize 'ifList' > > > in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > > > value is undefined and if 'enumInterfaces()' returns with an error > > > without assigning 'ifList' you may end up calling 'free_netif()' with > > > an undefined, non-NULL value. > > > > > > Best regards, > > > Volker > > > > > > On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias > > > wrote: > > > > > > > > Hello, please review this small fix ; > > > > > > > > > > > > > > > > When returning from Java_java_net_NetworkInterface_getAll > (windows > > > version), we have to free resources to avoid leaks. > > > > > > > > In some special cases this is not done . > > > > > > > > > > > > > > > > > > > > > > > > Bug : > > > > > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8209994 > > > > > > > > > > > > > > > > change : > > > > > > > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, Matthias From chris.hegarty at oracle.com Tue Aug 28 14:29:52 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 28 Aug 2018 15:29:52 +0100 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <4f3acead81ab421297f84c7127c51f8d@sap.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> <4f3acead81ab421297f84c7127c51f8d@sap.com> Message-ID: <6A024150-87BF-481A-AB89-A0DCD438ED27@oracle.com> > On 28 Aug 2018, at 15:25, Baesken, Matthias wrote: > > Thanks Volki ! > Can I have a second review please ? Reviewed. -Chris. > Best regards, Matthias > >> -----Original Message----- >> From: Volker Simonis >> Sent: Dienstag, 28. August 2018 14:07 >> To: Baesken, Matthias >> Cc: net-dev ; Chris Hegarty >> ; Brian Burkhalter >> >> Subject: Re: [XS] RFR: 8209994: windows: >> Java_java_net_NetworkInterface_getAll misses releasing interface-list >> >> Thanks for updating the change. >> >> Looks good now! >> >> Regards, >> Volker >> On Tue, Aug 28, 2018 at 9:19 AM Baesken, Matthias >> wrote: >>> >>>> >>>> the change looks good but I think you should also initialize 'ifList' >>>> in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its >>>> value is undefined and if 'enumInterfaces()' returns with an error >>>> without assigning 'ifList' you may end up calling 'free_netif()' with >>>> an undefined, non-NULL value. >>>> >>> >>> >>> Hi Volki, >>> looking at the coding, your sceanario ***should*** not happen ; >> however to be on the safe side it is for sure better to do the initialization >>> you propose. >>> >>> Looking a bit more at the coding, there is a >> Java_java_net_NetworkInterface_getAll_XP that has similar issues (missing >> free_netif calls in case of "early" returns ). >>> I adjusted this as well in the second webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ >>> >>> >>> Best regards , Matthias >>> >>> >>>> -----Original Message----- >>>> From: Volker Simonis >>>> Sent: Montag, 27. August 2018 17:38 >>>> To: Baesken, Matthias >>>> Cc: net-dev >>>> Subject: Re: [XS] RFR: 8209994: windows: >>>> Java_java_net_NetworkInterface_getAll misses releasing interface-list >>>> >>>> Hi Matthias, >>>> >>>> the change looks good but I think you should also initialize 'ifList' >>>> in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its >>>> value is undefined and if 'enumInterfaces()' returns with an error >>>> without assigning 'ifList' you may end up calling 'free_netif()' with >>>> an undefined, non-NULL value. >>>> >>>> Best regards, >>>> Volker >>>> >>>> On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias >>>> wrote: >>>>> >>>>> Hello, please review this small fix ; >>>>> >>>>> >>>>> >>>>> When returning from Java_java_net_NetworkInterface_getAll >> (windows >>>> version), we have to free resources to avoid leaks. >>>>> >>>>> In some special cases this is not done . >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Bug : >>>>> >>>>> >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8209994 >>>>> >>>>> >>>>> >>>>> change : >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks, Matthias From matthias.baesken at sap.com Tue Aug 28 14:30:46 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 28 Aug 2018 14:30:46 +0000 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: <6A024150-87BF-481A-AB89-A0DCD438ED27@oracle.com> References: <805fae76c34642a1bb4a48c15064aa04@sap.com> <4f3acead81ab421297f84c7127c51f8d@sap.com> <6A024150-87BF-481A-AB89-A0DCD438ED27@oracle.com> Message-ID: <37cc474526c1476a9fdd0b22738144b5@sap.com> Thanks Chris ! > -----Original Message----- > From: Chris Hegarty > Sent: Dienstag, 28. August 2018 16:30 > To: Baesken, Matthias > Cc: Volker Simonis ; net-dev dev at openjdk.java.net>; Brian Burkhalter > Subject: Re: [XS] RFR: 8209994: windows: > Java_java_net_NetworkInterface_getAll misses releasing interface-list > > > > On 28 Aug 2018, at 15:25, Baesken, Matthias > wrote: > > > > Thanks Volki ! > > Can I have a second review please ? > > Reviewed. > > -Chris. > > > Best regards, Matthias > > > >> -----Original Message----- > >> From: Volker Simonis > >> Sent: Dienstag, 28. August 2018 14:07 > >> To: Baesken, Matthias > >> Cc: net-dev ; Chris Hegarty > >> ; Brian Burkhalter > >> > >> Subject: Re: [XS] RFR: 8209994: windows: > >> Java_java_net_NetworkInterface_getAll misses releasing interface-list > >> > >> Thanks for updating the change. > >> > >> Looks good now! > >> > >> Regards, > >> Volker > >> On Tue, Aug 28, 2018 at 9:19 AM Baesken, Matthias > >> wrote: > >>> > >>>> > >>>> the change looks good but I think you should also initialize 'ifList' > >>>> in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > >>>> value is undefined and if 'enumInterfaces()' returns with an error > >>>> without assigning 'ifList' you may end up calling 'free_netif()' with > >>>> an undefined, non-NULL value. > >>>> > >>> > >>> > >>> Hi Volki, > >>> looking at the coding, your sceanario ***should*** not happen ; > >> however to be on the safe side it is for sure better to do the initialization > >>> you propose. > >>> > >>> Looking a bit more at the coding, there is a > >> Java_java_net_NetworkInterface_getAll_XP that has similar issues > (missing > >> free_netif calls in case of "early" returns ). > >>> I adjusted this as well in the second webrev : > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ > >>> > >>> > >>> Best regards , Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: Volker Simonis > >>>> Sent: Montag, 27. August 2018 17:38 > >>>> To: Baesken, Matthias > >>>> Cc: net-dev > >>>> Subject: Re: [XS] RFR: 8209994: windows: > >>>> Java_java_net_NetworkInterface_getAll misses releasing interface-list > >>>> > >>>> Hi Matthias, > >>>> > >>>> the change looks good but I think you should also initialize 'ifList' > >>>> in 'Java_java_net_NetworkInterface_getAll()' with NULL otherwise its > >>>> value is undefined and if 'enumInterfaces()' returns with an error > >>>> without assigning 'ifList' you may end up calling 'free_netif()' with > >>>> an undefined, non-NULL value. > >>>> > >>>> Best regards, > >>>> Volker > >>>> > >>>> On Mon, Aug 27, 2018 at 5:13 PM Baesken, Matthias > >>>> wrote: > >>>>> > >>>>> Hello, please review this small fix ; > >>>>> > >>>>> > >>>>> > >>>>> When returning from Java_java_net_NetworkInterface_getAll > >> (windows > >>>> version), we have to free resources to avoid leaks. > >>>>> > >>>>> In some special cases this is not done . > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Bug : > >>>>> > >>>>> > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK-8209994 > >>>>> > >>>>> > >>>>> > >>>>> change : > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8209994/ > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Thanks, Matthias From brian.burkhalter at oracle.com Tue Aug 28 14:49:13 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 28 Aug 2018 07:49:13 -0700 Subject: [XS] RFR: 8209994: windows: Java_java_net_NetworkInterface_getAll misses releasing interface-list In-Reply-To: References: <805fae76c34642a1bb4a48c15064aa04@sap.com> Message-ID: +3 Thanks, Brian On Aug 28, 2018, at 12:19 AM, Baesken, Matthias wrote: > looking at the coding, your sceanario ***should*** not happen ; however to be on the safe side it is for sure better to do the initialization > you propose. > > Looking a bit more at the coding, there is a Java_java_net_NetworkInterface_getAll_XP that has similar issues (missing free_netif calls in case of "early" returns ). > I adjusted this as well in the second webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8209994.1/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias.baesken at sap.com Tue Aug 28 15:56:03 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 28 Aug 2018 15:56:03 +0000 Subject: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: References: <92622c8a0eaa441f90469192d60ccf2d@sap.com> Message-ID: <9c1e1162542b4227898e486fce73a90b@sap.com> > not strictly necessary, since WSAGetLastError() only refers to socket calls I'll try to find out more about this. > However, I think your proposal is fine for code cleanliness sake. Yes , then I will do it for code cleanliness sake (and because MS recommends anyway to get the error immediately). When looking at the WSAGetLastError() usages in the whole JDK Windows code I found 2 strange usages , Both times it looks like WSAGetLastError() is used to get the error of the malloc call , in case this really works, WSAGetLastError might indeed alias in some way to GetLastError : jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c 384static jboolean 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, 386 SOCKETADDRESS *netif, jint timeout) 387{ .............. 396 ReplyBuffer = (VOID *)malloc(ReplySize); 397 if (ReplyBuffer == NULL) { 398 IcmpCloseHandle(hIcmpFile); 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate memory"); 400 return JNI_FALSE; 401 } jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c 306static jboolean 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, 308 SOCKETADDRESS *netif, jint timeout) 309{ ......... 326 ReplyBuffer = (VOID *)malloc(ReplySize); 327 if (ReplyBuffer == NULL) { 328 IcmpCloseHandle(hIcmpFile); 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate memory"); 330 return JNI_FALSE; 331 } Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Dienstag, 28. August 2018 14:23 > To: Baesken, Matthias > Cc: net-dev > Subject: Re: WSAGetLastError usage in > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > Hi Matthias, > > not strictly necessary, since WSAGetLastError() only refers to socket > calls and GetIntField() is unlikely to call socket functions... > However, I think your proposal is fine for code cleanliness sake. > > Best Regards, Thomas > > On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias > wrote: > > Hello, > > > > the MSDN docu about WSAGetLastError warns to get the error-code > > ***immediately*** after occurance. > > > > See : > > > > > > > > https://msdn.microsoft.com/de- > de/library/windows/desktop/ms741580(v=vs.85).aspx > > > > > > > > " ... If a function call's return value indicates that error or other > > relevant data was returned in the error code, > > > > WSAGetLastError should be called immediately ..." > > > > > > > > However in windows SocketInputStream.c , this was not done; we noticed > very > > seldom errors because of this (not reproducible however) so > > > > we had a fix for this in our code base for a long time. > > > > > > > > Should we change this as well in OpenJDK , for example from : > > > > > > > > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > > > > > > > > > > > 120 nread = recv(fd, bufP, len, 0); > > > > 121 if (nread > 0) { > > > > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); > > > > 123 } else { > > > > 124 if (nread < 0) { > > > > 125 // Check if the socket has been closed since we last checked. > > > > 126 // This could be a reason for recv failing. > > > > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { > > > > 128 JNU_ThrowByName(env, "java/net/SocketException", "Socket > > closed"); > > > > 129 } else { > > > > 130 switch (WSAGetLastError()) { > > > > > > > > to : > > > > > > > > 120 nread = recv(fd, bufP, len, 0); > > > > 121 if (nread > 0) { > > > > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); > > > > 123 } else { > > > > 124 if (nread < 0) { > > > > 125 int err = WSAGetLastError(); > > > > ... > > > > switch (err) { > > > > > > > > > > > > Thanks and best regards, Matthias From thomas.stuefe at gmail.com Tue Aug 28 16:32:49 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 28 Aug 2018 18:32:49 +0200 Subject: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <9c1e1162542b4227898e486fce73a90b@sap.com> References: <92622c8a0eaa441f90469192d60ccf2d@sap.com> <9c1e1162542b4227898e486fce73a90b@sap.com> Message-ID: On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias wrote: >> not strictly necessary, since WSAGetLastError() only refers to socket calls > > I'll try to find out more about this. > >> However, I think your proposal is fine for code cleanliness sake. > > Yes , then I will do it for code cleanliness sake (and because MS recommends anyway to get the error immediately). > > When looking at the WSAGetLastError() usages in the whole JDK Windows code I found 2 strange usages , > Both times it looks like WSAGetLastError() is used to get the error of the malloc call , in case this really works, > WSAGetLastError might indeed alias in some way to GetLastError : > > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c > > 384static jboolean > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > 386 SOCKETADDRESS *netif, jint timeout) > 387{ > .............. > 396 ReplyBuffer = (VOID *)malloc(ReplySize); > 397 if (ReplyBuffer == NULL) { > 398 IcmpCloseHandle(hIcmpFile); > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate memory"); > 400 return JNI_FALSE; > 401 } > Thats plain wrong. One should use errno instead, however, the Windows variant of NET_ThrowNew expects a WSA error code for the error number, it does not handle errno. What I would do to keep the change simple is just passing 0 or -1 as errorNum argument to NET_ThrowNew - in that case it will say "Unspecified socket error" in the exception. Just make sure that 0 or -1, whatever you use, is not one of the WSAxxxxx constants, see winsock_errors[] table in net_util_md.c. > > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c > > 306static jboolean > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > 308 SOCKETADDRESS *netif, jint timeout) > 309{ > ......... > 326 ReplyBuffer = (VOID *)malloc(ReplySize); > 327 if (ReplyBuffer == NULL) { > 328 IcmpCloseHandle(hIcmpFile); > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate memory"); > 330 return JNI_FALSE; > 331 } > > Yes, this may be wrong if WSAGetLastError() uses GetLastError() internally, so I would move the WSAGetLastError() call up. Good catches! ..Thomas > Best regards, Matthias > > > > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Dienstag, 28. August 2018 14:23 >> To: Baesken, Matthias >> Cc: net-dev >> Subject: Re: WSAGetLastError usage in >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> Hi Matthias, >> >> not strictly necessary, since WSAGetLastError() only refers to socket >> calls and GetIntField() is unlikely to call socket functions... >> However, I think your proposal is fine for code cleanliness sake. >> >> Best Regards, Thomas >> >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias >> wrote: >> > Hello, >> > >> > the MSDN docu about WSAGetLastError warns to get the error-code >> > ***immediately*** after occurance. >> > >> > See : >> > >> > >> > >> > https://msdn.microsoft.com/de- >> de/library/windows/desktop/ms741580(v=vs.85).aspx >> > >> > >> > >> > " ... If a function call's return value indicates that error or other >> > relevant data was returned in the error code, >> > >> > WSAGetLastError should be called immediately ..." >> > >> > >> > >> > However in windows SocketInputStream.c , this was not done; we noticed >> very >> > seldom errors because of this (not reproducible however) so >> > >> > we had a fix for this in our code base for a long time. >> > >> > >> > >> > Should we change this as well in OpenJDK , for example from : >> > >> > >> > >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> > >> > >> > >> > >> > >> > 120 nread = recv(fd, bufP, len, 0); >> > >> > 121 if (nread > 0) { >> > >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); >> > >> > 123 } else { >> > >> > 124 if (nread < 0) { >> > >> > 125 // Check if the socket has been closed since we last checked. >> > >> > 126 // This could be a reason for recv failing. >> > >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { >> > >> > 128 JNU_ThrowByName(env, "java/net/SocketException", "Socket >> > closed"); >> > >> > 129 } else { >> > >> > 130 switch (WSAGetLastError()) { >> > >> > >> > >> > to : >> > >> > >> > >> > 120 nread = recv(fd, bufP, len, 0); >> > >> > 121 if (nread > 0) { >> > >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte *)bufP); >> > >> > 123 } else { >> > >> > 124 if (nread < 0) { >> > >> > 125 int err = WSAGetLastError(); >> > >> > ... >> > >> > switch (err) { >> > >> > >> > >> > >> > >> > Thanks and best regards, Matthias From matthias.baesken at sap.com Wed Aug 29 07:46:23 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 29 Aug 2018 07:46:23 +0000 Subject: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c Message-ID: > What I would do to keep the change simple is just passing 0 or -1 as > errorNum argument to NET_ThrowNew - in that case it will say > "Unspecified socket error" in the exception. Just make sure that 0 or > -1, whatever you use, is not one of the WSAxxxxx constants Hi Thomas, according to https://docs.microsoft.com/en-us/windows/desktop/winsock/windows-sockets-error-codes-2 both 0 and -1 are fine . Instead of using NET_ThrowNew(env, -1, "Unable to allocate memory"); We could also call : JNU_ThrowOutOfMemoryError(env, "Native heap allocation failed"); (or would this be problematic because of compatibility?). Btw I opened a bug : https://bugs.openjdk.java.net/browse/JDK-8210147 for the issues . Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Dienstag, 28. August 2018 18:33 > To: Baesken, Matthias > Cc: net-dev > Subject: Re: WSAGetLastError usage in > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias > wrote: > >> not strictly necessary, since WSAGetLastError() only refers to socket calls > > > > I'll try to find out more about this. > > > >> However, I think your proposal is fine for code cleanliness sake. > > > > Yes , then I will do it for code cleanliness sake (and because MS > recommends anyway to get the error immediately). > > > > When looking at the WSAGetLastError() usages in the whole JDK > Windows code I found 2 strange usages , > > Both times it looks like WSAGetLastError() is used to get the error of > the malloc call , in case this really works, > > WSAGetLastError might indeed alias in some way to GetLastError : > > > > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c > > > > 384static jboolean > > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > > 386 SOCKETADDRESS *netif, jint timeout) > > 387{ > > .............. > > 396 ReplyBuffer = (VOID *)malloc(ReplySize); > > 397 if (ReplyBuffer == NULL) { > > 398 IcmpCloseHandle(hIcmpFile); > > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > memory"); > > 400 return JNI_FALSE; > > 401 } > > > > Thats plain wrong. > > One should use errno instead, however, the Windows variant of > NET_ThrowNew expects a WSA error code for the error number, it does > not handle errno. > > What I would do to keep the change simple is just passing 0 or -1 as > errorNum argument to NET_ThrowNew - in that case it will say > "Unspecified socket error" in the exception. Just make sure that 0 or > -1, whatever you use, is not one of the WSAxxxxx constants, see > winsock_errors[] table in net_util_md.c. > > > > > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c > > > > 306static jboolean > > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > > 308 SOCKETADDRESS *netif, jint timeout) > > 309{ > > ......... > > 326 ReplyBuffer = (VOID *)malloc(ReplySize); > > 327 if (ReplyBuffer == NULL) { > > 328 IcmpCloseHandle(hIcmpFile); > > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > memory"); > > 330 return JNI_FALSE; > > 331 } > > > > > > Yes, this may be wrong if WSAGetLastError() uses GetLastError() > internally, so I would move the WSAGetLastError() call up. > > Good catches! > > ..Thomas > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: Thomas St?fe > >> Sent: Dienstag, 28. August 2018 14:23 > >> To: Baesken, Matthias > >> Cc: net-dev > >> Subject: Re: WSAGetLastError usage in > >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> > >> Hi Matthias, > >> > >> not strictly necessary, since WSAGetLastError() only refers to socket > >> calls and GetIntField() is unlikely to call socket functions... > >> However, I think your proposal is fine for code cleanliness sake. > >> > >> Best Regards, Thomas > >> > >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias > >> wrote: > >> > Hello, > >> > > >> > the MSDN docu about WSAGetLastError warns to get the error-code > >> > ***immediately*** after occurance. > >> > > >> > See : > >> > > >> > > >> > > >> > https://msdn.microsoft.com/de- > >> de/library/windows/desktop/ms741580(v=vs.85).aspx > >> > > >> > > >> > > >> > " ... If a function call's return value indicates that error or other > >> > relevant data was returned in the error code, > >> > > >> > WSAGetLastError should be called immediately ..." > >> > > >> > > >> > > >> > However in windows SocketInputStream.c , this was not done; we > noticed > >> very > >> > seldom errors because of this (not reproducible however) so > >> > > >> > we had a fix for this in our code base for a long time. > >> > > >> > > >> > > >> > Should we change this as well in OpenJDK , for example from : > >> > > >> > > >> > > >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> > > >> > > >> > > >> > > >> > > >> > 120 nread = recv(fd, bufP, len, 0); > >> > > >> > 121 if (nread > 0) { > >> > > >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > *)bufP); > >> > > >> > 123 } else { > >> > > >> > 124 if (nread < 0) { > >> > > >> > 125 // Check if the socket has been closed since we last checked. > >> > > >> > 126 // This could be a reason for recv failing. > >> > > >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { > >> > > >> > 128 JNU_ThrowByName(env, "java/net/SocketException", > "Socket > >> > closed"); > >> > > >> > 129 } else { > >> > > >> > 130 switch (WSAGetLastError()) { > >> > > >> > > >> > > >> > to : > >> > > >> > > >> > > >> > 120 nread = recv(fd, bufP, len, 0); > >> > > >> > 121 if (nread > 0) { > >> > > >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > *)bufP); > >> > > >> > 123 } else { > >> > > >> > 124 if (nread < 0) { > >> > > >> > 125 int err = WSAGetLastError(); > >> > > >> > ... > >> > > >> > switch (err) { > >> > > >> > > >> > > >> > > >> > > >> > Thanks and best regards, Matthias From thomas.stuefe at gmail.com Wed Aug 29 09:22:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 29 Aug 2018 11:22:19 +0200 Subject: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: References: Message-ID: Hi Matthias, On Wed, Aug 29, 2018 at 9:46 AM, Baesken, Matthias wrote: > >> What I would do to keep the change simple is just passing 0 or -1 as >> errorNum argument to NET_ThrowNew - in that case it will say >> "Unspecified socket error" in the exception. Just make sure that 0 or >> -1, whatever you use, is not one of the WSAxxxxx constants > > Hi Thomas, according to > > https://docs.microsoft.com/en-us/windows/desktop/winsock/windows-sockets-error-codes-2 > > both 0 and -1 are fine . > > Instead of using > > NET_ThrowNew(env, -1, "Unable to allocate memory"); > > We could also call : > > JNU_ThrowOutOfMemoryError(env, "Native heap allocation failed"); > > (or would this be problematic because of compatibility?). > > That should be fine too. Best Regards, Thomas > > Btw I opened a bug : > > https://bugs.openjdk.java.net/browse/JDK-8210147 > > for the issues . > > Best regards, Matthias > > > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Dienstag, 28. August 2018 18:33 >> To: Baesken, Matthias >> Cc: net-dev >> Subject: Re: WSAGetLastError usage in >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias >> wrote: >> >> not strictly necessary, since WSAGetLastError() only refers to socket calls >> > >> > I'll try to find out more about this. >> > >> >> However, I think your proposal is fine for code cleanliness sake. >> > >> > Yes , then I will do it for code cleanliness sake (and because MS >> recommends anyway to get the error immediately). >> > >> > When looking at the WSAGetLastError() usages in the whole JDK >> Windows code I found 2 strange usages , >> > Both times it looks like WSAGetLastError() is used to get the error of >> the malloc call , in case this really works, >> > WSAGetLastError might indeed alias in some way to GetLastError : >> > >> > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c >> > >> > 384static jboolean >> > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, >> > 386 SOCKETADDRESS *netif, jint timeout) >> > 387{ >> > .............. >> > 396 ReplyBuffer = (VOID *)malloc(ReplySize); >> > 397 if (ReplyBuffer == NULL) { >> > 398 IcmpCloseHandle(hIcmpFile); >> > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate >> memory"); >> > 400 return JNI_FALSE; >> > 401 } >> > >> >> Thats plain wrong. >> >> One should use errno instead, however, the Windows variant of >> NET_ThrowNew expects a WSA error code for the error number, it does >> not handle errno. >> >> What I would do to keep the change simple is just passing 0 or -1 as >> errorNum argument to NET_ThrowNew - in that case it will say >> "Unspecified socket error" in the exception. Just make sure that 0 or >> -1, whatever you use, is not one of the WSAxxxxx constants, see >> winsock_errors[] table in net_util_md.c. >> >> > >> > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c >> > >> > 306static jboolean >> > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, >> > 308 SOCKETADDRESS *netif, jint timeout) >> > 309{ >> > ......... >> > 326 ReplyBuffer = (VOID *)malloc(ReplySize); >> > 327 if (ReplyBuffer == NULL) { >> > 328 IcmpCloseHandle(hIcmpFile); >> > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate >> memory"); >> > 330 return JNI_FALSE; >> > 331 } >> > >> > >> >> Yes, this may be wrong if WSAGetLastError() uses GetLastError() >> internally, so I would move the WSAGetLastError() call up. >> >> Good catches! >> >> ..Thomas >> >> > Best regards, Matthias >> > >> > >> > >> > >> >> -----Original Message----- >> >> From: Thomas St?fe >> >> Sent: Dienstag, 28. August 2018 14:23 >> >> To: Baesken, Matthias >> >> Cc: net-dev >> >> Subject: Re: WSAGetLastError usage in >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> >> >> Hi Matthias, >> >> >> >> not strictly necessary, since WSAGetLastError() only refers to socket >> >> calls and GetIntField() is unlikely to call socket functions... >> >> However, I think your proposal is fine for code cleanliness sake. >> >> >> >> Best Regards, Thomas >> >> >> >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias >> >> wrote: >> >> > Hello, >> >> > >> >> > the MSDN docu about WSAGetLastError warns to get the error-code >> >> > ***immediately*** after occurance. >> >> > >> >> > See : >> >> > >> >> > >> >> > >> >> > https://msdn.microsoft.com/de- >> >> de/library/windows/desktop/ms741580(v=vs.85).aspx >> >> > >> >> > >> >> > >> >> > " ... If a function call's return value indicates that error or other >> >> > relevant data was returned in the error code, >> >> > >> >> > WSAGetLastError should be called immediately ..." >> >> > >> >> > >> >> > >> >> > However in windows SocketInputStream.c , this was not done; we >> noticed >> >> very >> >> > seldom errors because of this (not reproducible however) so >> >> > >> >> > we had a fix for this in our code base for a long time. >> >> > >> >> > >> >> > >> >> > Should we change this as well in OpenJDK , for example from : >> >> > >> >> > >> >> > >> >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > 120 nread = recv(fd, bufP, len, 0); >> >> > >> >> > 121 if (nread > 0) { >> >> > >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte >> *)bufP); >> >> > >> >> > 123 } else { >> >> > >> >> > 124 if (nread < 0) { >> >> > >> >> > 125 // Check if the socket has been closed since we last checked. >> >> > >> >> > 126 // This could be a reason for recv failing. >> >> > >> >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { >> >> > >> >> > 128 JNU_ThrowByName(env, "java/net/SocketException", >> "Socket >> >> > closed"); >> >> > >> >> > 129 } else { >> >> > >> >> > 130 switch (WSAGetLastError()) { >> >> > >> >> > >> >> > >> >> > to : >> >> > >> >> > >> >> > >> >> > 120 nread = recv(fd, bufP, len, 0); >> >> > >> >> > 121 if (nread > 0) { >> >> > >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte >> *)bufP); >> >> > >> >> > 123 } else { >> >> > >> >> > 124 if (nread < 0) { >> >> > >> >> > 125 int err = WSAGetLastError(); >> >> > >> >> > ... >> >> > >> >> > switch (err) { >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > Thanks and best regards, Matthias From matthias.baesken at sap.com Wed Aug 29 13:15:26 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 29 Aug 2018 13:15:26 +0000 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c Message-ID: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> Please review this change, it adjusts some usages of WSAGetLastError in windows net - coding. Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8210147/ Bug : https://bugs.openjdk.java.net/browse/JDK-8210147 Thanks , Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Mittwoch, 29. August 2018 11:22 > To: Baesken, Matthias > Cc: net-dev > Subject: Re: 8210147: adjust some WSAGetLastError usages in windows > network coding - was : RE: WSAGetLastError usage in > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > Hi Matthias, > > On Wed, Aug 29, 2018 at 9:46 AM, Baesken, Matthias > wrote: > > > >> What I would do to keep the change simple is just passing 0 or -1 as > >> errorNum argument to NET_ThrowNew - in that case it will say > >> "Unspecified socket error" in the exception. Just make sure that 0 or > >> -1, whatever you use, is not one of the WSAxxxxx constants > > > > Hi Thomas, according to > > > > https://docs.microsoft.com/en-us/windows/desktop/winsock/windows- > sockets-error-codes-2 > > > > both 0 and -1 are fine . > > > > Instead of using > > > > NET_ThrowNew(env, -1, "Unable to allocate memory"); > > > > We could also call : > > > > JNU_ThrowOutOfMemoryError(env, "Native heap allocation failed"); > > > > (or would this be problematic because of compatibility?). > > > > > > That should be fine too. > > Best Regards, Thomas > > > > > Btw I opened a bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8210147 > > > > for the issues . > > > > Best regards, Matthias > > > > > > > >> -----Original Message----- > >> From: Thomas St?fe > >> Sent: Dienstag, 28. August 2018 18:33 > >> To: Baesken, Matthias > >> Cc: net-dev > >> Subject: Re: WSAGetLastError usage in > >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> > >> On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias > >> wrote: > >> >> not strictly necessary, since WSAGetLastError() only refers to socket > calls > >> > > >> > I'll try to find out more about this. > >> > > >> >> However, I think your proposal is fine for code cleanliness sake. > >> > > >> > Yes , then I will do it for code cleanliness sake (and because MS > >> recommends anyway to get the error immediately). > >> > > >> > When looking at the WSAGetLastError() usages in the whole JDK > >> Windows code I found 2 strange usages , > >> > Both times it looks like WSAGetLastError() is used to get the error of > >> the malloc call , in case this really works, > >> > WSAGetLastError might indeed alias in some way to GetLastError : > >> > > >> > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c > >> > > >> > 384static jboolean > >> > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > >> > 386 SOCKETADDRESS *netif, jint timeout) > >> > 387{ > >> > .............. > >> > 396 ReplyBuffer = (VOID *)malloc(ReplySize); > >> > 397 if (ReplyBuffer == NULL) { > >> > 398 IcmpCloseHandle(hIcmpFile); > >> > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > >> memory"); > >> > 400 return JNI_FALSE; > >> > 401 } > >> > > >> > >> Thats plain wrong. > >> > >> One should use errno instead, however, the Windows variant of > >> NET_ThrowNew expects a WSA error code for the error number, it does > >> not handle errno. > >> > >> What I would do to keep the change simple is just passing 0 or -1 as > >> errorNum argument to NET_ThrowNew - in that case it will say > >> "Unspecified socket error" in the exception. Just make sure that 0 or > >> -1, whatever you use, is not one of the WSAxxxxx constants, see > >> winsock_errors[] table in net_util_md.c. > >> > >> > > >> > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c > >> > > >> > 306static jboolean > >> > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > >> > 308 SOCKETADDRESS *netif, jint timeout) > >> > 309{ > >> > ......... > >> > 326 ReplyBuffer = (VOID *)malloc(ReplySize); > >> > 327 if (ReplyBuffer == NULL) { > >> > 328 IcmpCloseHandle(hIcmpFile); > >> > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > >> memory"); > >> > 330 return JNI_FALSE; > >> > 331 } > >> > > >> > > >> > >> Yes, this may be wrong if WSAGetLastError() uses GetLastError() > >> internally, so I would move the WSAGetLastError() call up. > >> > >> Good catches! > >> > >> ..Thomas > >> > >> > Best regards, Matthias > >> > > >> > > >> > > >> > > >> >> -----Original Message----- > >> >> From: Thomas St?fe > >> >> Sent: Dienstag, 28. August 2018 14:23 > >> >> To: Baesken, Matthias > >> >> Cc: net-dev > >> >> Subject: Re: WSAGetLastError usage in > >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> >> > >> >> Hi Matthias, > >> >> > >> >> not strictly necessary, since WSAGetLastError() only refers to socket > >> >> calls and GetIntField() is unlikely to call socket functions... > >> >> However, I think your proposal is fine for code cleanliness sake. > >> >> > >> >> Best Regards, Thomas > >> >> > >> >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias > >> >> wrote: > >> >> > Hello, > >> >> > > >> >> > the MSDN docu about WSAGetLastError warns to get the error-code > >> >> > ***immediately*** after occurance. > >> >> > > >> >> > See : > >> >> > > >> >> > > >> >> > > >> >> > https://msdn.microsoft.com/de- > >> >> de/library/windows/desktop/ms741580(v=vs.85).aspx > >> >> > > >> >> > > >> >> > > >> >> > " ... If a function call's return value indicates that error or other > >> >> > relevant data was returned in the error code, > >> >> > > >> >> > WSAGetLastError should be called immediately ..." > >> >> > > >> >> > > >> >> > > >> >> > However in windows SocketInputStream.c , this was not done; we > >> noticed > >> >> very > >> >> > seldom errors because of this (not reproducible however) so > >> >> > > >> >> > we had a fix for this in our code base for a long time. > >> >> > > >> >> > > >> >> > > >> >> > Should we change this as well in OpenJDK , for example from : > >> >> > > >> >> > > >> >> > > >> >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > 120 nread = recv(fd, bufP, len, 0); > >> >> > > >> >> > 121 if (nread > 0) { > >> >> > > >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > >> *)bufP); > >> >> > > >> >> > 123 } else { > >> >> > > >> >> > 124 if (nread < 0) { > >> >> > > >> >> > 125 // Check if the socket has been closed since we last > checked. > >> >> > > >> >> > 126 // This could be a reason for recv failing. > >> >> > > >> >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { > >> >> > > >> >> > 128 JNU_ThrowByName(env, "java/net/SocketException", > >> "Socket > >> >> > closed"); > >> >> > > >> >> > 129 } else { > >> >> > > >> >> > 130 switch (WSAGetLastError()) { > >> >> > > >> >> > > >> >> > > >> >> > to : > >> >> > > >> >> > > >> >> > > >> >> > 120 nread = recv(fd, bufP, len, 0); > >> >> > > >> >> > 121 if (nread > 0) { > >> >> > > >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > >> *)bufP); > >> >> > > >> >> > 123 } else { > >> >> > > >> >> > 124 if (nread < 0) { > >> >> > > >> >> > 125 int err = WSAGetLastError(); > >> >> > > >> >> > ... > >> >> > > >> >> > switch (err) { > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > Thanks and best regards, Matthias From thomas.stuefe at gmail.com Wed Aug 29 13:22:24 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 29 Aug 2018 15:22:24 +0200 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> References: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> Message-ID: Looks fine. Regards, Thomas On Wed, Aug 29, 2018 at 3:15 PM, Baesken, Matthias wrote: > Please review this change, it adjusts some usages of WSAGetLastError in windows net - coding. > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8210147/ > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8210147 > > Thanks , Matthias > > > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Mittwoch, 29. August 2018 11:22 >> To: Baesken, Matthias >> Cc: net-dev >> Subject: Re: 8210147: adjust some WSAGetLastError usages in windows >> network coding - was : RE: WSAGetLastError usage in >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> Hi Matthias, >> >> On Wed, Aug 29, 2018 at 9:46 AM, Baesken, Matthias >> wrote: >> > >> >> What I would do to keep the change simple is just passing 0 or -1 as >> >> errorNum argument to NET_ThrowNew - in that case it will say >> >> "Unspecified socket error" in the exception. Just make sure that 0 or >> >> -1, whatever you use, is not one of the WSAxxxxx constants >> > >> > Hi Thomas, according to >> > >> > https://docs.microsoft.com/en-us/windows/desktop/winsock/windows- >> sockets-error-codes-2 >> > >> > both 0 and -1 are fine . >> > >> > Instead of using >> > >> > NET_ThrowNew(env, -1, "Unable to allocate memory"); >> > >> > We could also call : >> > >> > JNU_ThrowOutOfMemoryError(env, "Native heap allocation failed"); >> > >> > (or would this be problematic because of compatibility?). >> > >> > >> >> That should be fine too. >> >> Best Regards, Thomas >> >> > >> > Btw I opened a bug : >> > >> > https://bugs.openjdk.java.net/browse/JDK-8210147 >> > >> > for the issues . >> > >> > Best regards, Matthias >> > >> > >> > >> >> -----Original Message----- >> >> From: Thomas St?fe >> >> Sent: Dienstag, 28. August 2018 18:33 >> >> To: Baesken, Matthias >> >> Cc: net-dev >> >> Subject: Re: WSAGetLastError usage in >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> >> >> On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias >> >> wrote: >> >> >> not strictly necessary, since WSAGetLastError() only refers to socket >> calls >> >> > >> >> > I'll try to find out more about this. >> >> > >> >> >> However, I think your proposal is fine for code cleanliness sake. >> >> > >> >> > Yes , then I will do it for code cleanliness sake (and because MS >> >> recommends anyway to get the error immediately). >> >> > >> >> > When looking at the WSAGetLastError() usages in the whole JDK >> >> Windows code I found 2 strange usages , >> >> > Both times it looks like WSAGetLastError() is used to get the error of >> >> the malloc call , in case this really works, >> >> > WSAGetLastError might indeed alias in some way to GetLastError : >> >> > >> >> > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c >> >> > >> >> > 384static jboolean >> >> > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, >> >> > 386 SOCKETADDRESS *netif, jint timeout) >> >> > 387{ >> >> > .............. >> >> > 396 ReplyBuffer = (VOID *)malloc(ReplySize); >> >> > 397 if (ReplyBuffer == NULL) { >> >> > 398 IcmpCloseHandle(hIcmpFile); >> >> > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate >> >> memory"); >> >> > 400 return JNI_FALSE; >> >> > 401 } >> >> > >> >> >> >> Thats plain wrong. >> >> >> >> One should use errno instead, however, the Windows variant of >> >> NET_ThrowNew expects a WSA error code for the error number, it does >> >> not handle errno. >> >> >> >> What I would do to keep the change simple is just passing 0 or -1 as >> >> errorNum argument to NET_ThrowNew - in that case it will say >> >> "Unspecified socket error" in the exception. Just make sure that 0 or >> >> -1, whatever you use, is not one of the WSAxxxxx constants, see >> >> winsock_errors[] table in net_util_md.c. >> >> >> >> > >> >> > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c >> >> > >> >> > 306static jboolean >> >> > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, >> >> > 308 SOCKETADDRESS *netif, jint timeout) >> >> > 309{ >> >> > ......... >> >> > 326 ReplyBuffer = (VOID *)malloc(ReplySize); >> >> > 327 if (ReplyBuffer == NULL) { >> >> > 328 IcmpCloseHandle(hIcmpFile); >> >> > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate >> >> memory"); >> >> > 330 return JNI_FALSE; >> >> > 331 } >> >> > >> >> > >> >> >> >> Yes, this may be wrong if WSAGetLastError() uses GetLastError() >> >> internally, so I would move the WSAGetLastError() call up. >> >> >> >> Good catches! >> >> >> >> ..Thomas >> >> >> >> > Best regards, Matthias >> >> > >> >> > >> >> > >> >> > >> >> >> -----Original Message----- >> >> >> From: Thomas St?fe >> >> >> Sent: Dienstag, 28. August 2018 14:23 >> >> >> To: Baesken, Matthias >> >> >> Cc: net-dev >> >> >> Subject: Re: WSAGetLastError usage in >> >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> >> >> >> >> Hi Matthias, >> >> >> >> >> >> not strictly necessary, since WSAGetLastError() only refers to socket >> >> >> calls and GetIntField() is unlikely to call socket functions... >> >> >> However, I think your proposal is fine for code cleanliness sake. >> >> >> >> >> >> Best Regards, Thomas >> >> >> >> >> >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias >> >> >> wrote: >> >> >> > Hello, >> >> >> > >> >> >> > the MSDN docu about WSAGetLastError warns to get the error-code >> >> >> > ***immediately*** after occurance. >> >> >> > >> >> >> > See : >> >> >> > >> >> >> > >> >> >> > >> >> >> > https://msdn.microsoft.com/de- >> >> >> de/library/windows/desktop/ms741580(v=vs.85).aspx >> >> >> > >> >> >> > >> >> >> > >> >> >> > " ... If a function call's return value indicates that error or other >> >> >> > relevant data was returned in the error code, >> >> >> > >> >> >> > WSAGetLastError should be called immediately ..." >> >> >> > >> >> >> > >> >> >> > >> >> >> > However in windows SocketInputStream.c , this was not done; we >> >> noticed >> >> >> very >> >> >> > seldom errors because of this (not reproducible however) so >> >> >> > >> >> >> > we had a fix for this in our code base for a long time. >> >> >> > >> >> >> > >> >> >> > >> >> >> > Should we change this as well in OpenJDK , for example from : >> >> >> > >> >> >> > >> >> >> > >> >> >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > 120 nread = recv(fd, bufP, len, 0); >> >> >> > >> >> >> > 121 if (nread > 0) { >> >> >> > >> >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte >> >> *)bufP); >> >> >> > >> >> >> > 123 } else { >> >> >> > >> >> >> > 124 if (nread < 0) { >> >> >> > >> >> >> > 125 // Check if the socket has been closed since we last >> checked. >> >> >> > >> >> >> > 126 // This could be a reason for recv failing. >> >> >> > >> >> >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { >> >> >> > >> >> >> > 128 JNU_ThrowByName(env, "java/net/SocketException", >> >> "Socket >> >> >> > closed"); >> >> >> > >> >> >> > 129 } else { >> >> >> > >> >> >> > 130 switch (WSAGetLastError()) { >> >> >> > >> >> >> > >> >> >> > >> >> >> > to : >> >> >> > >> >> >> > >> >> >> > >> >> >> > 120 nread = recv(fd, bufP, len, 0); >> >> >> > >> >> >> > 121 if (nread > 0) { >> >> >> > >> >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte >> >> *)bufP); >> >> >> > >> >> >> > 123 } else { >> >> >> > >> >> >> > 124 if (nread < 0) { >> >> >> > >> >> >> > 125 int err = WSAGetLastError(); >> >> >> > >> >> >> > ... >> >> >> > >> >> >> > switch (err) { >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > Thanks and best regards, Matthias From matthias.baesken at sap.com Wed Aug 29 13:25:12 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 29 Aug 2018 13:25:12 +0000 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: References: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> Message-ID: <87f37123fb8f4674baa668f04a87bdbc@sap.com> Thanks Thomas ! Can I have a second review please ? Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Mittwoch, 29. August 2018 15:22 > To: Baesken, Matthias > Cc: net-dev > Subject: Re: RFR [XS]: 8210147: adjust some WSAGetLastError usages in > windows network coding - was : RE: WSAGetLastError usage in > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > Looks fine. > > Regards, Thomas > > On Wed, Aug 29, 2018 at 3:15 PM, Baesken, Matthias > wrote: > > Please review this change, it adjusts some usages of WSAGetLastError > in windows net - coding. > > > > Change : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8210147/ > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8210147 > > > > Thanks , Matthias > > > > > > > >> -----Original Message----- > >> From: Thomas St?fe > >> Sent: Mittwoch, 29. August 2018 11:22 > >> To: Baesken, Matthias > >> Cc: net-dev > >> Subject: Re: 8210147: adjust some WSAGetLastError usages in windows > >> network coding - was : RE: WSAGetLastError usage in > >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> > >> Hi Matthias, > >> > >> On Wed, Aug 29, 2018 at 9:46 AM, Baesken, Matthias > >> wrote: > >> > > >> >> What I would do to keep the change simple is just passing 0 or -1 as > >> >> errorNum argument to NET_ThrowNew - in that case it will say > >> >> "Unspecified socket error" in the exception. Just make sure that 0 or > >> >> -1, whatever you use, is not one of the WSAxxxxx constants > >> > > >> > Hi Thomas, according to > >> > > >> > https://docs.microsoft.com/en- > us/windows/desktop/winsock/windows- > >> sockets-error-codes-2 > >> > > >> > both 0 and -1 are fine . > >> > > >> > Instead of using > >> > > >> > NET_ThrowNew(env, -1, "Unable to allocate memory"); > >> > > >> > We could also call : > >> > > >> > JNU_ThrowOutOfMemoryError(env, "Native heap allocation failed"); > >> > > >> > (or would this be problematic because of compatibility?). > >> > > >> > > >> > >> That should be fine too. > >> > >> Best Regards, Thomas > >> > >> > > >> > Btw I opened a bug : > >> > > >> > https://bugs.openjdk.java.net/browse/JDK-8210147 > >> > > >> > for the issues . > >> > > >> > Best regards, Matthias > >> > > >> > > >> > > >> >> -----Original Message----- > >> >> From: Thomas St?fe > >> >> Sent: Dienstag, 28. August 2018 18:33 > >> >> To: Baesken, Matthias > >> >> Cc: net-dev > >> >> Subject: Re: WSAGetLastError usage in > >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> >> > >> >> On Tue, Aug 28, 2018 at 5:56 PM, Baesken, Matthias > >> >> wrote: > >> >> >> not strictly necessary, since WSAGetLastError() only refers to socket > >> calls > >> >> > > >> >> > I'll try to find out more about this. > >> >> > > >> >> >> However, I think your proposal is fine for code cleanliness sake. > >> >> > > >> >> > Yes , then I will do it for code cleanliness sake (and because MS > >> >> recommends anyway to get the error immediately). > >> >> > > >> >> > When looking at the WSAGetLastError() usages in the whole JDK > >> >> Windows code I found 2 strange usages , > >> >> > Both times it looks like WSAGetLastError() is used to get the > error of > >> >> the malloc call , in case this really works, > >> >> > WSAGetLastError might indeed alias in some way to GetLastError : > >> >> > > >> >> > jdk/src/java.base/windows/native/libnet/Inet6AddressImpl.c > >> >> > > >> >> > 384static jboolean > >> >> > 385ping6(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > >> >> > 386 SOCKETADDRESS *netif, jint timeout) > >> >> > 387{ > >> >> > .............. > >> >> > 396 ReplyBuffer = (VOID *)malloc(ReplySize); > >> >> > 397 if (ReplyBuffer == NULL) { > >> >> > 398 IcmpCloseHandle(hIcmpFile); > >> >> > 399 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > >> >> memory"); > >> >> > 400 return JNI_FALSE; > >> >> > 401 } > >> >> > > >> >> > >> >> Thats plain wrong. > >> >> > >> >> One should use errno instead, however, the Windows variant of > >> >> NET_ThrowNew expects a WSA error code for the error number, it > does > >> >> not handle errno. > >> >> > >> >> What I would do to keep the change simple is just passing 0 or -1 as > >> >> errorNum argument to NET_ThrowNew - in that case it will say > >> >> "Unspecified socket error" in the exception. Just make sure that 0 or > >> >> -1, whatever you use, is not one of the WSAxxxxx constants, see > >> >> winsock_errors[] table in net_util_md.c. > >> >> > >> >> > > >> >> > jdk/src/java.base/windows/native/libnet/Inet4AddressImpl.c > >> >> > > >> >> > 306static jboolean > >> >> > 307ping4(JNIEnv *env, HANDLE hIcmpFile, SOCKETADDRESS *sa, > >> >> > 308 SOCKETADDRESS *netif, jint timeout) > >> >> > 309{ > >> >> > ......... > >> >> > 326 ReplyBuffer = (VOID *)malloc(ReplySize); > >> >> > 327 if (ReplyBuffer == NULL) { > >> >> > 328 IcmpCloseHandle(hIcmpFile); > >> >> > 329 NET_ThrowNew(env, WSAGetLastError(), "Unable to allocate > >> >> memory"); > >> >> > 330 return JNI_FALSE; > >> >> > 331 } > >> >> > > >> >> > > >> >> > >> >> Yes, this may be wrong if WSAGetLastError() uses GetLastError() > >> >> internally, so I would move the WSAGetLastError() call up. > >> >> > >> >> Good catches! > >> >> > >> >> ..Thomas > >> >> > >> >> > Best regards, Matthias > >> >> > > >> >> > > >> >> > > >> >> > > >> >> >> -----Original Message----- > >> >> >> From: Thomas St?fe > >> >> >> Sent: Dienstag, 28. August 2018 14:23 > >> >> >> To: Baesken, Matthias > >> >> >> Cc: net-dev > >> >> >> Subject: Re: WSAGetLastError usage in > >> >> >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> >> >> > >> >> >> Hi Matthias, > >> >> >> > >> >> >> not strictly necessary, since WSAGetLastError() only refers to socket > >> >> >> calls and GetIntField() is unlikely to call socket functions... > >> >> >> However, I think your proposal is fine for code cleanliness sake. > >> >> >> > >> >> >> Best Regards, Thomas > >> >> >> > >> >> >> On Tue, Aug 28, 2018 at 2:13 PM, Baesken, Matthias > >> >> >> wrote: > >> >> >> > Hello, > >> >> >> > > >> >> >> > the MSDN docu about WSAGetLastError warns to get the error- > code > >> >> >> > ***immediately*** after occurance. > >> >> >> > > >> >> >> > See : > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > https://msdn.microsoft.com/de- > >> >> >> de/library/windows/desktop/ms741580(v=vs.85).aspx > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > " ... If a function call's return value indicates that error or other > >> >> >> > relevant data was returned in the error code, > >> >> >> > > >> >> >> > WSAGetLastError should be called immediately ..." > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > However in windows SocketInputStream.c , this was not done; > we > >> >> noticed > >> >> >> very > >> >> >> > seldom errors because of this (not reproducible however) so > >> >> >> > > >> >> >> > we had a fix for this in our code base for a long time. > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > Should we change this as well in OpenJDK , for example from : > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > 120 nread = recv(fd, bufP, len, 0); > >> >> >> > > >> >> >> > 121 if (nread > 0) { > >> >> >> > > >> >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > >> >> *)bufP); > >> >> >> > > >> >> >> > 123 } else { > >> >> >> > > >> >> >> > 124 if (nread < 0) { > >> >> >> > > >> >> >> > 125 // Check if the socket has been closed since we last > >> checked. > >> >> >> > > >> >> >> > 126 // This could be a reason for recv failing. > >> >> >> > > >> >> >> > 127 if ((*env)->GetIntField(env, fdObj, IO_fd_fdID) == -1) { > >> >> >> > > >> >> >> > 128 JNU_ThrowByName(env, > "java/net/SocketException", > >> >> "Socket > >> >> >> > closed"); > >> >> >> > > >> >> >> > 129 } else { > >> >> >> > > >> >> >> > 130 switch (WSAGetLastError()) { > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > to : > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > 120 nread = recv(fd, bufP, len, 0); > >> >> >> > > >> >> >> > 121 if (nread > 0) { > >> >> >> > > >> >> >> > 122 (*env)->SetByteArrayRegion(env, data, off, nread, (jbyte > >> >> *)bufP); > >> >> >> > > >> >> >> > 123 } else { > >> >> >> > > >> >> >> > 124 if (nread < 0) { > >> >> >> > > >> >> >> > 125 int err = WSAGetLastError(); > >> >> >> > > >> >> >> > ... > >> >> >> > > >> >> >> > switch (err) { > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > Thanks and best regards, Matthias From christoph.langer at sap.com Thu Aug 30 07:59:27 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 30 Aug 2018 07:59:27 +0000 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <87f37123fb8f4674baa668f04a87bdbc@sap.com> References: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> <87f37123fb8f4674baa668f04a87bdbc@sap.com> Message-ID: <275d5292b58544b6a1dec62ce0221c8c@sap.com> Hi Matthias, > Can I have a second review please ? +1 Best regards Christoph From matthias.baesken at sap.com Thu Aug 30 08:02:02 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 30 Aug 2018 08:02:02 +0000 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <275d5292b58544b6a1dec62ce0221c8c@sap.com> References: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> <87f37123fb8f4674baa668f04a87bdbc@sap.com> <275d5292b58544b6a1dec62ce0221c8c@sap.com> Message-ID: <91f79bef76df4492997a03b56231c4f6@sap.com> Thanks ! > -----Original Message----- > From: Langer, Christoph > Sent: Donnerstag, 30. August 2018 09:59 > To: Baesken, Matthias ; net-dev dev at openjdk.java.net> > Subject: RE: RFR [XS]: 8210147: adjust some WSAGetLastError usages in > windows network coding - was : RE: WSAGetLastError usage in > jdk/src/java.base/windows/native/libnet/SocketInputStream.c > > Hi Matthias, > > > Can I have a second review please ? > +1 > > Best regards > Christoph From chris.hegarty at oracle.com Thu Aug 30 10:20:59 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 30 Aug 2018 11:20:59 +0100 Subject: RFR [XS]: 8210147: adjust some WSAGetLastError usages in windows network coding - was : RE: WSAGetLastError usage in jdk/src/java.base/windows/native/libnet/SocketInputStream.c In-Reply-To: <91f79bef76df4492997a03b56231c4f6@sap.com> References: <8739deb9d175402b9a2ea2df8fc762d5@sap.com> <87f37123fb8f4674baa668f04a87bdbc@sap.com> <275d5292b58544b6a1dec62ce0221c8c@sap.com> <91f79bef76df4492997a03b56231c4f6@sap.com> Message-ID: <08582B54-8195-4F6A-B662-E9505DC290C1@oracle.com> Post-push Review ;-) I think this looks good. Thanks for catching and fix it Matthias. -Chris. > On 30 Aug 2018, at 09:02, Baesken, Matthias wrote: > > Thanks ! > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Donnerstag, 30. August 2018 09:59 >> To: Baesken, Matthias ; net-dev > dev at openjdk.java.net> >> Subject: RE: RFR [XS]: 8210147: adjust some WSAGetLastError usages in >> windows network coding - was : RE: WSAGetLastError usage in >> jdk/src/java.base/windows/native/libnet/SocketInputStream.c >> >> Hi Matthias, >> >>> Can I have a second review please ? >> +1 >> >> Best regards >> Christoph > From andrewluotechnologies at outlook.com Thu Aug 30 17:32:01 2018 From: andrewluotechnologies at outlook.com (Andrew Luo) Date: Thu, 30 Aug 2018 17:32:01 +0000 Subject: [PATCH] SOCK_CLOEXEC for opening sockets References: <83f34e37-5cb6-832d-2858-26377d1c8e9d@oracle.com> <3f50f583-a990-fb70-da1d-df2c7b26e08e@oracle.com> <6d445479-e05e-1dcc-fe25-81ee17acb219@oracle.com> <6E376834-21A9-4F2D-B11A-BA2224814444@oracle.com> <79144fd8-f11f-9384-a81c-a32c368fcd8a@oracle.com> <8A6F5B28-8587-48B8-AE58-EFDCEB5D5DD2@oracle.com> <700d8570-b8c6-9406-1204-8d4507610fab@oracle.com> Message-ID: Just checking to see if anyone has any other comments/feedback on this. Thanks, -Andrew -----Original Message----- From: Andrew Luo Sent: Monday, July 30, 2018 9:44 PM To: 'Andrew Luo' ; Ivan Gerasimov ; Chris Hegarty Cc: net-dev at openjdk.java.net Subject: RE: [PATCH] SOCK_CLOEXEC for opening sockets Fixed a bug, updated (I was calling accept directly instead of a helper function that calls accept then fcntl): diff --git a/make/lib/Lib-jdk.net.gmk b/make/lib/Lib-jdk.net.gmk --- a/make/lib/Lib-jdk.net.gmk +++ b/make/lib/Lib-jdk.net.gmk @@ -33,9 +33,11 @@ NAME := extnet, \ OPTIMIZATION := LOW, \ CFLAGS := $(CFLAGS_JDKLIB), \ + EXTRA_HEADER_DIRS := \ + java.base:libnet, \ LDFLAGS := $(LDFLAGS_JDKLIB) \ $(call SET_SHARED_LIBRARY_ORIGIN), \ - LIBS := -ljava, \ + LIBS := -ljava -lnet, \ LIBS_solaris := -lsocket, \ LIBS_linux := -ljvm, \ )) diff --git a/src/java.base/aix/native/libnio/ch/AixPollPort.c b/src/java.base/aix/native/libnio/ch/AixPollPort.c --- a/src/java.base/aix/native/libnio/ch/AixPollPort.c +++ b/src/java.base/aix/native/libnio/ch/AixPollPort.c @@ -30,6 +30,7 @@ #include "jlong.h" #include "sun_nio_ch_AixPollPort.h" +#include "net_util_md.h" #include #include @@ -137,7 +138,7 @@ JNIEXPORT void JNICALL Java_sun_nio_ch_AixPollPort_socketpair(JNIEnv* env, jclass clazz, jintArray sv) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) == -1) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) == -1) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); } else { jint res[2]; diff --git a/src/java.base/linux/native/libnet/linux_close.c b/src/java.base/linux/native/libnet/linux_close.c --- a/src/java.base/linux/native/libnet/linux_close.c +++ b/src/java.base/linux/native/libnet/linux_close.c @@ -24,6 +24,7 @@ */ #include +#include #include #include #include @@ -395,8 +396,32 @@ BLOCKING_IO_RETURN_INT( s, sendto(s, msg, len, flags, to, tolen) ); } -int NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen) { - BLOCKING_IO_RETURN_INT( s, accept(s, addr, addrlen) ); +static int acceptFcntlCloexec(int s, struct sockaddr *addr, socklen_t *addrlen) { + int cs; + cs = accept(s, addr, addrlen); + if (cs != -1) { + // Best effort - return value is intentionally ignored since the connected + // socket was successfully created. + fcntl(cs, F_SETFD, FD_CLOEXEC); + } + return cs; +} + +JNIEXPORT int JNICALL +NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen) { + static int (*accept4functionpointer)(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags) = NULL; + static int accept4initialized = 0; + int cs; + + if (!accept4initialized) { + accept4functionpointer = dlsym(RTLD_DEFAULT, "accept4"); + accept4initialized = 1; + } + + if (accept4functionpointer != NULL) { + BLOCKING_IO_RETURN_INT( s, accept4functionpointer(s, addr, addrlen, SOCK_CLOEXEC) ); + } + BLOCKING_IO_RETURN_INT( s, acceptFcntlCloexec(s, addr, addrlen) ); } int NET_Connect(int s, struct sockaddr *addr, int addrlen) { diff --git a/src/java.base/unix/native/libnet/Inet4AddressImpl.c b/src/java.base/unix/native/libnet/Inet4AddressImpl.c --- a/src/java.base/unix/native/libnet/Inet4AddressImpl.c +++ b/src/java.base/unix/native/libnet/Inet4AddressImpl.c @@ -264,7 +264,7 @@ int connect_rv = -1; // open a TCP socket - fd = socket(AF_INET, SOCK_STREAM, 0); + fd = NET_Socket(AF_INET, SOCK_STREAM, 0); if (fd == -1) { // note: if you run out of fds, you may not be able to load // the exception class, and get a NoClassDefFoundError instead. @@ -503,7 +503,7 @@ // Let's try to create a RAW socket to send ICMP packets. // This usually requires "root" privileges, so it's likely to fail. - fd = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP); + fd = NET_Socket(AF_INET, SOCK_RAW, IPPROTO_ICMP); if (fd == -1) { return tcp_ping4(env, &sa, netif, timeout, ttl); } else { diff --git a/src/java.base/unix/native/libnet/Inet6AddressImpl.c b/src/java.base/unix/native/libnet/Inet6AddressImpl.c --- a/src/java.base/unix/native/libnet/Inet6AddressImpl.c +++ b/src/java.base/unix/native/libnet/Inet6AddressImpl.c @@ -461,7 +461,7 @@ int connect_rv = -1; // open a TCP socket - fd = socket(AF_INET6, SOCK_STREAM, 0); + fd = NET_Socket(AF_INET6, SOCK_STREAM, 0); if (fd == -1) { // note: if you run out of fds, you may not be able to load // the exception class, and get a NoClassDefFoundError instead. @@ -711,7 +711,7 @@ // Let's try to create a RAW socket to send ICMP packets. // This usually requires "root" privileges, so it's likely to fail. - fd = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); + fd = NET_Socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); if (fd == -1) { return tcp_ping6(env, &sa, netif, timeout, ttl); } else { diff --git a/src/java.base/unix/native/libnet/NetworkInterface.c b/src/java.base/unix/native/libnet/NetworkInterface.c --- a/src/java.base/unix/native/libnet/NetworkInterface.c +++ b/src/java.base/unix/native/libnet/NetworkInterface.c @@ -1055,7 +1055,7 @@ static int openSocket(JNIEnv *env, int proto) { int sock; - if ((sock = socket(proto, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(proto, SOCK_DGRAM, 0)) < 0) { // If EPROTONOSUPPORT is returned it means we don't have // support for this proto so don't throw an exception. if (errno != EPROTONOSUPPORT) { @@ -1078,9 +1078,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1315,9 +1315,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1590,9 +1590,9 @@ int sock, alreadyV6 = 0; struct lifreq if2; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1616,7 +1616,7 @@ strncpy(if2.lifr_name, ifname, sizeof(if2.lifr_name) - 1); if (ioctl(sock, SIOCGLIFNETMASK, (char *)&if2) < 0) { close(sock); - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1941,9 +1941,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; diff --git a/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c b/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c --- a/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c +++ b/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c @@ -904,7 +904,7 @@ return; } - if ((fd = socket(domain, SOCK_DGRAM, 0)) == -1) { + if ((fd = NET_Socket(domain, SOCK_DGRAM, 0)) == -1) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "Error creating socket"); return; diff --git a/src/java.base/unix/native/libnet/PlainSocketImpl.c b/src/java.base/unix/native/libnet/PlainSocketImpl.c --- a/src/java.base/unix/native/libnet/PlainSocketImpl.c +++ b/src/java.base/unix/native/libnet/PlainSocketImpl.c @@ -77,7 +77,7 @@ int sv[2]; #ifdef AF_UNIX - if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) { + if (NET_SocketPair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) { return -1; } #else @@ -178,7 +178,7 @@ return; } - if ((fd = socket(domain, type, 0)) == -1) { + if ((fd = NET_Socket(domain, type, 0)) == -1) { /* note: if you run out of fds, you may not be able to load * the exception class, and get a NoClassDefFoundError * instead. diff --git a/src/java.base/unix/native/libnet/SdpSupport.c b/src/java.base/unix/native/libnet/SdpSupport.c --- a/src/java.base/unix/native/libnet/SdpSupport.c +++ b/src/java.base/unix/native/libnet/SdpSupport.c @@ -57,7 +57,7 @@ #if defined(__solaris__) int domain = ipv6_available() ? AF_INET6 : AF_INET; - s = socket(domain, SOCK_STREAM, PROTO_SDP); + s = NET_Socket(domain, SOCK_STREAM, PROTO_SDP); #elif defined(__linux__) /** * IPv6 not supported by SDP on Linux @@ -66,7 +66,7 @@ JNU_ThrowIOException(env, "IPv6 not supported"); return -1; } - s = socket(AF_INET_SDP, SOCK_STREAM, 0); + s = NET_Socket(AF_INET_SDP, SOCK_STREAM, 0); #else /* not supported on other platforms at this time */ s = -1; diff --git a/src/java.base/unix/native/libnet/net_util_md.c b/src/java.base/unix/native/libnet/net_util_md.c --- a/src/java.base/unix/native/libnet/net_util_md.c +++ b/src/java.base/unix/native/libnet/net_util_md.c @@ -131,6 +131,52 @@ return (result == -1) ? 0 : 1; } + /* + * Creates a socket. + * Uses SOCK_CLOEXEC during creation or FD_CLOEXEC post-creation, + where possible + * and on a best-effort basis, on systems that support it. + */ + JNIEXPORT int JNICALL + NET_Socket(int domain, int type, int protocol) { + int s; + #if defined(SOCK_CLOEXEC) + s = socket(domain, type | SOCK_CLOEXEC, protocol); + if (s != -1 || (s == -1 && errno != EINVAL)) { + return s; + } + #endif + s = socket(domain, type, protocol); + if (s != -1) { + // Best effort - return value is intentionally ignored since the socket + // was successfully created. + fcntl(s, F_SETFD, FD_CLOEXEC); + } + return s; + } + +/* + * Creates a pair of connected sockets. + * Uses SOCK_CLOEXEC during creation or FD_CLOEXEC post-creation, where +possible + * and on a best-effort basis, on systems that support it. + */ +JNIEXPORT int JNICALL +NET_SocketPair(int domain, int type, int protocol, int socket_vector[2]) { + int s; +#if defined(SOCK_CLOEXEC) + s = socketpair(domain, type | SOCK_CLOEXEC, protocol, socket_vector); + if (s != -1 || (s == -1 && errno != EINVAL)) { + return s; + } +#endif + s = socketpair(domain, type, protocol, socket_vector); + if (s != -1) { + // Best effort - return value is intentionally ignored since the socket + // was successfully created. + fcntl(s, F_SETFD, FD_CLOEXEC); + } + return s; +} + #ifdef __solaris__ static int init_tcp_max_buf, init_udp_max_buf; static int tcp_max_buf; @@ -295,7 +341,7 @@ SOCKETADDRESS sa; socklen_t sa_len = sizeof(SOCKETADDRESS); - fd = socket(AF_INET6, SOCK_STREAM, 0) ; + fd = NET_Socket(AF_INET6, SOCK_STREAM, 0) ; if (fd < 0) { /* * TODO: We really cant tell since it may be an unrelated error @@ -402,7 +448,7 @@ /* Do a simple dummy call, and try to figure out from that */ int one = 1; int rv, s; - s = socket(PF_INET, SOCK_STREAM, 0); + s = NET_Socket(PF_INET, SOCK_STREAM, 0); if (s < 0) { return JNI_FALSE; } diff --git a/src/java.base/unix/native/libnet/net_util_md.h b/src/java.base/unix/native/libnet/net_util_md.h --- a/src/java.base/unix/native/libnet/net_util_md.h +++ b/src/java.base/unix/native/libnet/net_util_md.h @@ -77,6 +77,9 @@ * Functions */ +JNIEXPORT int JNICALL NET_Socket(int domain, int type, int protocol); +JNIEXPORT int JNICALL NET_SocketPair(int domain, int type, int +protocol, int socket_vector[2]); + int NET_Timeout(JNIEnv *env, int s, long timeout, jlong nanoTimeStamp); int NET_Read(int s, void* buf, size_t len); int NET_NonBlockingRead(int s, void* buf, size_t len); @@ -88,7 +91,7 @@ flags, const struct sockaddr *to, int tolen); int NET_Writev(int s, const struct iovec * vector, int count); int NET_Connect(int s, struct sockaddr *addr, int addrlen); -int NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen); +JNIEXPORT int JNICALL NET_Accept(int s, struct sockaddr *addr, +socklen_t *addrlen); int NET_SocketClose(int s); int NET_Dup2(int oldfd, int newfd); int NET_Poll(struct pollfd *ufds, unsigned int nfds, int timeout); diff --git a/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c b/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c --- a/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c +++ b/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c @@ -56,6 +56,7 @@ #include "jlong.h" #include "nio.h" #include "nio_util.h" +#include "net_util_md.h" #include "sun_nio_ch_FileDispatcherImpl.h" #include "java_lang_Long.h" @@ -67,7 +68,7 @@ Java_sun_nio_ch_FileDispatcherImpl_init(JNIEnv *env, jclass cl) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); return; } diff --git a/src/java.base/unix/native/libnio/ch/Net.c b/src/java.base/unix/native/libnio/ch/Net.c --- a/src/java.base/unix/native/libnio/ch/Net.c +++ b/src/java.base/unix/native/libnio/ch/Net.c @@ -198,7 +198,7 @@ int type = (stream ? SOCK_STREAM : SOCK_DGRAM); int domain = (ipv6_available() && preferIPv6) ? AF_INET6 : AF_INET; - fd = socket(domain, type, 0); + fd = NET_Socket(domain, type, 0); if (fd < 0) { return handleSocketError(env, errno); } diff --git a/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c b/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c --- a/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c +++ b/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c @@ -92,7 +92,7 @@ * accept() was called. */ for (;;) { - newfd = accept(ssfd, &sa.sa, &sa_len); + newfd = NET_Accept(ssfd, &sa.sa, &sa_len); if (newfd >= 0) { break; } diff --git a/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c b/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c --- a/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c +++ b/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c @@ -31,6 +31,7 @@ #include #include #include "jni_util.h" +#include "net_util_md.h" #include "jdk_net_LinuxSocketOptions.h" /* @@ -86,7 +87,7 @@ (JNIEnv *env, jobject unused) { int one = 1; int rv, s; - s = socket(PF_INET, SOCK_STREAM, 0); + s = NET_Socket(PF_INET, SOCK_STREAM, 0); if (s < 0) { return JNI_FALSE; } @@ -103,7 +104,7 @@ static jint socketOptionSupported(jint sockopt) { jint one = 1; jint rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return 0; } diff --git a/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c b/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c --- a/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c +++ b/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c @@ -31,11 +31,12 @@ #include #include #include "jni_util.h" +#include "net_util_md.h" static jint socketOptionSupported(jint sockopt) { jint one = 1; jint rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return 0; } diff --git a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c --- a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c +++ b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c @@ -157,7 +157,7 @@ sock_flow_props_t props; int rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return JNI_FALSE; } diff --git a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h --- a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h +++ b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h @@ -36,6 +36,7 @@ #include "jdk_net_SocketFlow.h" #include "SolarisSocketOptions.h" #include "jdk_net_SolarisSocketOptions.h" +#include "net_util_md.h" #ifndef SO_FLOW_SLA #define SO_FLOW_SLA 0x1018 diff --git a/src/jdk.sctp/unix/native/libsctp/SctpNet.c b/src/jdk.sctp/unix/native/libsctp/SctpNet.c --- a/src/jdk.sctp/unix/native/libsctp/SctpNet.c +++ b/src/jdk.sctp/unix/native/libsctp/SctpNet.c @@ -151,7 +151,7 @@ Java_sun_nio_ch_sctp_SctpNet_init (JNIEnv *env, jclass cl) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); return; } @@ -180,7 +180,7 @@ return 0; } - fd = socket(domain, (oneToOne ? SOCK_STREAM : SOCK_SEQPACKET), IPPROTO_SCTP); + fd = NET_Socket(domain, (oneToOne ? SOCK_STREAM : SOCK_SEQPACKET), + IPPROTO_SCTP); if (fd < 0) { return handleSocketError(env, errno); Thanks, -Andrew -----Original Message----- From: Andrew Luo Sent: Monday, July 30, 2018 5:08 PM To: Andrew Luo ; Ivan Gerasimov ; Chris Hegarty Cc: net-dev at openjdk.java.net Subject: RE: [PATCH] SOCK_CLOEXEC for opening sockets I've updated my patch. Let me know if this approach looks good to you. I'm using dlsym to get the function pointer to accept4 at runtime so we can support compiling on newer Linux systems and using the binaries on older Linux systems (where the newer Linux system has accept4, and the older Linux system does not). I know I still need to edit the other Unix (Solaris, AIX, BSD) files but I wanted to put this out there first to see if the approach looks good. diff --git a/make/lib/Lib-jdk.net.gmk b/make/lib/Lib-jdk.net.gmk --- a/make/lib/Lib-jdk.net.gmk +++ b/make/lib/Lib-jdk.net.gmk @@ -33,9 +33,11 @@ NAME := extnet, \ OPTIMIZATION := LOW, \ CFLAGS := $(CFLAGS_JDKLIB), \ + EXTRA_HEADER_DIRS := \ + java.base:libnet, \ LDFLAGS := $(LDFLAGS_JDKLIB) \ $(call SET_SHARED_LIBRARY_ORIGIN), \ - LIBS := -ljava, \ + LIBS := -ljava -lnet, \ LIBS_solaris := -lsocket, \ LIBS_linux := -ljvm, \ )) diff --git a/src/java.base/aix/native/libnio/ch/AixPollPort.c b/src/java.base/aix/native/libnio/ch/AixPollPort.c --- a/src/java.base/aix/native/libnio/ch/AixPollPort.c +++ b/src/java.base/aix/native/libnio/ch/AixPollPort.c @@ -30,6 +30,7 @@ #include "jlong.h" #include "sun_nio_ch_AixPollPort.h" +#include "net_util_md.h" #include #include @@ -137,7 +138,7 @@ JNIEXPORT void JNICALL Java_sun_nio_ch_AixPollPort_socketpair(JNIEnv* env, jclass clazz, jintArray sv) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) == -1) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) == -1) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); } else { jint res[2]; diff --git a/src/java.base/linux/native/libnet/linux_close.c b/src/java.base/linux/native/libnet/linux_close.c --- a/src/java.base/linux/native/libnet/linux_close.c +++ b/src/java.base/linux/native/libnet/linux_close.c @@ -24,6 +24,7 @@ */ #include +#include #include #include #include @@ -395,7 +396,31 @@ BLOCKING_IO_RETURN_INT( s, sendto(s, msg, len, flags, to, tolen) ); } -int NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen) { +static int acceptFcntlCloexec(int s, struct sockaddr *addr, socklen_t *addrlen) { + int cs; + cs = accept(s, addr, addrlen); + if (cs != -1) { + // Best effort - return value is intentionally ignored since the connected + // socket was successfully created. + fcntl(cs, F_SETFD, FD_CLOEXEC); + } + return cs; +} + +JNIEXPORT int JNICALL +NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen) { + static int (*accept4functionpointer)(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags) = NULL; + static int accept4initialized = 0; + int cs; + + if (!accept4initialized) { + accept4functionpointer = dlsym(RTLD_DEFAULT, "accept4"); + accept4initialized = 1; + } + + if (accept4functionpointer != NULL) { + BLOCKING_IO_RETURN_INT( s, accept4functionpointer(s, addr, addrlen, SOCK_CLOEXEC) ); + } BLOCKING_IO_RETURN_INT( s, accept(s, addr, addrlen) ); } diff --git a/src/java.base/unix/native/libnet/Inet4AddressImpl.c b/src/java.base/unix/native/libnet/Inet4AddressImpl.c --- a/src/java.base/unix/native/libnet/Inet4AddressImpl.c +++ b/src/java.base/unix/native/libnet/Inet4AddressImpl.c @@ -264,7 +264,7 @@ int connect_rv = -1; // open a TCP socket - fd = socket(AF_INET, SOCK_STREAM, 0); + fd = NET_Socket(AF_INET, SOCK_STREAM, 0); if (fd == -1) { // note: if you run out of fds, you may not be able to load // the exception class, and get a NoClassDefFoundError instead. @@ -503,7 +503,7 @@ // Let's try to create a RAW socket to send ICMP packets. // This usually requires "root" privileges, so it's likely to fail. - fd = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP); + fd = NET_Socket(AF_INET, SOCK_RAW, IPPROTO_ICMP); if (fd == -1) { return tcp_ping4(env, &sa, netif, timeout, ttl); } else { diff --git a/src/java.base/unix/native/libnet/Inet6AddressImpl.c b/src/java.base/unix/native/libnet/Inet6AddressImpl.c --- a/src/java.base/unix/native/libnet/Inet6AddressImpl.c +++ b/src/java.base/unix/native/libnet/Inet6AddressImpl.c @@ -461,7 +461,7 @@ int connect_rv = -1; // open a TCP socket - fd = socket(AF_INET6, SOCK_STREAM, 0); + fd = NET_Socket(AF_INET6, SOCK_STREAM, 0); if (fd == -1) { // note: if you run out of fds, you may not be able to load // the exception class, and get a NoClassDefFoundError instead. @@ -711,7 +711,7 @@ // Let's try to create a RAW socket to send ICMP packets. // This usually requires "root" privileges, so it's likely to fail. - fd = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); + fd = NET_Socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); if (fd == -1) { return tcp_ping6(env, &sa, netif, timeout, ttl); } else { diff --git a/src/java.base/unix/native/libnet/NetworkInterface.c b/src/java.base/unix/native/libnet/NetworkInterface.c --- a/src/java.base/unix/native/libnet/NetworkInterface.c +++ b/src/java.base/unix/native/libnet/NetworkInterface.c @@ -1055,7 +1055,7 @@ static int openSocket(JNIEnv *env, int proto) { int sock; - if ((sock = socket(proto, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(proto, SOCK_DGRAM, 0)) < 0) { // If EPROTONOSUPPORT is returned it means we don't have // support for this proto so don't throw an exception. if (errno != EPROTONOSUPPORT) { @@ -1078,9 +1078,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1315,9 +1315,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1590,9 +1590,9 @@ int sock, alreadyV6 = 0; struct lifreq if2; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1616,7 +1616,7 @@ strncpy(if2.lifr_name, ifname, sizeof(if2.lifr_name) - 1); if (ioctl(sock, SIOCGLIFNETMASK, (char *)&if2) < 0) { close(sock); - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; @@ -1941,9 +1941,9 @@ static int openSocketWithFallback(JNIEnv *env, const char *ifname) { int sock; - if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET, SOCK_DGRAM, 0)) < 0) { if (errno == EPROTONOSUPPORT) { - if ((sock = socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { + if ((sock = NET_Socket(AF_INET6, SOCK_DGRAM, 0)) < 0) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "IPV6 Socket creation failed"); return -1; diff --git a/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c b/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c --- a/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c +++ b/src/java.base/unix/native/libnet/PlainDatagramSocketImpl.c @@ -904,7 +904,7 @@ return; } - if ((fd = socket(domain, SOCK_DGRAM, 0)) == -1) { + if ((fd = NET_Socket(domain, SOCK_DGRAM, 0)) == -1) { JNU_ThrowByNameWithMessageAndLastError (env, JNU_JAVANETPKG "SocketException", "Error creating socket"); return; diff --git a/src/java.base/unix/native/libnet/PlainSocketImpl.c b/src/java.base/unix/native/libnet/PlainSocketImpl.c --- a/src/java.base/unix/native/libnet/PlainSocketImpl.c +++ b/src/java.base/unix/native/libnet/PlainSocketImpl.c @@ -77,7 +77,7 @@ int sv[2]; #ifdef AF_UNIX - if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) { + if (NET_SocketPair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) { return -1; } #else @@ -178,7 +178,7 @@ return; } - if ((fd = socket(domain, type, 0)) == -1) { + if ((fd = NET_Socket(domain, type, 0)) == -1) { /* note: if you run out of fds, you may not be able to load * the exception class, and get a NoClassDefFoundError * instead. diff --git a/src/java.base/unix/native/libnet/SdpSupport.c b/src/java.base/unix/native/libnet/SdpSupport.c --- a/src/java.base/unix/native/libnet/SdpSupport.c +++ b/src/java.base/unix/native/libnet/SdpSupport.c @@ -57,7 +57,7 @@ #if defined(__solaris__) int domain = ipv6_available() ? AF_INET6 : AF_INET; - s = socket(domain, SOCK_STREAM, PROTO_SDP); + s = NET_Socket(domain, SOCK_STREAM, PROTO_SDP); #elif defined(__linux__) /** * IPv6 not supported by SDP on Linux @@ -66,7 +66,7 @@ JNU_ThrowIOException(env, "IPv6 not supported"); return -1; } - s = socket(AF_INET_SDP, SOCK_STREAM, 0); + s = NET_Socket(AF_INET_SDP, SOCK_STREAM, 0); #else /* not supported on other platforms at this time */ s = -1; diff --git a/src/java.base/unix/native/libnet/net_util_md.c b/src/java.base/unix/native/libnet/net_util_md.c --- a/src/java.base/unix/native/libnet/net_util_md.c +++ b/src/java.base/unix/native/libnet/net_util_md.c @@ -131,6 +131,52 @@ return (result == -1) ? 0 : 1; } + /* + * Creates a socket. + * Uses SOCK_CLOEXEC during creation or FD_CLOEXEC post-creation, + where possible + * and on a best-effort basis, on systems that support it. + */ + JNIEXPORT int JNICALL + NET_Socket(int domain, int type, int protocol) { + int s; + #if defined(SOCK_CLOEXEC) + s = socket(domain, type | SOCK_CLOEXEC, protocol); + if (s != -1 || (s == -1 && errno != EINVAL)) { + return s; + } + #endif + s = socket(domain, type, protocol); + if (s != -1) { + // Best effort - return value is intentionally ignored since the socket + // was successfully created. + fcntl(s, F_SETFD, FD_CLOEXEC); + } + return s; + } + +/* + * Creates a pair of connected sockets. + * Uses SOCK_CLOEXEC during creation or FD_CLOEXEC post-creation, where +possible + * and on a best-effort basis, on systems that support it. + */ +JNIEXPORT int JNICALL +NET_SocketPair(int domain, int type, int protocol, int socket_vector[2]) { + int s; +#if defined(SOCK_CLOEXEC) + s = socketpair(domain, type | SOCK_CLOEXEC, protocol, socket_vector); + if (s != -1 || (s == -1 && errno != EINVAL)) { + return s; + } +#endif + s = socketpair(domain, type, protocol, socket_vector); + if (s != -1) { + // Best effort - return value is intentionally ignored since the socket + // was successfully created. + fcntl(s, F_SETFD, FD_CLOEXEC); + } + return s; +} + #ifdef __solaris__ static int init_tcp_max_buf, init_udp_max_buf; static int tcp_max_buf; @@ -295,7 +341,7 @@ SOCKETADDRESS sa; socklen_t sa_len = sizeof(SOCKETADDRESS); - fd = socket(AF_INET6, SOCK_STREAM, 0) ; + fd = NET_Socket(AF_INET6, SOCK_STREAM, 0) ; if (fd < 0) { /* * TODO: We really cant tell since it may be an unrelated error @@ -402,7 +448,7 @@ /* Do a simple dummy call, and try to figure out from that */ int one = 1; int rv, s; - s = socket(PF_INET, SOCK_STREAM, 0); + s = NET_Socket(PF_INET, SOCK_STREAM, 0); if (s < 0) { return JNI_FALSE; } diff --git a/src/java.base/unix/native/libnet/net_util_md.h b/src/java.base/unix/native/libnet/net_util_md.h --- a/src/java.base/unix/native/libnet/net_util_md.h +++ b/src/java.base/unix/native/libnet/net_util_md.h @@ -77,6 +77,9 @@ * Functions */ +JNIEXPORT int JNICALL NET_Socket(int domain, int type, int protocol); +JNIEXPORT int JNICALL NET_SocketPair(int domain, int type, int +protocol, int socket_vector[2]); + int NET_Timeout(JNIEnv *env, int s, long timeout, jlong nanoTimeStamp); int NET_Read(int s, void* buf, size_t len); int NET_NonBlockingRead(int s, void* buf, size_t len); @@ -88,7 +91,7 @@ flags, const struct sockaddr *to, int tolen); int NET_Writev(int s, const struct iovec * vector, int count); int NET_Connect(int s, struct sockaddr *addr, int addrlen); -int NET_Accept(int s, struct sockaddr *addr, socklen_t *addrlen); +JNIEXPORT int JNICALL NET_Accept(int s, struct sockaddr *addr, +socklen_t *addrlen); int NET_SocketClose(int s); int NET_Dup2(int oldfd, int newfd); int NET_Poll(struct pollfd *ufds, unsigned int nfds, int timeout); diff --git a/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c b/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c --- a/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c +++ b/src/java.base/unix/native/libnio/ch/FileDispatcherImpl.c @@ -56,6 +56,7 @@ #include "jlong.h" #include "nio.h" #include "nio_util.h" +#include "net_util_md.h" #include "sun_nio_ch_FileDispatcherImpl.h" #include "java_lang_Long.h" @@ -67,7 +68,7 @@ Java_sun_nio_ch_FileDispatcherImpl_init(JNIEnv *env, jclass cl) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); return; } diff --git a/src/java.base/unix/native/libnio/ch/Net.c b/src/java.base/unix/native/libnio/ch/Net.c --- a/src/java.base/unix/native/libnio/ch/Net.c +++ b/src/java.base/unix/native/libnio/ch/Net.c @@ -198,7 +198,7 @@ int type = (stream ? SOCK_STREAM : SOCK_DGRAM); int domain = (ipv6_available() && preferIPv6) ? AF_INET6 : AF_INET; - fd = socket(domain, type, 0); + fd = NET_Socket(domain, type, 0); if (fd < 0) { return handleSocketError(env, errno); } diff --git a/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c b/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c --- a/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c +++ b/src/java.base/unix/native/libnio/ch/ServerSocketChannelImpl.c @@ -92,7 +92,7 @@ * accept() was called. */ for (;;) { - newfd = accept(ssfd, &sa.sa, &sa_len); + newfd = NET_Accept(ssfd, &sa.sa, &sa_len); if (newfd >= 0) { break; } diff --git a/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c b/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c --- a/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c +++ b/src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c @@ -31,6 +31,7 @@ #include #include #include "jni_util.h" +#include "net_util_md.h" #include "jdk_net_LinuxSocketOptions.h" /* @@ -86,7 +87,7 @@ (JNIEnv *env, jobject unused) { int one = 1; int rv, s; - s = socket(PF_INET, SOCK_STREAM, 0); + s = NET_Socket(PF_INET, SOCK_STREAM, 0); if (s < 0) { return JNI_FALSE; } @@ -103,7 +104,7 @@ static jint socketOptionSupported(jint sockopt) { jint one = 1; jint rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return 0; } diff --git a/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c b/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c --- a/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c +++ b/src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c @@ -31,11 +31,12 @@ #include #include #include "jni_util.h" +#include "net_util_md.h" static jint socketOptionSupported(jint sockopt) { jint one = 1; jint rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return 0; } diff --git a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c --- a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c +++ b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.c @@ -157,7 +157,7 @@ sock_flow_props_t props; int rv, s; - s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); + s = NET_Socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); if (s < 0) { return JNI_FALSE; } diff --git a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h --- a/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h +++ b/src/jdk.net/solaris/native/libextnet/SolarisSocketOptions.h @@ -36,6 +36,7 @@ #include "jdk_net_SocketFlow.h" #include "SolarisSocketOptions.h" #include "jdk_net_SolarisSocketOptions.h" +#include "net_util_md.h" #ifndef SO_FLOW_SLA #define SO_FLOW_SLA 0x1018 diff --git a/src/jdk.sctp/unix/native/libsctp/SctpNet.c b/src/jdk.sctp/unix/native/libsctp/SctpNet.c --- a/src/jdk.sctp/unix/native/libsctp/SctpNet.c +++ b/src/jdk.sctp/unix/native/libsctp/SctpNet.c @@ -151,7 +151,7 @@ Java_sun_nio_ch_sctp_SctpNet_init (JNIEnv *env, jclass cl) { int sp[2]; - if (socketpair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { + if (NET_SocketPair(PF_UNIX, SOCK_STREAM, 0, sp) < 0) { JNU_ThrowIOExceptionWithLastError(env, "socketpair failed"); return; } @@ -180,7 +180,7 @@ return 0; } - fd = socket(domain, (oneToOne ? SOCK_STREAM : SOCK_SEQPACKET), IPPROTO_SCTP); + fd = NET_Socket(domain, (oneToOne ? SOCK_STREAM : SOCK_SEQPACKET), + IPPROTO_SCTP); if (fd < 0) { return handleSocketError(env, errno); Thanks, -Andrew -----Original Message----- From: net-dev On Behalf Of Andrew Luo Sent: Thursday, July 26, 2018 11:31 AM To: Ivan Gerasimov ; Chris Hegarty Cc: net-dev at openjdk.java.net Subject: RE: [PATCH] SOCK_CLOEXEC for opening sockets Do we need to support compiling on a newer kernel with newer headers and then running the compiled binaries on an older platform? If so, then I think we probably have to use dlsym to call accept4 at runtime rather than compile time... Thanks, -Andrew -----Original Message----- From: net-dev On Behalf Of Ivan Gerasimov Sent: Wednesday, July 25, 2018 10:04 PM To: Chris Hegarty Cc: net-dev at openjdk.java.net Subject: Re: [PATCH] SOCK_CLOEXEC for opening sockets Hi Chris! A couple of minor comments. 1) if (s != -1 || (s == -1 && errno != EINVAL)) { This can be simplified to if (s != -1 || errno != EINVAL) { 2) What about sockets created with accept(): Shouldn't NET_Accept be modified to set O_CLOEXEC as well? On Linux accept4() can be used to pass SOCK_CLOEXEC flag. With kind regards, Ivan On 7/25/18 5:49 AM, Chris Hegarty wrote: > Alan, > >> On 25 Jul 2018, at 08:24, Alan Bateman wrote: >> >> ... >> >> As I said previously, the patch isn't complete so native code calling fork/exec may still have to deal with other file descriptors that are inherited into the child. I don't object to doing this in phases of course but somehow we have managed to get by for 20 years without this being an issue. > I added the following to the JIRA description to make this clear: > > "This JIRA issue, by itself, does not completely resolve the problem > of native code calling fork/exec, it may still have to deal with other > file descriptors that are inherited by the child. Instead this JIRA > issue is targeted at the socket and channels areas only, other areas > should be tackled on a phased approach though separate JIRA issues." > >> The updates to the various site to use the NET_* functions are fine. However, I think the new functions in net_util_md.c could be cleaner. I think it would be better to fallback to socket/socketpair + fcntl when the initial call fails with EINVAL. > Agreed. How about this ( trying to reduce the ifdef blocks, and keep > them relatively clean ) : > > --- > JNIEXPORT int JNICALL > NET_Socket(int domain, int type, int protocol) { > int s; > #if defined(SOCK_CLOEXEC) > s = socket(domain, type | SOCK_CLOEXEC, protocol); > if (s != -1 || (s == -1 && errno != EINVAL)) { > return s; > } > #endif > s = socket(domain, type, protocol); > if (s != -1) { > // Best effort - return value is intentionally ignored since the socket > // was successfully created. > fcntl(s, F_SETFD, FD_CLOEXEC); > } > return s; > } > --- > > Updated webrev: > http://cr.openjdk.java.net/~chegar/8207335/webrev.01/ > > -Chris. -- With kind regards, Ivan Gerasimov