EOF excption in HTTP 1.1 server interaction
Daniel Fuchs
daniel.fuchs at oracle.com
Fri May 18 11:42:18 UTC 2018
Hi Simon,
I have done some more investigation, and
I believe the issue here is that the `express` server
doesn't like the connection upgrade that our HTTP Client
asks for.
As a result instead of ignoring the upgrade and simply
replying with HTTP/1.1 the server seems to close the
connection just after receiving the requests headers,
without sending anything back to the client.
This is what is causing the EOF exception, because the
connection gets closed right away when trying to read
the response headers.
This can be easily demonstrated by running the small fake client
below at the end of this email.
You will see that if you comment out the line:
"Connection: Upgrade, HTTP2-Settings",
from the request headers, then the server sends back the response.
Otherwise, it just closes the connection, and nothing is received
by the client.
One work around is to explicitly configure your request to be sent
over HTTP/1.1 (otherwise, by default, the client tries the upgrade
to HTTP/2):
HttpRequest getRequest = HttpRequest.newBuilder()
.uri(uri)
.version(HttpClient.Version.HTTP_1_1)
.GET().build();
In that case the client will use HTTP/1.1 and won't send
the upgrade headers and everything appears to work fine.
Thank you for reporting this issue to us though, as it lead
us to log a couple of RFEs to help better diagnosis.
Hope this helps,
-- daniel
-------------------------------------------------
static final String[] request = {
"GET /simple.html HTTP/1.1",
"Connection: Upgrade, HTTP2-Settings",
"Content-Length: 0",
"Host: localhost:8080",
"HTTP2-Settings: AAEAAEAAAAIAAAABAAMAAABkAAQBAAAAAAUAAEAA",
"Upgrade: h2c",
"User-Agent: Java-http-client/11-internal",
"",
};
public static void main(String[] args) throws IOException {
try (Socket s = new Socket("localhost", 8080)) {
OutputStream so = s.getOutputStream();
for (var line : request) {
System.out.println("> " + line);
so.write(line.getBytes(StandardCharsets.US_ASCII));
so.write('\r');
so.write('\n');
}
so.flush();
InputStream si = s.getInputStream();
BufferedReader reader = new BufferedReader(
new InputStreamReader(si, StandardCharsets.UTF_8));
reader.lines()
.map(line -> "< " + line)
.forEach(System.out::println);
}
}
------------------------------------------------------------------
On 16/05/2018 17:24, Simon Roberts wrote:
> OK, here goes.
>
> 1) install node 8.11 from here: https://nodejs.org/
> this should consist (though I could be wrong for mac or windows!) of
> expanding the archive, and adding the contained 'bin' directory to your
> path.
> You should be able to execute the commands:
> node --version
> (note two minus-signs. Should report v8.11.??)
> and
> npm -version
> (note single minus sign! Might report 5.6?? but might be older, not a
> big deal)
>
> 2) clone this: https://github.com/SimonHGR/RESTserver.git
>
> 3) in the resulting directory (RESTserver) enter:
> npm install
> (it'll download a bunch of javascript libraries)
>
> 4) still in the same directory enter:
> npm start
>
> (it should report Express listening on port 8080)
>
> You should now be able to do
>
> curl -v http://localhost:8080/simple.html
>
> and see:
> --------------------------------------------------------------------
> * Trying 127.0.0.1...
> * Connected to localhost (127.0.0.1) port 8080 (#0)
> > GET /simple.html HTTP/1.1
> > Host: localhost:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < X-Powered-By: Express
> < Content-Type: text/html; charset=utf-8
> < Content-Length: 95
> < ETag: W/"5f-zPY21XEeilaiEZvppUEI+1JlLE8"
> < Date: Wed, 16 May 2018 16:15:35 GMT
> < Connection: keep-alive
> <
> <html>
> <head>
> </head>
> <body>
> <h1>A Heading</h1>
> <p>A paragraph</p>
> </body>
> </html>
> * Connection #0 to host localhost left intact
>
> --------------------------------------------------------------------
>
> Notes, as a result of the discussion, I deliberately added code, at line
> 56 of server.js, to try to ensure that it's sending \r\n. This code
> isn't "normal" in node, but I believe it's irrelevant anyway, as I've
> already shown that httpClient is failing before it ever processes the
> body (perhaps the headers aren't being sent with \r\n!?)
>
> Also, I did try, last night, to turn off the HTTP2.0 mode in httpClient
> using:
>
> HttpClient client =
> HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build();
>
> but that did not change anything.
>
> I'll also note that when trying my "trivial serer", it reproduced the
> problem when it was sending the wrong line-endings.
>
> Anyway, hopefully you can look at the protocol exchange and see what
> node is doing that's wrong.
>
> If you need any updates to the node server, just holler, I'll be around
> and able to put a few minutes into this here and there most of today (at
> least up through 3pm Mountain time).
>
> Cheers
>
>
> On Wed, May 16, 2018 at 9:49 AM Simon Roberts
> <simon at dancingcloudservices.com <mailto:simon at dancingcloudservices.com>>
> wrote:
>
> Thanks for investigating this, Chris. I will put together a trivial
> node server and instructions on how to set it up (it's delightfully
> simple) Should be much less than an hour, if I can give it my
> un-distracted attention...
>
> On Wed, May 16, 2018 at 8:58 AM Chris Hegarty
> <chris.hegarty at oracle.com <mailto:chris.hegarty at oracle.com>> wrote:
>
> Bernd,
>
> > On 16 May 2018, at 00:43, Bernd Eckenfels
> <ecki at zusammenkunft.net <mailto:ecki at zusammenkunft.net>> wrote:
> >
> > ...
> > For the httpclient code, the following improvements are IMHO
> possible:
> >
> > • As you mentioned the EOF should contain the callsite
> and not be transorted from a worker thread context. This can
> either be done by rethrwing the EOF Exception or by actually
> constructing them in the read(). The same (or arguably even
> worse) Problem is a „connection refused“ type of exception which
> also has no clear calcite.
>
> Agreed. The synchronous send should recreate exceptions
> before throwing.
>
> I filed https://bugs.openjdk.java.net/browse/JDK-8203298
>
> > • Instead of throwing an EOF exception when the read
> Buffer exceeds the Body lenth I would return a short read till
> the last available Byte and only throw at the next read. This
> way the handler can get all of the partial Body. This is however
> not a good optimization for BodyHander.asString().
>
> Agreed.
>
> I think that the client implementation can provide a more
> helpful exception detail message, than what it does today.
>
> I filed https://bugs.openjdk.java.net/browse/JDK-8203302
>
> While maybe not helpful in the string case, it is important
> that all response body is delivered to the handler before
> EOF. The following has been file to ensure that that is the
> case.
>
> I filed https://bugs.openjdk.java.net/browse/JDK-8203303
>
> > It is still unlear why the node.js expresss Server has
> Problems. For that I think its a good idea to trace it either
> with Wireshark/tcpdump or Maybe one of the web app Debugging
> reverse proxies.
>
> It would be helpful to us to understand exactly what the
> problem is here. I can try out node.js myself, or if anyone
> has some instructions to help set that up it would be
> helpful.
>
> -Chris.
>
>
>
> --
> Simon Roberts
> (303) 249 3613
>
>
>
> --
> Simon Roberts
> (303) 249 3613
>
More information about the net-dev
mailing list