EOF excption in HTTP 1.1 server interaction

Wed May 16 21:23:17 UTC 2018

Your node server lies about its Content-Length.

That's the #1 reason for your issues, lack of truth about the
Content-Length.

When you under-send (Content-Length longer then actual response body
content), you get the java.io.EOFException: EOF reached while reading from
your initial message.

When you over-send (Content-Length is shorter then your actual response
body content), you get the truncated content in the HttpClient (this
behavior follows the http spec)

I went ahead and put together a demo project to show this at (no npm, no
node, using Java and Jetty)

https://github.com/joakime/jdk-http-client-example

Note: with the jetty, you can't under-send, you'll get a server error
instead.

Run ServerMain (in your IDE) - and leave it be.

Run ClientMain to test all three scenarios.

Example output:

*Regular request, server determines Content-Length*
request: http://localhost:9090/hello
response headers: jdk.incubator.http.ImmutableHeaders at d9a98b4c
{content-length=[59], content-type=[text/html;charset=utf-8], date=[Wed, 16
May 2018 21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} }
response to get: <html><body><h1>Heading</h1><p>Some Text</p></body></html>

*Server sends BAD Content-Length (under-send)*
request: http://localhost:9090/hello?forced-len=40
response headers: jdk.incubator.http.ImmutableHeaders at d9a98af4
{content-length=[40], content-type=[text/html;charset=utf-8], date=[Wed, 16
May 2018 21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} }
response to get: <html><body><h1>Heading</h1><p>Some Text

*Server sends BAD Content-Length (over-send)*
request: http://localhost:9090/hello?forced-len=70
response headers: jdk.incubator.http.ImmutableHeaders at 40425da6
{cache-control=[must-revalidate,no-cache,no-store], content-length=[360],
content-type=[text/html;charset=iso-8859-1], date=[Wed, 16 May 2018
21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} }
response to get: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 insufficient content written</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /hello. Reason:
<pre>    insufficient content written</pre></p><hr><a href="
http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>

</body>
</html>

Process finished with exit code 0

On Wed, May 16, 2018 at 11:24 AM, Simon Roberts <
simon at dancingcloudservices.com> wrote:

> OK, here goes.
>
> 1) install node 8.11 from here: https://nodejs.org/
> this should consist (though I could be wrong for mac or windows!) of
> expanding the archive, and adding the contained 'bin' directory to your
> path.
> You should be able to execute the commands:
> node --version
> (note two minus-signs. Should report v8.11.??)
> and
> npm -version
> (note single minus sign! Might report 5.6?? but might be older, not a big
> deal)
>
> 2) clone this: https://github.com/SimonHGR/RESTserver.git
>
> 3) in the resulting directory (RESTserver) enter:
> npm install
> (it'll download a bunch of javascript libraries)
>
> 4) still in the same directory enter:
> npm start
>
> (it should report Express listening on port 8080)
>
> You should now be able to do
>
> curl -v http://localhost:8080/simple.html
>
> and see:
> --------------------------------------------------------------------
> *   Trying 127.0.0.1...
> * Connected to localhost (127.0.0.1) port 8080 (#0)
> > GET /simple.html HTTP/1.1
> > Host: localhost:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < X-Powered-By: Express
> < Content-Type: text/html; charset=utf-8
> < Content-Length: 95
> < ETag: W/"5f-zPY21XEeilaiEZvppUEI+1JlLE8"
> < Date: Wed, 16 May 2018 16:15:35 GMT
> < Connection: keep-alive
> <
> <html>
> <head>
> </head>
> <body>
>   <h1>A Heading</h1>
>   <p>A paragraph</p>
> </body>
> </html>
> * Connection #0 to host localhost left intact
>
> --------------------------------------------------------------------
>
> Notes, as a result of the discussion, I deliberately added code, at line
> 56 of server.js, to try to ensure that it's sending \r\n. This code isn't
> "normal" in node, but I believe it's irrelevant anyway, as I've already
> shown that httpClient is failing before it ever processes the body (perhaps
> the headers aren't being sent with \r\n!?)
>
> Also, I did try, last night, to turn off the HTTP2.0 mode in httpClient
> using:
>
>  HttpClient client = HttpClient.newBuilder().version(HttpClient.Version.
> HTTP_1_1).build();
>
> but that did not change anything.
>
> I'll also note that when trying my "trivial serer", it reproduced the
> problem when it was sending the wrong line-endings.
>
> Anyway, hopefully you can look at the protocol exchange and see what node
> is doing that's wrong.
>
> If you need any updates to the node server, just holler, I'll be around
> and able to put a few minutes into this here and there most of today (at
> least up through 3pm Mountain time).
>
> Cheers
>
>
> On Wed, May 16, 2018 at 9:49 AM Simon Roberts <simon at dancingcloudservices.
> com> wrote:
>
>> Thanks for investigating this, Chris. I will put together a trivial node
>> server and instructions on how to set it up (it's delightfully simple)
>> Should be much less than an hour, if I can give it my un-distracted
>> attention...
>>
>> On Wed, May 16, 2018 at 8:58 AM Chris Hegarty <chris.hegarty at oracle.com>
>> wrote:
>>
>>> Bernd,
>>>
>>> > On 16 May 2018, at 00:43, Bernd Eckenfels <ecki at zusammenkunft.net>
>>> wrote:
>>> >
>>> > ...
>>> > For the httpclient code, the following improvements are IMHO possible:
>>> >
>>> >       • As you mentioned the EOF should contain the callsite and not
>>> be transorted from a worker thread context. This can either be done by
>>> rethrwing the EOF Exception or by actually constructing them in the read().
>>> The same (or arguably even worse) Problem is a „connection refused“ type of
>>> exception which also has no clear calcite.
>>>
>>> Agreed. The synchronous send should recreate exceptions
>>> before throwing.
>>>
>>> I filed https://bugs.openjdk.java.net/browse/JDK-8203298
>>>
>>> >       • Instead of throwing an EOF exception when the read Buffer
>>> exceeds the Body lenth I would return a short read till the last available
>>> Byte and only throw at the next read. This way the handler can get all of
>>> the partial Body. This is however not a good optimization for
>>> BodyHander.asString().
>>>
>>> Agreed.
>>>
>>> I think that the client implementation can provide a more
>>> helpful exception detail message, than what it does today.
>>>
>>> I filed https://bugs.openjdk.java.net/browse/JDK-8203302
>>>
>>> While maybe not helpful in the string case, it is important
>>> that all response body is delivered to the handler before
>>> EOF. The following has been file to ensure that that is the
>>> case.
>>>
>>> I filed https://bugs.openjdk.java.net/browse/JDK-8203303
>>>
>>> > It is still unlear why the node.js expresss Server has Problems. For
>>> that I think its a good idea to trace it either with Wireshark/tcpdump or
>>> Maybe one of the web app Debugging reverse proxies.
>>>
>>> It would be helpful to us to understand exactly what the
>>> problem is here. I can try out node.js myself, or if anyone
>>> has some instructions to help set that up it would be
>>> helpful.
>>>
>>> -Chris.
>>
>>
>>
>> --
>> Simon Roberts
>> (303) 249 3613
>>
>>
>
> --
> Simon Roberts
> (303) 249 3613
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180516/11f754fb/attachment-0001.html>