EOF excption in HTTP 1.1 server interaction
Bernd Eckenfels
ecki at zusammenkunft.net
Tue May 15 23:43:22 UTC 2018
Hello Simon,
I am also just a user here, but I certainly agree with you that the sample Code should be able to communicate with (well behaved) public web Servers.
So if no TLS or htttp2 is involved, I dont think this is the Problem here.
The following is a test Server I used (baed on yours), it does produce a correct response (i.e. no exception) if the Parameter is fail=false and a aborted exception if the Parameter fail=true (because it writes one Byte less than the Content-length would announce and therefore the Client reads past the payload)
In the fail case it does indeed not return a Body result (but that is IMHO ok the sample Code must not deal with web Servers which are defect)
For the httpclient code, the following improvements are IMHO possible:
a) As you mentioned the EOF should contain the callsite and not be transorted from a worker thread context. This can either be done by rethrwing the EOF Exception or by actually constructing them in the read(). The same (or arguably even worse) Problem is a „connection refused“ type of exception which also has no clear callsite.
b) Instead of throwing an EOF exception when the read Buffer exceeds the Body lenth I would return a short read till the last available Byte and only throw at the next read. This way the handler can get all of the partial Body. This is however not a good optimization for BodyHander.asString().
It is still unlear why the node.js expresss Server has Problems. For that I think its a good idea to trace it either with Wireshark/tcpdump or Maybe one of the web app Debugging reverse proxies.
Gruss
Bernd
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.net.ServerSocket;
import java.net.Socket;
public class FakeSever
{
static final boolean fail = true;
static final String body = "<html><body><h1>Heading</h1><p>Some Text</p></body></html>";
public static final String[] response = {
"HTTP/1.1 200 OK",
"X-Powered-By: Expressd",
"Content-Type: text/html; charset=utf-8",
"Content-Length: " + (body.length()+(fail?3:2)),
"ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
"Date: Tue, 15 May 2018 20:34:49 GMT",
"Connection: keep-alive",
"",
body,
};
public static void main(String[] args) throws Throwable {
try (ServerSocket ss = new ServerSocket(8080);) {
System.out.println("Listening...");
try (Socket s = ss.accept();) {
System.out.println("Accepted " + s);
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("< " + line);
if ("".equals(line)) break;
}
Writer out = new OutputStreamWriter(s.getOutputStream());
for (String st : response) {
out.write(st + "\r\n");
System.out.println("> " + st);
}
out.flush();
}
}
}
}
--
http://bernd.eckenfels.net
Von: Simon Roberts
Gesendet: Mittwoch, 16. Mai 2018 00:02
An: net-dev at openjdk.java.net
Betreff: Re: EOF excption in HTTP 1.1 server interaction
No, HTTP 2.0 is not supported by nodejs (... current LTS release, eight-dot-something) and by my fake server. I believe that there might be something about the way that node is doing that ignoring that's causing the problem.
If there's an easy way for me to dump the entire transaction, and anyone cares why the send request fails (this thing **never calls the BodyHandler** in case I've not made that clear) then I'm happy to run one more test, if it helps. I guess if anyone would like to see it, I can install wireshark. Let me know if you want me to do that. (And to be clear, this is now for the benefit of anyone *else* who wants to pursue this, not for me, I'm gong to continue learning this API against a server that it likes.
On Tue, May 15, 2018 at 3:17 PM Bernd Eckenfels <ecki at zusammenkunft.net> wrote:
When you talk about HTT/2.0 Upgrades, do you also deal with TLS? You can use a network tracer instead of curl to debug the whole exchange.
But you might need to turn on TLS debug to dump the session key (so the protocol analyser like wireshark can actually decrypt it)
BTW you added one empty line too much in your Simulator Server now, so it’s 62 bytes.
Gruss
Bernd
--
http://bernd.eckenfels.net
From: net-dev <net-dev-bounces at openjdk.java.net> on behalf of Simon Roberts <simon at dancingcloudservices.com>
Sent: Tuesday, May 15, 2018 10:47:35 PM
To: net-dev at openjdk.java.net
Subject: Re: EOF excption in HTTP 1.1 server interaction
Well, I give up. I'ts something to do with nodejs (which would seem like a popular enough server to be of interest, but whatever) and perhaps the way that node responds when asked to perform an HTTP2.0 upgrade.
Node generated a response that caused httpClient to fail. I used curl to try to extract that response and pasted it into my "fake server". The response from my fake server works. Of course, the one remaining difference is that I cannot tell how node might be reacting to the upgrade request, since curl didn't issue that part of the request. So, I think the problem is there.
public static final String[] response = {
"HTTP/1.1 200 OK",
"X-Powered-By: Expressd",
"Content-Type: text/html; charset=utf-8",
"Content-Length: 60",
"ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
"Date: Tue, 15 May 2018 20:34:49 GMT",
"Connection: keep-alive",
"",
"",
"<html><body><h1>Heading</h1><p>Some Text</p></body></html>",
""
};
public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("< " + line);
if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
out.print(st + "\r\n");
System.out.println("> " + st);
}
out.flush();
s.close();
}
}
On Tue, May 15, 2018 at 1:55 PM Simon Roberts <simon at dancingcloudservices.com> wrote:
Wooops, bran failure. JLS 3.10.4 :( Will fix to use \r\n and get back...
On Tue, May 15, 2018 at 1:43 PM Bernd Eckenfels <ecki at zusammenkunft.net> wrote:
Try using out.print(st+“\n\r“); instead. (And Account for the extra bytes in the body as well or output the last string without the EOLs.
Gruss
Bernd
--
http://bernd.eckenfels.net
From: net-dev <net-dev-bounces at openjdk.java.net> on behalf of Simon Roberts <simon at dancingcloudservices.com>
Sent: Tuesday, May 15, 2018 8:44:08 PM
To: net-dev at openjdk.java.net
Subject: Re: EOF excption in HTTP 1.1 server interaction
Thanks for the clarification; as I mentioned, I tried a number of variations, with and without the "excess" empty lines. I also copied the output from a curl session with a server that provides a successful interaction with the client (so the content length etc were all correct). In every case the *send* failed regardless. I'm finding myself inclined to believe that something about the way the upgrade to 2.0 request is being handled is relevant. That said, I'm unsure what line endings I'm sending in my fake server, but I'm running on Linux, not windows (perhaps that's your point).
So that we can finally put to bed one way or the other the suggestion that this is the server-side's fault, perhaps you could indicate an exact (minimal) string, that you believe should work in my fake-server? I'm not an expert in the HTTP specification, so it could be pretty inefficient to keep cycling round this loop ;)
On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels <ecki at zusammenkunft.net> wrote:
Your example code writes 2 (emp5) lines more than the Content-Length includes. You should also use crlf for the end of http headers (Sample works only on Windows)ö
Gruss
Bernd
Gruss
Bernd
--
http://bernd.eckenfels.net
From: net-dev <net-dev-bounces at openjdk.java.net> on behalf of Simon Roberts <simon at dancingcloudservices.com>
Sent: Tuesday, May 15, 2018 7:22:27 PM
To: net-dev at openjdk.java.net
Subject: EOF excption in HTTP 1.1 server interaction
(Added subject line, sorry, not sure how I missed that in the first place!)
I can pretty much confirm this has nothing to do with content length. I wrote the code below to allow experimentation and even as it stands, which I believe has a correct content length, and a bunch of other stuff removed from the response) the client still fails. I also tried it with various combinations of the response lines commented out, and all of them present, failure every time.
If you'd like to suggest the combination that "should" work, I'd be happy to try it of course.
public final class Main {
public static final String[] response = {
"HTTP/1.1 200 OK",
// "X-Powered-By: Express",
// "Content-Type: text/html; charset=utf-8",
"Content-Length: 58",
// "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
// "Date: Tue, 15 May 2018 00:18:47 GMT",
// "Connection: keep-alive",
"",
"<html><body><h1>Heading</h1><p>Some Text</p></body></html>",
"",
""
};
public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("< " + line);
if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
out.println(st);
System.out.println("> " + st);
}
out.flush();
s.close();
}
}
On Tue, May 15, 2018 at 9:46 AM Chris Hegarty <chris.hegarty at oracle.com> wrote:
Simon,
Only a partial reply, I’ll reply with other details later.
> On 15 May 2018, at 16:10, Simon Roberts <simon at dancingcloudservices.com> wrote:
>
> If I understand you correctly, you are saying that the "simple" version of the code-code by the way that the project's main web page presents as it's first example--is so simple that it's actually unusable in any production scenario.
That is not what I am saying. What I am saying is that the String handler is a convenience handler that buffers all the response data and returns it once decoded. In some circumstances it is just not possible to return, as a String, the response data, if the server behaves incorrectly. In the scenario you are encountering it appears that the server is returning too little data. It may not be possible to always decode this partial data. And even if you do decode this partial data, something further up the stack is likely to fail, if parsing JSON for example.
My point is that if you want fault tolerance in the case of a misbehaving server there are other ways to achieve that.
> I know I cannot use code for production that fails to read any data from a (presumably) merely mis-configured server (a server from which all other tools successfully read data.
What do you hope to do with partial data read from the server? If it’s JSON can you decode it, or a gif can you display it?
> Did I interpret correctly, or did I miss something? If correctly, I'd be surprised if that doesn't strike you as sub-optimal to the point your pride in your work would want to make it more usable.
We do have pride in our work. I am engaging here to try to help you.
> It doesn't seem inappropriate to report a "warning" situation using an exception? Would not an aggregate return that includes data, headers, status code, ... and warnings be more helpful in this case?
There are other ways to achieve that, but IMO doing so for the String handler would not be helpful to the vast majority of Java developers, that would not check the carried warning.
> Mis-configured servers (assuming that's the cause) are not uncommon around the web.
Sure, but many clients will not be able to operate correctly with such, it’s a matter of where and how they fail.
> But perhaps more importantly, whatever the problem is, it is not fixed by your code. The error is actually thrown by the *send* call, as I've now determined as a result of trying your code. (I modified it very slightly so as to complete it, and catch the exception.)
D’oh! Apologies, that’s what I get for sending something without testing it more carefully, but you seem to have gotten past it.
-Chris
--
Simon Roberts
(303) 249 3613
--
Simon Roberts
(303) 249 3613
--
Simon Roberts
(303) 249 3613
--
Simon Roberts
(303) 249 3613
--
Simon Roberts
(303) 249 3613
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180516/6f02f3d6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 38BB691674DF41C9AC70F98ABC83096A.png
Type: image/png
Size: 158 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180516/6f02f3d6/38BB691674DF41C9AC70F98ABC83096A-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: A7327B1F18ED4869A3E0D3D6C79FAB10.png
Type: image/png
Size: 158 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180516/6f02f3d6/A7327B1F18ED4869A3E0D3D6C79FAB10-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AB135F9478894B69AAAC1CE96D081B40.png
Type: image/png
Size: 159 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180516/6f02f3d6/AB135F9478894B69AAAC1CE96D081B40-0001.png>
More information about the net-dev
mailing list