BufferedInputStream#fill() hanging in SocketInputStream#read0()

[David Holmes]

Andreas,

I can't comment on the patch but I was just reading up on this issue.

It seems to me that the fundamental problem here is the very notion of 
trying to use a buffered-stream in these circumstances. Or one step 
further the notion of using a stream-metaphor for a socket in these 
circumstances is not appropriate. When you have a protocol like HTTP 
there is a very clear definition of end-of-data: request -> response. 
When you send a request you get back a response and there is a definite 
end-of-response indicator. To that end your "stream" must be protocol aware.

In a way your patch is one step towards making the stream 
protocol-aware: that there is a known expected amount of data to receive.

Sorry not particularly helpful ... this just reminded me of 20+ years 
ago in Data Comms classes writing protocol specific packet processing 
code :) streams are only good when there is a continuous flow of data to 
read.

David

Andreas Kohn said the following on 09/07/10 01:01:
> Hi,
> 
> in an application that makes heavy use of JAX-RS, and for that reason
> small HTTP connections, I frequently observe situations where
> BufferedInputStream would hang like this:
> 
> "ae03a305-557e-4db6-b9b8-2bf50f056aaf" prio=10 tid=0x00002aab0175a000 nid=0x30dc runnable [0x0000000048cb3000]
>    java.lang.Thread.State: RUNNABLE
>         at java.net.SocketInputStream.socketRead0(Native Method)
>         at java.net.SocketInputStream.read(SocketInputStream.java:129)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>         - locked <0x00002aaab605b670> (a java.io.BufferedInputStream)
>         at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
>         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632)
>         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1072)
> 
> Typically when looking into those hangs with a debugger it seems that the HTTP response
> was received fully, but the BufferedIInputStream tries to fill its buffer with more bytes than
> were originally requested by the HttpClient. The HTTP server is now waiting for the client to
> continue sending requests, while the client is stuck, hoping that the server will send more bytes
> to fill its buffer with.
> 
> Currently the only option I have in this case is restart the application (even killing the server
> does not lead to a connection reset for some reason!), which in many cases is quite unacceptable.
> Disabling Keep-Alive connections helps if the hanging occurs when reading response bodies, but not
> when the hanging occurs while the header is being read.
> 
> 
> I believe this essentially the issue 6192696 (BufferedInputStream.read(byte[], int, int)
> can block if the entire buffer can't be filled). I understand that this issue had had multiple
> fix-attempts, which could not be done due reliance on a proper and performing #available() implementation. 
> 
> But, wouldn't it be enough to just prevent fill() from filling more than the user wanted to
> read in the first place? For the HTTP example the Content-Length gives that information, and the
> server will not send more than that, but on the other hand it is safe for the connection to block
> until those bytes are received.
> 
> Attached is a patch that implements that idea, am I missing something here? Note that I left #read()
> without arguments alone.
> I've been running with this patch for a few weeks now in test environments, and it seems to have no
> negative impact on classloading speed as detailed in the original bug comment trail.
> 
> Regards,
> --
> Andreas
> 
>