How to reuse TCP connection when using a proxy with java.net.http.HttpClient in HTTP/1.1

[Daniel Fuchs]

Hi Nicolas,

If I understand correctly you are closing the input stream without
draining the bytes. With HTTP/1.1 this will force the connection
to get closed. This is the expected behavior.

best regards,

-- daniel

On 11/01/2021 20:29, Nicolas Henneaux wrote:
> Hi,
> 
> I have a problem when using Java HttpClient with a proxy and HTTP/1.1 
> version. When using a stream body handler, there is no re-use of the TCP 
> connection.
> When using another body handler 
> (java.net.http.HttpResponse.BodyHandlers#ofString() 
> or java.net.http.HttpResponse.BodyHandlers#discarding) there is no such 
> problem.
> When using HTTP/2, there is no problem as well.
> 
> The following code reproduce the problem. The re-use can be checked 
> using /netstat -tn  | grep '3128' /
> class ProxyHttp1Check {
>      public static void main(String[] args) throws Exception {
>          HttpClient httpClient = HttpClient.newBuilder()
>              .version(HttpClient.Version.HTTP_1_1)
>              
> .proxy(ProxySelector.of(InetSocketAddress.createUnresolved("my-proxy.net 
> <http://my-proxy.net>", 3128)))
>              .build();
>          HttpRequest httpRequest = 
> HttpRequest.newBuilder(URI.create("https://my-website.test 
> <https://my-website.test>")).build();
>          for (int i = 0; i < 1_000_000_000; i++) {
>              httpClient
>                  .sendAsync(httpRequest, 
> HttpResponse.BodyHandlers.ofInputStream())
>                  .get()
>                  .body()
>                  .close(); // It seems this is closing the TCP connection
>          }
>      }
> }
> 
> The close comes from the client as underlined in the following dump.
> 
> T 2021/01/11 19:24:59.880557 client-ip:57422 -> proxy-ip:3128 [S] #265
> T 2021/01/11 19:24:59.881067 proxy-ip:3128 -> client-ip:57422 [AS] #266
> T 2021/01/11 19:24:59.881098 client-ip:57422 -> proxy-ip:3128 [A] #267
> T 2021/01/11 19:24:59.881671 client-ip:57422 -> proxy-ip:3128 [AP] #268
>    43 4f 4e 4e 45 43 54 20    61 70 69 2d 74 65 73 74    CONNECT api-test
>    2e 61 77 73 2e 62 61 6d    62 6f 72 61 2e 63 6f 6d   
>   .aws.bambora.com <http://aws.bambora.com>
>    3a 34 34 33 20 48 54 54    50 2f 31 2e 31 0d 0a 0d    :443 HTTP/1.1...
>    0a                                                    .
> T 2021/01/11 19:24:59.882077 proxy-ip:3128 -> client-ip:57422 [A] #269
> T 2021/01/11 19:24:59.883605 proxy-ip:3128 -> client-ip:57422 [AP] #270
>    48 54 54 50 2f 31 2e 31    20 32 30 30 20 43 6f 6e    HTTP/1.1 200 Con
>    6e 65 63 74 69 6f 6e 20    65 73 74 61 62 6c 69 73    nection establis
>    68 65 64 0d 0a 0d 0a                                  hed....
> T 2021/01/11 19:24:59.883627 client-ip:57422 -> proxy-ip:3128 [A] #271
> T 2021/01/11 19:24:59.933932 client-ip:57422 -> proxy-ip:3128 [AP] #272
> T 2021/01/11 19:24:59.952473 proxy-ip:3128 -> client-ip:57422 [AP] #273
> T 2021/01/11 19:24:59.952515 client-ip:57422 -> proxy-ip:3128 [A] #274
> *T 2021/01/11 19:25:00.298765 client-ip:57422 -> proxy-ip:3128 [AF] #275*
> T 2021/01/11 19:25:00.299777 proxy-ip:3128 -> client-ip:57422 [AF] #276
> T 2021/01/11 19:25:00.299809 client-ip:57422 -> proxy-ip:3128 [A] #277
> 
> I have tested it on Java 11.0.9.1, 14.0.2 and 15.0.1 on Ubuntu 20.4.1.
> 
>   $ /usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -version
> openjdk version "11.0.9.1" 2020-11-04
> OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04)
> OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04, mixed 
> mode, sharing)
>   $ java -version
> openjdk version "14.0.2" 2020-07-14
> OpenJDK Runtime Environment (build 14.0.2+12-Ubuntu-120.04)
> OpenJDK 64-Bit Server VM (build 14.0.2+12-Ubuntu-120.04, mixed mode, 
> sharing)
>   $ java -version
> openjdk version "15.0.1" 2020-10-20
> OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)
> $ uname -a
> Linux enix-002 5.8.0-36-generic #40~20.04.1-Ubuntu SMP Wed Jan 6 
> 10:15:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
> 
> Thanks in advance for your help,
> 
> Best regards,
> 
> Nicolas Henneaux