[httpclient] HTTP2: Memory Leak with Proxy
Albert Schimpf
albi646 at gmx.de
Wed Aug 8 13:45:05 UTC 2018
Chris,
that's good to hear, I'll try it out.
Best,
Albert
On 07.08.2018 18:53, Chris Hegarty wrote:
> Albert,
>
> I haven’t yet looked at what happens in JDK 10, but just to say, since things have moved on a lot in JDK 11 EA, that the same test runs with a reasonable amount of memory and CPU with JDK 11 EA [1]. I updated the test a little, since some the names have been changed with the standardisation ( see javadoc for more details [2] ).
>
> -Chris.
>
> [1] http://jdk.java.net/11/
> [2] https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/package-summary.html
>
> ---
>
> $ cat BadProxyLeak.java
>
> import java.net.InetSocketAddress;
> import java.net.ProxySelector;
> import java.net.URI;
> import java.net.http.HttpClient;
> import java.net.http.HttpRequest;
> import java.net.http.HttpResponse;
> import java.util.concurrent.TimeUnit;
>
> public class BadProxyLeak {
> public static void main(String[] args) throws InterruptedException {
> // 1st heap dump
> // Thread.sleep(10000);
>
> {
> HttpClient client = HttpClient.newBuilder()
> // malicious proxy?
> .proxy(ProxySelector.of(new InetSocketAddress("165.165.248.90", 8080)))
> .build();
>
> HttpRequest req = HttpRequest
> // target is not relevant
> .newBuilder(URI.create("https://www.google.de"))
> // leak occurs only with HTTP_2
> .version(HttpClient.Version.HTTP_2)
> // .version(HttpClient.Version.HTTP_1_1)
> .GET()
> .build();
>
>
> // use same client for every request
> // most likely happens on the first retry, if not try again or increase i
> for (int i = 0; i < 10; i++) {
> System.out.println("Request " + i);
> // body handler is not relevant
> HttpResponse.BodyHandler<?> t = HttpResponse.BodyHandlers.ofString();
>
> try {
> // happens with both async and sync send
> client.sendAsync(req, t).get(15, TimeUnit.SECONDS);
> // client.send(req, handler);
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>
> // 3rd heap dump
> System.out.println("Generating heap dump manually");
> // space is not released
> Thread.sleep(60000);
> }
> }
>
>
>> On 7 Aug 2018, at 17:05, Chris Hegarty <chris.hegarty at oracle.com> wrote:
>>
>>
>>> On 7 Aug 2018, at 16:55, Albert Schimpf <albi646 at gmx.de> wrote:
>>>
>>> Hi,
>>>
>>> by bad I mean that this proxy is the only one out of ~1000 proxies which causes this behavior. It's also the only one which causes SSLExceptions (General SSL engine problem) and EOFExceptions. I don't think that particular proxy is a valid proxy. Using curl indicates that this is a SSL handshake problem (https):
>>>
>>> * Rebuilt URL to: www.google.de/
>>> * Trying 165.165.248.90…
>> Oh, you mean the proxy at that actual IP address. Ok got it.
>>
>>
>>> * TCP_NODELAY set
>>> * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0)
>>> * successfully set certificate verify locations:
>>> * CAfile: /etc/ssl/certs/ca-certificates.crt
>>> CApath: /etc/ssl/certs
>>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>>> * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080
>>> * Closing connection 0
>>> curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080
>>>
>>> I don't know that much about the protocol itself, so I don't think I can help very much.
>>>
>>> Thank you for looking into this!
>>
>> -Chris.
>>
>>
>>> Best,
>>> Albert
>>>
>>>
>>> On 07.08.2018 12:05, Chris Hegarty wrote:
>>>> Hi Albert,
>>>>
>>>> Very strange indeed. Thanks for reporting it, I’ll investigate.
>>>>
>>>> What do your mean by “bad proxy”. What is bad about it, and how does it behave?
>>>>
>>>> -Chris.
>>>>
>>>>
>>>>> On 7 Aug 2018, at 10:25, Albert Schimpf <albi646 at gmx.de> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I stumbled upon some strange behavior when using the new Java httpclient.
>>>>>
>>>>> The issue is very simple to reproduce. Send a GET request via a known bad proxy:
>>>>>
>>>>> HttpClient client = HttpClient.newBuilder()
>>>>> .proxy(ProxySelector.of(BAD_PROXY))
>>>>> .build();
>>>>>
>>>>> HttpRequest req = HttpRequest
>>>>> // target is not relevant
>>>>> .newBuilder(...)
>>>>> .GET()
>>>>> .build();
>>>>>
>>>>> // body handler is not relevant
>>>>> HttpResponse.BodyHandler<?> t = HttpResponse.BodyHandler.asString();
>>>>>
>>>>> // happens with both async and sync send
>>>>> client.sendAsync(req, t).get(30, TimeUnit.SECONDS);
>>>>>
>>>>> The result is that the heap size increases dramatically (to about 1.5GB) and resources are not released. CPU consumption increases by a constant factor, too. I have tried many variations of the above code, and the only thing which seems to work (i.e. heap size does not explode) is to set the HTTP version to 1.1.
>>>>>
>>>>> In my main application this leads to both memory and CPU starvation (4GB memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst.
>>>>>
>>>>> I have attached a working example code with a bad proxy. I uploaded the generated garbage collection log and three heap dumps (before, during, and after the request) to dropbox:
>>>>>
>>>>> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip
>>>>>
>>>>> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue 100% of times.
>>>>>
>>>>> Am I doing something wrong? Is this to be expected if one somehow happens to use a bad proxy? If this is to be expected, how can I protect my application against such behavior?
>>>>>
>>>>>
>>>>> Best,
>>>>> Albert
>>>>> <mon.png><Main.java>
More information about the net-dev
mailing list