<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Isn’t the HttpClient almost always used to access other services?<div class=""><br class=""></div><div class="">Why would a developer access a malicious service?</div><div class=""><br class=""></div><div class="">I also think there are lots of ways for a service to crash the client - .e.g it could attempt to return a very large response - if the client uses a memory buffered reader, it will cause an OOM as well.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 29, 2024, at 2:42 PM, Andy Boothe <<a href="mailto:andy.boothe@gmail.com" class="">andy.boothe@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Following up here.</div><div class=""><br class=""></div><div class="">I believe I have discovered that it is possible to craft a malicious HTTP response that can cause the built-in HttpURLConnection and HttpClient implementations to throw exceptions. Specifically, HttpURLConnection can be made to throw a NegativeArraySizeException, and HttpClient can be made to throw an OutOfMemoryError. Proof of this behavior is in the attached (very simple) Java programs.<br class=""></div><div class=""><br class=""></div><div class="">This seems like A Bad Thing to me.</div><div class=""><br class=""></div><div class="">I've moved from the dev list to this list based on a recommendation from that list. Is this the right list? If not, can you point me in the right direction? Perhaps a security list?</div><div class=""><br class=""></div><div class="">Thank you,<br class=""></div><div class=""><br clear="all" class=""></div><div dir="ltr" class=""><div class=""><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class="">Andy Boothe<div class=""><b class="">Email</b>: <a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a></div><div class=""><b class="">Mobile</b>: (979) 574-1089<br class=""></div></div></div></div></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 24, 2024 at 4:47 PM Andy Boothe <<a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="">Hello,</div><div class=""><br class=""></div><div class="">I'm moving this thread from jdk-dev to this list on the sage advice of Pavel Rappo.</div><div class=""><br class=""></div><div class="">As a brief recap, it looks like HttpClient and HttpURLConnection do not currently support a way to set the maximum acceptable response header length. As a result, sending HTTP requests with these classes that result in a response with very long headers causes an OutOfMemoryError and a NegativeArraySizeException, respectively. (Simple programs for reproducing the issue are attached.) This seems like A Bad Thing. There is a (very brief) discussion in the thread about how to handle, but of course you guys are the experts.</div><div class=""><br class=""></div><div class="">If my head is on straight and this turns out to be a real issue as opposed to a mistake on my part, I'm keen to help however I can. <br class=""></div><div class=""><br class=""></div><div class=""><div class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class="">Andy Boothe<div class=""><b class="">Email</b>: <a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a></div><div class=""><b class="">Mobile</b>: (979) 574-1089<br class=""></div></div></div></div><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br class="">From: <b class="gmail_sendername" dir="auto">Pavel Rappo</b> <span dir="auto" class=""><<a href="mailto:pavel.rappo@oracle.com" target="_blank" class="">pavel.rappo@oracle.com</a>></span><br class="">Date: Wed, Jul 24, 2024 at 4:30 PM<br class="">Subject: Re: Very long response headers and java.net.http.HttpClient?<br class="">To: Andy Boothe <<a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a>><br class="">Cc: <a href="mailto:jdk-dev@openjdk.org" target="_blank" class="">jdk-dev@openjdk.org</a> <<a href="mailto:jdk-dev@openjdk.org" target="_blank" class="">jdk-dev@openjdk.org</a>><br class=""></div><br class=""><br class="">
<div class="">
<div class=""><font size="2" class=""><span style="font-size:11pt" class="">
<div class="">A proper list would be net-dev at <a href="http://openjdk.java.net/" target="_blank" class="">openjdk.java.net</a>.<br class="">
<br class="">
> On 24 Jul 2024, at 21:13, Andy Boothe <<a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a>> wrote:<br class="">
> <br class="">
> Hello,<br class="">
> <br class="">
> I'm documenting some guidelines for using java.net.http.HttpClient defensively for my team. For example: "Always set a request timeout", "Don't assume HTTP response entities are small and/or will fit in memory", etc.<br class="">
> <br class="">
> One guideline I'd like to document is "Set a maximum for HTTP response header size." However, I can't seem to find a way to set that limit, either in documentation or in OpenJDK code.<br class="">
> <br class="">
> I tried my best to search the archives for this mailing list for any mentions, but came up empty.<br class="">
> <br class="">
> To make sure my head is on straight and there isn't an undocumented limit set by default, I wrote the attached (very quick and dirty) client and server programs. LongResponseHeaderDemoServer opens a raw server socket and reads (what it assumes is) a well-formed
HTTP request, and then prints an HTTP response which includes a response header of infinite length. LongResponseHeaderDemoHttpClient uses java.net.http.HttpClient to make a request and print the response body.<br class="">
> <br class="">
> When I run LongResponseHeaderDemoServer in one terminal and make a curl request to the server in another terminal, this is what curl spits out:<br class="">
> <br class="">
> $ curl -vvv -D - <a href="http://localhost:3000/" target="_blank" class="">http://localhost:3000</a><br class="">
> * Host localhost:3000 was resolved.<br class="">
> * IPv6: ::1<br class="">
> * IPv4: 127.0.0.1<br class="">
> * Trying [::1]:3000...<br class="">
> * Connected to localhost (::1) port 3000<br class="">
> > GET / HTTP/1.1<br class="">
> > Host: localhost:3000<br class="">
> > User-Agent: curl/8.6.0<br class="">
> > Accept: */*<br class="">
> > <br class="">
> < HTTP/1.1 200 OK<br class="">
> HTTP/1.1 200 OK<br class="">
> < Content-Type: text/plain<br class="">
> Content-Type: text/plain<br class="">
> < Connection: close<br class="">
> Connection: close<br class="">
> < Content-Length: 3<br class="">
> Content-Length: 3<br class="">
> * Closing connection<br class="">
> curl: (100) A value or data field grew larger than allowed<br class="">
> <br class="">
> So curl detects the long response header and bails out. Safe and sane.<br class="">
> <br class="">
> However, when I run LongResponseHeaderDemoServer in one terminal and run LongResponseHeaderDemoHttpClient in another terminal, this is what happens:<br class="">
> <br class="">
> $ java LongResponseHeaderDemoHttpClient <br class="">
> Exception in thread "main" java.io.IOException: Requested array size exceeds VM limit<br class="">
> at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:966)<br class="">
> at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)<br class="">
> at LongResponseHeaderDemoHttpClient.main(LongResponseHeaderDemoHttpClient.java:13)<br class="">
> Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit<br class="">
> at java.base/java.util.Arrays.copyOf(Arrays.java:3541)<br class="">
> at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:242)<br class="">
> at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:806)<br class="">
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:246)<br class="">
> at java.net.http/jdk.internal.net.http.Http1HeaderParser.readResumeHeader(Http1HeaderParser.java:250)<br class="">
> at java.net.http/jdk.internal.net.http.Http1HeaderParser.parse(Http1HeaderParser.java:124)<br class="">
> at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Http1Response.java:605)<br class="">
> at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Http1Response.java:536)<br class="">
> at java.net.http/jdk.internal.net.http.Http1Response$Receiver.accept(Http1Response.java:527)<br class="">
> at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.tryAsyncReceive(Http1Response.java:583)<br class="">
> at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Http1AsyncReceiver.java:233)<br class="">
> at java.net.http/jdk.internal.net.http.Http1AsyncReceiver$$Lambda/0x00000008010dbd50.run(Unknown Source)<br class="">
> at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)<br class="">
> at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)<br class="">
> at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)<br class="">
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)<br class="">
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)<br class="">
> at java.base/java.lang.Thread.runWith(Thread.java:1596)<br class="">
> at java.base/java.lang.Thread.run(Thread.java:1583)<br class="">
> <br class="">
> Ostensibly, HttpClient just keeps on reading the never-ending header until it OOMs. This seems to confirm that there is no default limit to header size. It also seems like A Very Bad Thing to me. This suggests that any time a program makes an HTTP request
to an untrusted source using HttpClient, for example when crawling the web, they are at risk of an OOM.<br class="">
> <br class="">
> For grins, I also wrote an application LongResponseHeaderDemoHttpURLConnection that does the same thing as LongResponseHeaderDemoHttpClient, just using HttpURLConnection instead of HttpClient. When I run LongResponseHeaderDemoServer in one terminal and LongResponseHeaderDemoHttpURLConnection
in another terminal, this is what happens:<br class="">
> <br class="">
> $ java LongResponseHeaderDemoHttpURLConnection<br class="">
> Exception in thread "main" java.lang.NegativeArraySizeException: -1610612736<br class="">
> at java.base/sun.net.www.MessageHeader.mergeHeader(MessageHeader.java:526)<br class="">
> at java.base/sun.net.www.MessageHeader.parseHeader(MessageHeader.java:481)<br class="">
> at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:804)<br class="">
> at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:726)<br class="">
> at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1688)<br class="">
> at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)<br class="">
> at java.base/java.net.URL.openStream(URL.java:1161)<br class="">
> at LongResponseHeaderDemoHttpURLConnection.main(LongResponseHeaderDemoHttpURLConnection.java:12)<br class="">
> <br class="">
> So HttpURLConnection doesn't handle things gracefully either, but at least it doesn't OOM. That seems like a bug, too, but perhaps less severe.<br class="">
> <br class="">
> For reference, here's my java version:<br class="">
> <br class="">
> $ java -version<br class="">
> openjdk version "21.0.2" 2024-01-16 LTS<br class="">
> OpenJDK Runtime Environment Corretto-21.0.2.13.1 (build 21.0.2+13-LTS)<br class="">
> OpenJDK 64-Bit Server VM Corretto-21.0.2.13.1 (build 21.0.2+13-LTS, mixed mode, sharing)<br class="">
> <br class="">
> Can anyone check my work, and maybe reproduce? And ideally, can someone with more knowledge than me about java.net.http.HttpClient and/or java.net.HttpURLConnection please comment? Is this real, or have I made a mistake somewhere along the way? If it's real,
what's next? A bug report?<br class="">
> <br class="">
> Andy Boothe<br class="">
> Email: <a href="mailto:andy.boothe@gmail.com" target="_blank" class="">andy.boothe@gmail.com</a><br class="">
> Mobile: (979) 574-1089<br class="">
</div>
</span></font></div>
<div class=""><font size="2" class=""><span style="font-size:11pt" class="">
<div class=""><br class="">
</div>
</span></font></div>
</div>
</div></div></div>
</blockquote></div>
<span id="cid:f_lz7e35y50"><LongResponseHeaderDemoHttpClient.java></span><span id="cid:f_lz7e35yd1"><LongResponseHeaderDemoHttpURLConnection.java></span><span id="cid:f_lz7e35ye2"><LongResponseHeaderDemoServer.java></span></div></blockquote></div><br class=""></div></body></html>