<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi,<div class=""><br class=""></div><div class="">Recently I hit a situation where HTTP requests with the “new” Java HttpClient ran into TCP-connection-resets, but sadly only on GitHub hosted runners and so far I had no luck to reproduce locally. This is why I wanted to setup a reproducer for that.</div><div class=""><br class=""></div><div class="">While trying to build that reproducer, I ran into another unrelated, but locally reproducible j.l.AssertionError in recent versions of Java 11 (Java 17 and newer are fine). Reproducing the AE is pretty easy (although the reproducer is a bit big): clone <a href="https://github.com/snazy/http-client-tcp-reset-repro.git" class="">https://github.com/snazy/http-client-tcp-reset-repro.git</a> + `./gradlew test` using Java 11. I’ve opened <a href="https://bugs.openjdk.org/browse/JDK-8294773" class="">https://bugs.openjdk.org/browse/JDK-8294773</a> for this.</div><div class=""><br class=""></div><div class="">The original issue, for which I’m trying to build a reproducer for, is strange and hard to narrow down, because it only happens on GH hosted runners with the “new” Java HttpClient.</div><div class=""><br class=""></div><div class="">Background (for the TCP-conn-reset): We are using the plain old HttpURLConnection approach, and wanted to use the new HttpClient when it’s available (with Java 11+).</div><div class=""><br class=""></div><div class="">Requests with small payloads (in request and response) are fine - but bigger-ish (couple kb and more) payloads somewhat consistently run into TCP-connection-resets, which do not happen with <font color="#000000" class="">HttpURLConnection and the Apache HTTP client. “Somewhat consistently” means: it happens often, but not always (kinda flaky).</font></div><div class=""><font color="#000000" class=""><br class=""></font></div><div class=""><font color="#000000" class="">The reproducer I was trying to build for that uses Jetty 9, Jetty 11 + the HTTP server in the JDK on the server side + HttpURLConnection, Apache HTTP client and the new Java HttpClient on the client side. The matrix of http server/client implementations is that big, because I wanted to narrow down whether there’s a specific combination that causes the TCP connection reset.</font></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">Some things I tried without success:</span></font></div><div class=""><font color="#000000" class="">* Enable debug logging for the new HttpClient (no luck, debug logging changes the timing and the TCP conn reset doesn’t happen)</font></div><div class=""><font color="#000000" class="">* Delay the request processing with a “Thread.sleep(10)” lets the issue disappear as well</font></div><div class=""><font color="#000000" class="">* Using Java 17 + 18</font></div><div class=""><font color="#000000" class="">* Using a different OpenJDK <span style="caret-color: rgb(0, 0, 0);" class="">distribution (was Azul Zulu, tried Temurin)</span></font></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">* Using a local Docker container with CPU + memory somewhat similar to the GH hosted runner</span></font></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">* Using very similar IP settings (sysctl stuff for net.code + net.ipv4) locally</span></font></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font></div><div class=""><font color="#000000" class="">What I could figure out so far is that the new HttpClient creates a connection for mostly every HTTP request, it reuses existing connections locally. This happens even if there are HTTP request that were completely fine. (<— only on GH hosted runners).</font></div><div class=""><br class=""></div><div class="">I’ll continue to try to get a reproducer for the TCP-connection-reset issue.</div><div class=""><br class=""></div><div class="">Robert</div><div class=""><br class=""></div></body></html>