Virtual vs platform performance at 10K HTTP requests

[David]

Hi,

One last thing I wanted to share. I ran the benchmark with two servers and
one client (three machines). Using the following code to switch between
them. Both servers stop logging between 4400 and 5100 requests (between 8K
- 10K total) when using virtual threads. The platform threads continue
running. I hope this rules out the servers being an issue, a single target
reaches 8K. It is probably something within the client that cannot handle
more than 8K requests.

if(r.nextBoolean()){
    String response1 =  fetchJavaNineStyle(URI.create("
http://192.168.1.44:8080/v1/delay/" + plan.delay).toURL());
    blackhole.consume(response1);
} else {
    String response1 =  fetchJavaNineStyle(URI.create("
http://192.168.1.122:8080/v1/delay/" + plan.delay).toURL());
    blackhole.consume(response1);
}

On Thu, 27 Nov 2025 at 20:12, Daniel Andres Pelaez Lopez <
estigma88 at gmail.com> wrote:

> We are using only one EC2 (no horizontal scaling) with virtual threads to
> output video streaming, and we needed to tweak the OS configuration to make
> it work to support a lot of concurrent connections, we have seen 10K
> concurrent connections no issue using Spring Boot and JDK 24. Something to
> take a look https://docs.gatling.io/concepts/operations/
>
> El jue, 27 nov 2025 a las 13:54, Robert Engels (<robaho at me.com>) escribió:
>
>> This is it. The OS out of the box does not expect to deal with that many
>> connection requests in a short period - especially OSX. It often appears to
>> be a DoS attack if not reconfigured.
>>
>> > On Nov 27, 2025, at 12:46 PM, Alan Bateman <alan.bateman at oracle.com>
>> wrote:
>> >
>> > On 27/11/2025 16:38, Cay Horstmann wrote:
>> >> I did both with David's benchmark. Of course, the semaphore is the way
>> to go. With default Tomcat settings, 200 or 2000 permits work fine, and
>> then virtual threads give significantly better throughput than a small
>> number of platform threads. As one would expect.
>> >>
>> >> But back to those 10000 simultaneous connections. As David observed,
>> left unthrottled, virtual threads do really poorly. In my experiment,
>> *much* more poorly than platform threads, when using a
>> Executors.newCachedThreadPool().
>> >
>> > 10k or 50k+ concurrent connections shouldn't be an issue but it may
>> involve tweaking a number of sysctl.conf/equivalent settings to get there.
>> Elliot Barlas published a fun project at one point where he had several
>> million TCP connections with 2 virtual threads per connection.
>> >
>> > I think the issue we are discussing here is partly about storming the
>> server with SYN packets to establish the TCP connections. Servers typically
>> have rate limiting and other mitigations to avoid SYN flood attacks. This
>> is why we were asking about the connection backlog and other configuration.
>> It's possible the thread pool bench isn't going to see this because it's
>> using synchronous/blocking APIs to establish the TCP connection and so the
>> number of outstanding connects in progress is limited by the size of the
>> thread pool.
>> >
>> > The other part that seems relevant is that the protocol in the
>> benchmarks is HTTP/1.1 which supports persistent connections, essentially
>> re-use of a TCP connection if possible to avoid creating a new TCP
>> connection for each HTTP server. There's another layer of configuration
>> here. I don't know Tomcat's defaults but it is likely that it keep 100 or
>> 200 connections alive for future HTTP requests. Once the concurrency
>> exceeds this then it's likely there is a new TCP connection established for
>> each request. The thread pool bench is 4 threads and likely well below the
>> maximum number of persistent connections.
>> >
>> > So yes, moving from thread pools to virtual threads may shine light on
>> the "next" concurrency bottleneck, the bottle hidden by limiting the number
>> of threads. The guidance that we've given at conferences and in the JEPs is
>> to use a semaphore/equivalent to limit concurrent at the right level of
>> granularity, be that number of concurrent database connections or whatever
>> it might be.
>> >
>> >>
>> >> I am wondering why that would be, and whether it is something that is
>> worth addressing, because it seems like something that people could run
>> into in practice. FWIW, I stumbled upon
>> https://bugs.openjdk.org/browse/JDK-8360046 which addresses a somewhat
>> similar scenario.
>> >
>> > That one is an over subscription issue that arises with producer
>> starting a lot of virtual threads that "do nothing". It helps us to reduce
>> the gap between ForkJoinTasks that "do nothing" with virtual threads that
>> "do nothing".
>> >
>> > -Alan
>> >
>> >
>>
>
>
> --
> Daniel Andrés Pelaez López
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20251127/4f705ca3/attachment.htm>