Virtual vs platform performance at 10K HTTP requests

[Daniel Andres Pelaez Lopez]

We are using only one EC2 (no horizontal scaling) with virtual threads to
output video streaming, and we needed to tweak the OS configuration to make
it work to support a lot of concurrent connections, we have seen 10K
concurrent connections no issue using Spring Boot and JDK 24. Something to
take a look https://docs.gatling.io/concepts/operations/

El jue, 27 nov 2025 a las 13:54, Robert Engels (<robaho at me.com>) escribió:

> This is it. The OS out of the box does not expect to deal with that many
> connection requests in a short period - especially OSX. It often appears to
> be a DoS attack if not reconfigured.
>
> > On Nov 27, 2025, at 12:46 PM, Alan Bateman <alan.bateman at oracle.com>
> wrote:
> >
> > On 27/11/2025 16:38, Cay Horstmann wrote:
> >> I did both with David's benchmark. Of course, the semaphore is the way
> to go. With default Tomcat settings, 200 or 2000 permits work fine, and
> then virtual threads give significantly better throughput than a small
> number of platform threads. As one would expect.
> >>
> >> But back to those 10000 simultaneous connections. As David observed,
> left unthrottled, virtual threads do really poorly. In my experiment,
> *much* more poorly than platform threads, when using a
> Executors.newCachedThreadPool().
> >
> > 10k or 50k+ concurrent connections shouldn't be an issue but it may
> involve tweaking a number of sysctl.conf/equivalent settings to get there.
> Elliot Barlas published a fun project at one point where he had several
> million TCP connections with 2 virtual threads per connection.
> >
> > I think the issue we are discussing here is partly about storming the
> server with SYN packets to establish the TCP connections. Servers typically
> have rate limiting and other mitigations to avoid SYN flood attacks. This
> is why we were asking about the connection backlog and other configuration.
> It's possible the thread pool bench isn't going to see this because it's
> using synchronous/blocking APIs to establish the TCP connection and so the
> number of outstanding connects in progress is limited by the size of the
> thread pool.
> >
> > The other part that seems relevant is that the protocol in the
> benchmarks is HTTP/1.1 which supports persistent connections, essentially
> re-use of a TCP connection if possible to avoid creating a new TCP
> connection for each HTTP server. There's another layer of configuration
> here. I don't know Tomcat's defaults but it is likely that it keep 100 or
> 200 connections alive for future HTTP requests. Once the concurrency
> exceeds this then it's likely there is a new TCP connection established for
> each request. The thread pool bench is 4 threads and likely well below the
> maximum number of persistent connections.
> >
> > So yes, moving from thread pools to virtual threads may shine light on
> the "next" concurrency bottleneck, the bottle hidden by limiting the number
> of threads. The guidance that we've given at conferences and in the JEPs is
> to use a semaphore/equivalent to limit concurrent at the right level of
> granularity, be that number of concurrent database connections or whatever
> it might be.
> >
> >>
> >> I am wondering why that would be, and whether it is something that is
> worth addressing, because it seems like something that people could run
> into in practice. FWIW, I stumbled upon
> https://bugs.openjdk.org/browse/JDK-8360046 which addresses a somewhat
> similar scenario.
> >
> > That one is an over subscription issue that arises with producer
> starting a lot of virtual threads that "do nothing". It helps us to reduce
> the gap between ForkJoinTasks that "do nothing" with virtual threads that
> "do nothing".
> >
> > -Alan
> >
> >
>


-- 
Daniel Andrés Pelaez López
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20251127/7b3698be/attachment-0001.htm>