Virtual vs platform performance at 10K HTTP requests

[Cay Horstmann]

I did both with David's benchmark. Of course, the semaphore is the way to go. With default Tomcat settings, 200 or 2000 permits work fine, and then virtual threads give significantly better throughput than a small number of platform threads. As one would expect.

But back to those 10000 simultaneous connections. As David observed, left unthrottled, virtual threads do really poorly. In my experiment, *much* more poorly than platform threads, when using a Executors.newCachedThreadPool().

I am wondering why that would be, and whether it is something that is worth addressing, because it seems like something that people could run into in practice. FWIW, I stumbled upon https://bugs.openjdk.org/browse/JDK-8360046 which addresses a somewhat similar scenario.

Cheers,

Cay

Il 26/11/2025 12:18, Thomas Matthijs ha scritto:
> A more honest comparison would be using a Executors.newCachedThreadPool() so it is also unlimited connections, or using a Semaphore with 4 permits around the virtual
> 
> Regards
> 
> On Wed, Nov 26, 2025, at 11:32, David wrote:
>> Hi,
>>
>> There seems to be a limit causing a 12-30 second delay after establishing ~8K connections too quickly. 10K platform threads do not see or reach this limit.
>>
>> The simplest workaround I found was adding System.out.println("1"); right before making each request. The synchronized lock is probably doing the heavy lifting here. This makes the platform and virtual threads perform about the same.
>>
>> Increasing the server-side Tomcat settings also resolves the stalling:
>>
>>   * server.tomcat.accept-count=10000
>>   * server.tomcat.max-connections=20000
>>   * server.tomcat.threads.max=500
>>   * server.tomcat.threads.min-spare=50
>>
>> However, with these higher server limits, I'm seeing different throughput characteristics:
>>
>>   * Platform threads: 2.3 ops/s
>>   * Virtual threads: 0.9 ops/s (about half the speed)
>>
>>
>>     I assume it also benefits from being able to use HTTP persistent
>>     connections.
>>
>>
>> Just trying to understand, does that mean that virtual threads don't have this? Because running Socket Statistics in the background showing 17K established connections while running virtual threads vs 140 when using platform threads.
>>
>> At this point I am just trying to understand where this delay is coming from and why virtual threads trigger this.
>>
>> Thanks in advance!
>>
>> Kind regards,
>> David
>>
>>
>>
>> On Wed, 26 Nov 2025 at 10:33, Alan Bateman <alan.bateman at oracle.com <mailto:alan.bateman at oracle.com>> wrote:
>>
>>
>>
>>     On 26/11/2025 00:25, Robert Engels wrote:
>>     > Your platform test is limiting to at most 4 outstanding requests.
>>
>>     I assume it also benefits from being able to use HTTP persistent
>>     connections.
>>
>>     The benchmark using virtual threads is very different, it tries to
>>     establish 10k connections in a burst. Do you know what connection
>>     backlog is used by Tomcat? It may require adjusting net.core.somaxconn
>>     (kern.ipc.somaxconn on macOS) and other settings. ~8100 may be 8k and
>>     maybe there is something in the Tomcat or Spring connection that
>>     controls this.
>>
>>     -Alan
> 

-- 

Cay S. Horstmann | https://horstmann.com