Virtual vs platform performance at 10K HTTP requests

[Viktor Klang]

All of this is explained by Little's Law primarily, and Universal 
Scalability Law secondarily.

On 2025-11-27 17:38, Cay Horstmann wrote:
> I did both with David's benchmark. Of course, the semaphore is the way 
> to go. With default Tomcat settings, 200 or 2000 permits work fine, 
> and then virtual threads give significantly better throughput than a 
> small number of platform threads. As one would expect.
>
> But back to those 10000 simultaneous connections. As David observed, 
> left unthrottled, virtual threads do really poorly. In my experiment, 
> *much* more poorly than platform threads, when using a 
> Executors.newCachedThreadPool().
>
> I am wondering why that would be, and whether it is something that is 
> worth addressing, because it seems like something that people could 
> run into in practice. FWIW, I stumbled upon 
> https://bugs.openjdk.org/browse/JDK-8360046 which addresses a somewhat 
> similar scenario.
>
> Cheers,
>
> Cay
>
> Il 26/11/2025 12:18, Thomas Matthijs ha scritto:
>> A more honest comparison would be using a 
>> Executors.newCachedThreadPool() so it is also unlimited connections, 
>> or using a Semaphore with 4 permits around the virtual
>>
>> Regards
>>
>> On Wed, Nov 26, 2025, at 11:32, David wrote:
>>> Hi,
>>>
>>> There seems to be a limit causing a 12-30 second delay after 
>>> establishing ~8K connections too quickly. 10K platform threads do 
>>> not see or reach this limit.
>>>
>>> The simplest workaround I found was adding System.out.println("1"); 
>>> right before making each request. The synchronized lock is 
>>> probably doing the heavy lifting here. This makes the platform and 
>>> virtual threads perform about the same.
>>>
>>> Increasing the server-side Tomcat settings also resolves the stalling:
>>>
>>>   * server.tomcat.accept-count=10000
>>>   * server.tomcat.max-connections=20000
>>>   * server.tomcat.threads.max=500
>>>   * server.tomcat.threads.min-spare=50
>>>
>>> However, with these higher server limits, I'm seeing different 
>>> throughput characteristics:
>>>
>>>   * Platform threads: 2.3 ops/s
>>>   * Virtual threads: 0.9 ops/s (about half the speed)
>>>
>>>
>>>     I assume it also benefits from being able to use HTTP persistent
>>>     connections.
>>>
>>>
>>> Just trying to understand, does that mean that virtual threads don't 
>>> have this? Because running Socket Statistics in the background 
>>> showing 17K established connections while running virtual threads vs 
>>> 140 when using platform threads.
>>>
>>> At this point I am just trying to understand where this delay is 
>>> coming from and why virtual threads trigger this.
>>>
>>> Thanks in advance!
>>>
>>> Kind regards,
>>> David
>>>
>>>
>>>
>>> On Wed, 26 Nov 2025 at 10:33, Alan Bateman <alan.bateman at oracle.com 
>>> <mailto:alan.bateman at oracle.com>> wrote:
>>>
>>>
>>>
>>>     On 26/11/2025 00:25, Robert Engels wrote:
>>>     > Your platform test is limiting to at most 4 outstanding requests.
>>>
>>>     I assume it also benefits from being able to use HTTP persistent
>>>     connections.
>>>
>>>     The benchmark using virtual threads is very different, it tries to
>>>     establish 10k connections in a burst. Do you know what connection
>>>     backlog is used by Tomcat? It may require adjusting 
>>> net.core.somaxconn
>>>     (kern.ipc.somaxconn on macOS) and other settings. ~8100 may be 
>>> 8k and
>>>     maybe there is something in the Tomcat or Spring connection that
>>>     controls this.
>>>
>>>     -Alan
>>
>
-- 
Cheers,
√


Viktor Klang
Software Architect, Java Platform Group
Oracle