[External] : Re: Project Loom VirtualThreads hang

[Ron Pressler]

On 4 Jan 2023, at 23:07, Rob Bygrave <robin.bygrave at gmail.com<mailto:robin.bygrave at gmail.com>> wrote:

> An application employing 200 virtual threads will not perform significantly better than one employing 200 platform threads

It doesn't have to but we do desire it to not be materially worse. That is, it's more that we don't want to end up in conversations that go like "When we are > 1000 VT then we use Nima, JDBC, and just write simple blocking code but when we are under 200 VT we need to switch to PT and use WebFlux, R2DBC, Rx libraries, and write using callbacks and deal with errors/exceptions differently etc".


There’s a misunderstanding here. An application that end up 200 threads when it uses the style virtual threads were designed for — i.e. thread-per-request — can switch between virtual threads and platform threads by merely choosing an appropriate ExecutorService. There’s never a need to change the programming paradigm. The number of virtual threads used is not determined by the developer, but by the workload on the server.

This is a very important point that I try explaining in talks. Virtual threads are not an execution resource, but a business logic object like a string. If your job is to store a map of user IDs and names, the number of strings used cannot be any higher or lower than the number of users. Similarly for virtual threads. The number of virtual threads will be equal — no higher and no lower — than the number of concurrent tasks your server performs. If that number ends up being 200, i.e. your server operates at a low throughput or your latency is extremely low, then switching between thread implementation is trivial.


> The main difference between virtual threads and platform threads is that you can have many more virtual threads.

In terms of developer impact, I see it more like "With VT blocking is cheap" which then impacts fundamental library choices that developers make (server libs and db drivers) and even the style of code. As a developer we ideally want these choices to work at all the various levels of scale desired at "competitive performance" without having to change our fundamental choices (which imo is approx based on "Reactive" vs "Blocking").

Virtual threads make blocking cheap *because* they can be numerous. Put another way, blocking platform threads is expensive because it consumes a very limited time for a long duration, which, by Little’s law, places a low limit on throughput. The impact of blocking on server throughput due to the number of threads is orders of magnitude higher than due to any other effect.


For myself, I'm not looking for VT to be significantly better in this sub 200 Threads range but to be competitive. Having a "wide sweet spot" impacts choosing a library like Nima (which does not have PT option unlike say Jetty).

It really shouldn’t matter. Thread-per-request is the style that the Java platform supports best. Before virtual thread, it had a performance cost that forced some to pick a different programming style. With virtual threads available, you can use that style no matter what the load on your server is, and easily switch between thread implementation if it makes a difference.

— Ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230104/39f707b2/attachment.htm>