[External] : Re: Performance of pooling virtual threads vs. semaphores

[Ron Pressler]

> On 6 Jun 2024, at 02:27, robert engels <rengels at ix.netcom.com> wrote:
> 
> Totally agree, but then I guess I don’t understand how “most submissions would be internal” holds. It seems there is an unstated expectation of the workloads used with virtual threads. It may be as simple as “use virtual threads when your workload looks this…”, “use pooled virtual threads when your workload looks like this…”, use “platform threads…”, etc.
> 
> I’m not saying anything needs to change, only that I think more clarity on these dynamics would help. And it is totally ok to say “for now…”.
> 
> Just my two cents.
> 

To be clear, I don’t know if external submissions play a role here, only that this is one glaring aspect in which the benchmark differs quite significantly from the intended workload, and because we ultimately care about the actual workload — not a microbenchmark — if we were to spend time analysing performance, we’d rather do it on something that is more likely to help that goal.

As to your question, the scheduler privileges internal submissions somewhat because that tends to help real-world programs. In other words, it responds to what programs do, not the other way around. So I don’t think you should take that into account when using virtual threads. External submissions are also perfectly fine, they typically happen as a result of some external IO event, and they tend to mix with internal submissions. I wouldn’t worry about it, and I certainly can’t give performance advice on hypothetical workloads; there are too many variables.

The only general advice is: use virtual threads when there are many (let’s say thousands and upward) of concurrent tasks (i.e. tasks whose lifetimes naturally overlap), and give each task its own virtual thread. When the number of concurrent tasks is small (as is often the case in parallel computation workloads) then there is a chance that directly managing platform threads would perform better. But that’s just general, approximate advice. For best performance always profile your program and optimise based on its specific profile.

— Ron