[External] : Re: Project Loom VirtualThreads hang

[Ron Pressler]

P.S.

To explain my hint about benchmarking with many queues, let me say that often what makes the scheduler work harder is not the context-switching itself but finding a task (in this case, a runnable thread) to run. When the amount of contention is very high compared to the total number of threads, this may be hard and require expensive inter-core chatter. But in more realistic workloads, the level of contention is significantly lower than the total number of threads, so it’s easier for the scheduler to find a thread to schedule. I.e. in common conditions with, say, 50,000 threads, they will not all be contending for the same data structure, but small groups of them may be contending over multiple data structures, and there will be sufficiently many runnable threads to keep the scheduler from working hard to find things to run on other cores’ queues.

On 4 Jan 2023, at 15:27, Ron Pressler <ron.pressler at oracle.com<mailto:ron.pressler at oracle.com>> wrote:

Thank you.

To make your benchmark more interesting, I would suggest varying both the number of producers and consumers as well as the number of queues they contend over (e.g. 100,000 queues with 1 producer and 1 consumer each, 1000 queues with 100 producers and 100 consumers each etc.). This would also give you a sense of the kinds of benchmarks we’re using.

BTW, the impact on throughput that context switching has on message passing systems is related to the ratio between the average context switching duration and the average wait-time between messages, i.e. context-switching needs to be efficient only to the point that it is significantly smaller than the wait-time between messages. Once it’s small enough *in comparison*, reducing it further has little impact (see calculation here: https://inside.java/2020/08/07/loom-performance/).

— Ron

On 4 Jan 2023, at 04:59, robert engels <rengels at ix.netcom.com<mailto:rengels at ix.netcom.com>> wrote:

For some further data points. Using VthreadTest here<https://urldefense.com/v3/__https://github.com/robaho/vthread_test__;!!ACWV5N9M2RV99hQ!LFEPXV7dMYdKXhq8nPYKdugapqZ6YjxQ4-sIlUyqDBSyiNDaUEdmYFml7BXnDnv5A_d0yPXv-AoIPZanNQ$> (essentially a message passing system):

With 32 producers, 32 consumers, 500k messages on an 4/8 core machine:

1a. native threads w ABQ: 60-65% system cpu, 20% user cpu, 15-20% idle, total time 189 seconds
1b. vthreads w ABQ: 5-10% system cpu, 75% user cpu, 15% idle, total time 63 seconds
2a. native threads w RingBuffer, spin=1: 70% system cpu, 30% user cpu, 0% idle, total time 174 seconds
2b. vthreads w RingBuffer, spin=1: 13% system cpu, 85% user, 2% idle, total time 37 seconds
3a. native threads w RingBuffer, spin=32: 68% system cpu, 30% user cpu, 2% idle, total time 164 seconds
3b. vthreads w RingBuffer, spin=32: 13% system cpu, 85% user, 3% idle, total time 40 seconds

(ABQ is stdlib ArrayBlockingQueue)

The above times have a lot of variance which is not fully accounted for but the interesting thing is that the RingBuffer makes such a huge difference between 1 & 2.

Even in 2b, there is 13% taken up by the OS - I assume due to thread switching as there is no IO in the test, which means the scheduling can probably be improved.

I would expect a green thread system to approach 0% idle and 0% system utilization in this type of test. I am “fairly cetain” the code should able to use all carrier threads 100%. Maybe the system % is going to something else? (You can use the SpinTest - comment out the println - and see that 100% cpu bound “do nothing” test that allocates no objects still uses more than 25% system cpu - which seems odd).

Here is a async profile capture of 3b:

<PastedGraphic-3.png>

Notice that the vast majority of the time is used in internal context switching.

I can “fairly agree” with the project’s stated bias towards server systems with 1000’s of threads (I do think 1000’s of threads is enough vs. millions of threads), but I hope this can be addressed moving forward. I think the CSP (communicating sequential processes) model (close to Actor model) simplifies a lot of concurrent programming concerns but it requires highly efficient context switching and queues to work well.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230104/3e09cf59/attachment.htm>