<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
P.S.
<div class=""><br class="">
</div>
<div class="">To explain my hint about benchmarking with many queues, let me say that often what makes the scheduler work harder is not the context-switching itself but finding a task (in this case, a runnable thread) to run. When the amount of contention is
very high compared to the total number of threads, this may be hard and require expensive inter-core chatter. But in more realistic workloads, the level of contention is significantly lower than the total number of threads, so it’s easier for the scheduler
to find a thread to schedule. I.e. in common conditions with, say, 50,000 threads, they will not all be contending for the same data structure, but small groups of them may be contending over multiple data structures, and there will be sufficiently many runnable
threads to keep the scheduler from working hard to find things to run on other cores’ queues.<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 4 Jan 2023, at 15:27, Ron Pressler <<a href="mailto:ron.pressler@oracle.com" class="">ron.pressler@oracle.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Thank you.
<div class=""><br class="">
</div>
<div class="">To make your benchmark more interesting, I would suggest varying both the number of producers and consumers as well as the number of queues they contend over (e.g. 100,000 queues with 1 producer and 1 consumer each, 1000 queues with 100 producers
and 100 consumers each etc.). This would also give you a sense of the kinds of benchmarks we’re using.
<div class=""><br class="">
</div>
<div class="">BTW, the impact on throughput that context switching has on message passing systems is related to the ratio between the average context switching duration and the average wait-time between messages, i.e. context-switching needs to be efficient
only to the point that it is significantly smaller than the wait-time between messages. Once it’s small enough *in comparison*, reducing it further has little impact (see calculation here:
<a href="https://inside.java/2020/08/07/loom-performance/" class="">https://inside.java/2020/08/07/loom-performance/</a>).</div>
<div class=""><br class="">
</div>
<div class="">— Ron<br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On 4 Jan 2023, at 04:59, robert engels <<a href="mailto:rengels@ix.netcom.com" class="">rengels@ix.netcom.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
For some further data points. Using VthreadTest <a href="https://urldefense.com/v3/__https://github.com/robaho/vthread_test__;!!ACWV5N9M2RV99hQ!LFEPXV7dMYdKXhq8nPYKdugapqZ6YjxQ4-sIlUyqDBSyiNDaUEdmYFml7BXnDnv5A_d0yPXv-AoIPZanNQ$" class="">here</a> (essentially
a message passing system):
<div class=""><br class="">
</div>
<div class="">With 32 producers, 32 consumers, 500k messages on an 4/8 core machine:</div>
<div class=""><br class="">
</div>
<div class="">1a. native threads w ABQ: 60-65% system cpu, 20% user cpu, 15-20% idle, total time 189 seconds</div>
<div class="">1b. vthreads w ABQ: 5-10% system cpu, 75% user cpu, 15% idle, total time 63 seconds</div>
<div class="">2a. native threads w RingBuffer, spin=1: 70% system cpu, 30% user cpu, 0% idle, total time 174 seconds</div>
<div class="">2b. vthreads w RingBuffer, spin=1: 13% system cpu, 85% user, 2% idle, total time 37 seconds</div>
<div class="">
<div style="caret-color: rgb(0, 0, 0);" class="">3a. native threads w RingBuffer, spin=32: 68% system cpu, 30% user cpu, 2% idle, total time 164 seconds</div>
<div style="caret-color: rgb(0, 0, 0);" class="">3b. vthreads w RingBuffer, spin=32: 13% system cpu, 85% user, 3% idle, total time 40 seconds</div>
</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class="">(ABQ is stdlib ArrayBlockingQueue)</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class="">The above times have a lot of variance which is not fully accounted for but the interesting thing is that the RingBuffer makes such a huge difference between 1 & 2.</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class="">Even in 2b, there is 13% taken up by the OS - I assume due to thread switching as there is no IO in the test, which means the scheduling can probably be improved.</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class="">I would expect a green thread system to approach 0% idle and 0% system utilization in this type of test. I am “fairly cetain” the code should able to use all carrier threads 100%. Maybe the system % is going
to something else? (You can use the SpinTest - comment out the println - and see that 100% cpu bound “do nothing” test that allocates no objects still uses more than 25% system cpu - which seems odd).</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class="">Here is a async profile capture of 3b:</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0);" class=""><span id="cid:CE85336B-FD0B-40A9-9616-4944823FCAAC" class=""><PastedGraphic-3.png></span></div>
<div class=""><br class="">
</div>
<div class="">Notice that the vast majority of the time is used in internal context switching.</div>
<div class=""><br class="">
</div>
<div class="">I can “fairly agree” with the project’s stated bias towards server systems with 1000’s of threads (I do think 1000’s of threads is enough vs. millions of threads), but I hope this can be addressed moving forward. I think the CSP (communicating
sequential processes) model (close to Actor model) simplifies a lot of concurrent programming concerns but it requires highly efficient context switching and queues to work well.</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>