<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><div class=""><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span></div><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 25, 2024, at 8:34 PM, Robert Engels <<a href="mailto:robaho@me.com" class="">robaho@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">A couple more data points, I built a few more of the servers from <a href="https://github.com/TechEmpower/FrameworkBenchmarks/tree/master?tab=readme-ov-file" class="">https://github.com/TechEmpower/FrameworkBenchmarks/tree/master?tab=readme-ov-file</a> including several non-Java (Rust, Go) - and all showed roughly the same latency (~12-13ms) and throughput, so I suspect it is some limiting factor in the OS.<div class=""><br class=""></div><div class="">Now, how the Java one with platform threads achieved such low latency, I don’t know…</div><div class=""><div class=""><br class=""></div><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jun 25, 2024, at 8:37 AM, Robert Engels <<a href="mailto:robaho@me.com" class="">robaho@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I tested with the Loom EA build (Interestingly, I needed to turn off the OSX firewall - there was no prompt “Allow network access?” and no errors, but the client could not connect. Even if I added the java binary to the allowed list - something to be aware of).<div class=""><br class=""></div><div class="">With the default, it made no difference:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Thread Stats Avg Stdev Max +/- Stdev</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Latency 13.74ms 3.67ms 140.26ms 98.12%</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Req/Sec 1.18k 101.17 2.44k 92.82%</span></div></div><div class=""><div class=""><br class=""></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">With pollerMode=1: (note the huge increase in the max requests per thread)</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Thread Stats Avg Stdev Max +/- Stdev</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Latency 13.81ms 5.72ms 442.92ms 98.91%</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Req/Sec 1.18k 125.92 11.17k 95.90%</span></div></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font></div><div class=""><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">With pollerMode=2:</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Thread Stats Avg Stdev Max +/- Stdev</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Latency 13.62ms 3.25ms 440.39ms 97.55%</span></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Req/Sec 1.18k 104.44 5.19k 91.89%</span></div></div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><div style="color: rgb(255, 255, 255); font-family: Helvetica; font-size: 16px; caret-color: rgb(255, 255, 255);" class=""><span style="caret-color: rgb(0, 0, 0);" class="">So no option approaches the platform thread latency.</span></div><div style="color: rgb(255, 255, 255); font-family: Helvetica; font-size: 16px; caret-color: rgb(255, 255, 255);" class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class="">Here is the vthread summary captured using the default pollerMode:</span></font></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">iMac:httpserver-robaho robertengels$ jcmd 99219 Thread.vthread_summary</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">99219:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><root> [platform threads = 13, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- java.util.concurrent.ThreadPerTaskExecutor@3b764bce [platform threads = 0, virtual threads = 1024]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- java.util.concurrent.ScheduledThreadPoolExecutor@80820e0 [platform threads = 1, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- java.util.concurrent.ScheduledThreadPoolExecutor@4e40fef8 [platform threads = 0, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- java.util.concurrent.ScheduledThreadPoolExecutor@72e23c5b [platform threads = 1, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- ForkJoinPool-1/jdk.internal.vm.SharedThreadContainer@2b84658e [platform threads = 8, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- ForkJoinPool.commonPool/jdk.internal.vm.SharedThreadContainer@368551f2 [platform threads = 0, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> +-- java.util.concurrent.ThreadPoolExecutor@2f2ae4f0 [platform threads = 0, virtual threads = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85); min-height: 19px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Default virtual thread scheduler:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">java.util.concurrent.ForkJoinPool@7fb948b5[Running, parallelism = 8, size = 8, active = 4, running = 1, steals = 4360536, tasks = 0, submissions = 0]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85); min-height: 19px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Timeout schedulers:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0] java.util.concurrent.ScheduledThreadPoolExecutor@80820e0[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 2]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[1] java.util.concurrent.ScheduledThreadPoolExecutor@72e23c5b[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 4]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85); min-height: 19px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Read I/O pollers:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0] sun.nio.ch.KQueuePoller@3ff9782b [registered = 1011]</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85); min-height: 19px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Write I/O pollers:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0] sun.nio.ch.KQueuePoller@58e6b7f4 [registered = 0]</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">iMac:httpserver-robaho robertengels$ jcmd 99219 VM.version</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">99219:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">OpenJDK 64-Bit Server VM version 24-loom+1-17</span></div><div style="margin: 0px; font-stretch: normal; font-size: 14px; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgba(255, 255, 255, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">JDK 24.0.0</span></div></span></div></div><div style="color: rgb(255, 255, 255); font-family: Helvetica; font-size: 16px; caret-color: rgb(255, 255, 255);" class=""><br class=""></div></div></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font><blockquote type="cite" class=""><div class="">On Jun 25, 2024, at 6:02 AM, robert engels <<a href="mailto:robaho@me.com" class="">robaho@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="content-type" content="text/html; charset=utf-8" class=""><div dir="auto" class=""><div dir="ltr" class=""></div><div dir="ltr" class="">Will do and I’ll report back. </div><div dir="ltr" class=""><br class=""><blockquote type="cite" class="">On Jun 25, 2024, at 3:14 AM, Alan Bateman <<a href="mailto:Alan.Bateman@oracle.com" class="">Alan.Bateman@oracle.com</a>> wrote:<br class=""><br class=""></blockquote></div><blockquote type="cite" class=""><div dir="ltr" class="">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" class="">
On 24/06/2024 23:03, Robert Engels wrote:<br class="">
<blockquote type="cite" cite="mid:777E865C-A9CA-46C8-B057-EB9D9448E4DE@me.com" class="">
<div style="word-wrap:break-word; line-break:after-white-space" class="">
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
</div>
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
<br class="">
</div>
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
<span class="" style="font-family:Helvetica; font-size:16px">The
VT latency is a couple of orders of magnitude higher than
the platform latency.</span></div>
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
<span class="" style="font-family:Helvetica; font-size:16px"><br class="">
</span></div>
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
<span class="" style="font-family:Helvetica; font-size:16px">I
suspect that what is happening is that the virtual threads
are parking waiting for work too soon - or that the read
& writes are always causing a context switch -
regardless of if the call is very short (room in the kernel
socket buffers).</span></div>
<div class="" style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo;">
<span class="" style="font-family:Helvetica; font-size:16px"><br class="">
</span></div>
<div class="" style="margin: 0px; font-stretch: normal; line-height: normal;">
Can you suggest any remedies? It feels like the scheduler
needs some heuristics on average call time to determine and a
spin loop to avoid this sort of thing.</div>
<br class="">
</div>
</blockquote>
There may be a combination of things here. From the stack traces,
this looks like JDK 21 + macOS. There has been significant changes
to both the scheduler (ForkJoinPool) and the I/O poller mechanism
since then. Would you have time to out the latest EA build [1] to
see if you get similar latency? It would be interesting to see data
with -Djdk.pollerMode=1 and 2 (the default on macOS is "a"). Also
would be useful to see `jcmd <pid> Thread.vthread_summary`
output while the benchmark is running as that will reveal more about
the scheduler and how many virtual threads are blocked on socket
I/O.<br class="">
<br class="">
[1] <a class="moz-txt-link-freetext" href="https://jdk.java.net/loom/">https://jdk.java.net/loom/</a><br class="">
</div></blockquote></div></div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></div></div></div></blockquote></div><br class=""></div></div></body></html>