<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="FR" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Sorry, I mean preemptive time sharing then.
</span><span lang="EN-US" style="font-family:"Apple Color Emoji";mso-fareast-language:EN-US">😊</span><span lang="EN-US" style="mso-fareast-language:EN-US">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">(From user perspective, right now Loom doesn’t _<i>look</i>_ preemptive even if it is under the hood since switch can occur only at specific points in user code unless I’m missing something.)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">As I said, it would not be as bad as the NodeJS example but still a similar problem: a native thread pinned by CPU-bound request.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">If I have a Kub pod with CPU limit 2, I send 4 CPU-bound http requests concurrently, I really expect them to run concurrently, not 2 then 2 – it’s rather real world scenario.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Especially since the 2 first might be longer, I can’t always know in advance.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Sure, if you have 100% CPU usage all the time there is a problem, but that doesn’t mean incoming requests should be fully stuck as soon as 100% CPU usage is hit on a pod.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">(Note that 100% usage on Pod doesn’t mean 100% on the underlying worker node VM.)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">The problem here is not raw performance but scheduling fairness.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">In my opinion, for CPU-bound requests on vthreads, that would be even better to have slightly worse perf overall but maintain correct fairness.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Otherwise, we will have to be concerned each time about running enough carrier/native threads to support possible worst-case concurrent CPU-bound requests – compared with the awesome
simplification Loom gives regarding blocked IO handling.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">From a different perspective: I think Loom cheap stack/thread parking could be useful also for CPU-bound tasks. It could allow more concurrent CPU-bound tasks to make progress without
improving perf overall.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Arnaud<o:p></o:p></span></p>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">First, virtual thread scheduling is already preemptive — the JDK decides when to preempt a virtual thread without any explicit cooperation from user code. What you’re talking about is
making the decision to preempt based on some expired time-slice on the CPU. This is called time-sharing.</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">Second, no, this isn’t like Node.JS because the scheduler uses all cores. To saturate the scheduler you’d need to hog the CPU on *all* cores at the same time, i.e. reach 100% CPU utilisation.
It is true that the kernel scheduler will employ time-sharing at 100% CPU while the virtual threads scheduler currently doesn’t, but servers don’t normally run at 100% CPU, and when they do I don’t think people are happy with how well they behave under that
condition. I.e. the time-sharing offered by the OS is definitely useful for some things, but making servers work well at 100% CPU doesn’t appear to be one of them — as far as we know.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">Time-sharing does help when you have a small number of low-priority background processing jobs running on the server, but unlike Erlang or Go, Java easily offers that without adding time-sharing
to the virtual thread scheduler.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">So the question is, can time-sharing for virtual threads actually improve real workloads, and, if so, what should be the scheduling algorithm that would achieve that? To know the answer
to this question, we’ll need to see real workloads that could potentially be helped by time-sharing. I haven’t seen one yet, but if anyone finds any, we’d certainly take a close look at that.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">As for file IO, we’re currently working on employing io_uring (where available) to make that properly block the virtual thread rather than the OS thread. By the way, the current implementation
will actually temporarily increase the number of OS threads used by the scheduler when doing long filesystem operations.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="color:black">— Ron<o:p></o:p></span></p>
</div>
</div>
</div>
<DIV>
Unless otherwise stated above:<BR>
<BR>
Compagnie IBM France<BR>
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex<BR>
RCS Nanterre 552 118 465<BR>
Forme Sociale : S.A.S.<BR>
Capital Social : 664 069 390,60 €<BR>
SIRET : 552 118 465 03644 - Code NAF 6203Z<BR>
</DIV></body>
</html>