<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Helvetica Neue";
panose-1:2 0 5 3 0 0 0 2 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="FR" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">The average latency of different request types doesn’t matter much, I think.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">What matters (for “interactive” web apps at least) is rather the request latency relatively to its own intrinsic duration.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Example:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Adding 1s to a 10s request is not much worse than adding 10ms to a 100ms request.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Adding 1s to a 100ms request is a more serious problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Not sure to understand the last part “</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">some fixed set of background processing operations”</span><span lang="EN-US" style="mso-fareast-language:EN-US">,
is it about using non-Loom/classic executor when time sharing is more needed? (It was considered as useless when it was suggested earlier as a workaround, so I’m a bit confused).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Sounds like a good workaround but it adds some complexity for the developer that must distinguish in advance which request must go to Loom Executor and which request must go to classic
Executor. (Something Go developers don’t have to worry about now.)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Arnaud<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">On 9 Jan 2023, at 18:34, Robert Engels <</span><a href="mailto:rengels@ix.netcom.com"><span lang="EN-US">rengels@ix.netcom.com</span></a><span lang="EN-US">> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">I think what is not given enough weight in the analysis is that long running tasks are usually deprioritized by the scheduler - not equally time slices - reducing the latency for short requests
- increasing the latency for long/batch requests.<o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">Actually, this is covered as a special case. Even assuming perfect time-sharing, its effectiveness for virtual thread use cases is unclear until we obtain more data from the field.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">This is expected today based on the Linux (and many other) schedulers. The vthread scheduler is breaking from this - which is fine if it has good reasons to do so. <o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">An OS scheduler must be a reasonable compromise for many kinds of threads. Virtual threads are optimised for transaction-processing workloads. I assume it will take some years to gather sufficient
information from the field so that we can tweak our decisions based on data, but after spending years considering various hypotheses, I don’t see a reason to change what we have now without obtaining more data.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">You can read the Go rationale for adding time slicing into the scheduler here </span><a href="https://github.com/golang/go/issues/10958"><span lang="EN-US">https://github.com/golang/go/issues/10958</span></a><span lang="EN-US"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">There were multiple issues it addressed - I’m not sure all of them apply to Java. <o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">When we studied those motivations some years ago, it appeared they do not apply to Java. Once again, we must only solve problems faced by Java developers in the field.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US">— Ron<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:35.4pt">
<span lang="EN-US">On Jan 9, 2023, at 12:19 PM, Ron Pressler <</span><a href="mailto:ron.pressler@oracle.com"><span lang="EN-US">ron.pressler@oracle.com</span></a><span lang="EN-US">> wrote:<o:p></o:p></span></p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <span lang="EN-US"><o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">I think it would be interesting to explain in more detail the effects of scheduling, and why the question of time-sharing is not obvious
and so crucially depends on real-world data. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">Suppose you are given 10 tasks, 5 of them have a processing duration of 0ms, and 5 of them have a duration of 100ms. For simplicity, let’s
assume we have no parallelism. Both a shortest-task-first and a longest-task-first will complete all tasks in 500ms, but their average task latency will be quite different. That of the shortest-task-first scheduler will be 150ms (= (5*0 + 100 + 200 + 300 +
400 + 500)/10), while that of the longest-task-first scheduler will be 400ms (= 100 + 200 + 300 + 400 + 500 + 5*500)/10). A perfect time-sharing scheduler (with zero overhead and infinitesimal granularity) would yield an average latency of 250ms (= 0*5 + 500*5).
Those are big differences!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">But now let’s consider a server where an infinite stream of requests arrive from the outside, half of them with a processing duration of
0ms and half with a duration of 100ms. Regardless of how we schedule the requests in the queue that may form, because the average request duration is 50ms, as long as the request rate is less than or equal to 20 req/s the server will be stable. If the rate
is higher than that, the server will become unstable with requests piling up in an ever-growing queue and the latency will climb to infinity — again, regardless of scheduling policy.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">What about latency? The average latency will depend on the distribution of requests. Without time-sharing, it can range between 50ms and
100ms; with perfect time-sharing it may be much higher (i.e. worse). Perfect time-sharing will decrease the latency of the short tasks (to 0ms!) at the expense of increasing the latency of the long tasks.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">But say we conclude that reducing latencies of short tasks at the expense of long tasks is what everyone always wants; that’s not entirely
obvious, but not completely unreasonable, either. Suppose that at the same request rate of 20 per second, the probability for a long task weren't 0.5 but 0.55, or that instead of 100ms it takes 110 ms. In that situation time sharing can no longer help — the
server will destabilise. Alternatively, suppose that the probability of a long task is 0.05 or that its duration is 50 ms; time sharing is no longer effective, either. So at 20 req/s, within the band of 50-100ms or 0.05-0.5 probability time sharing can help;
above it or below it — it doesn’t.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">Keeping in mind that time-sharing can’t actually be perfect and that no request can actually have a duration of zero, I hope it is now clearer
why we’re so curious to find real-world cases and why simulations provide little insight. It’s easy to construct an artificial simulation at the operational band where time-sharing is effective, but it’s precisely because, in practice, it is most effective
when the server is on the edge of stability and becomes gradually less effective the further away we are from that tipping-point that the most important questions become: how often do servers operate within that operational band, where exactly along that band
do they commonly find themselves, and how does that situation arise in real-world scenarios? Only when we get real-wold data can we answer those questions and can consider the pros and cons, and only then can we either conclude that the work isn’t worth it
or be able to satisfactorily justify it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue"">(Note that for the classic case where time sharing helps— some fixed set of background processing operations — there is no need to add time-sharing
to the virtual thread scheduler, as a better solution is already available.)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:10.0pt;font-family:"Helvetica Neue""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:"Helvetica Neue"">— Ron<o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<DIV>
Unless otherwise stated above:<BR>
<BR>
Compagnie IBM France<BR>
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex<BR>
RCS Nanterre 552 118 465<BR>
Forme Sociale : S.A.S.<BR>
Capital Social : 664 069 390,60 €<BR>
SIRET : 552 118 465 03644 - Code NAF 6203Z<BR>
</DIV></body>
</html>