<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 6 Jan 2023, at 09:42, Arnaud Masson <<a href="mailto:arnaud.masson@fr.ibm.com" class="">arnaud.masson@fr.ibm.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">A long CPU-bound request (without any IO/yield) on a vthread would virtually “pin” its native thread, which is a thing we don’t generally want, no?<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">(A server can have both CPU-bound and IO-bound requests at the same time.)<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">That would be a bit like blocking the even loop on NodeJS:<span class="Apple-converted-space"> </span><a href="https://nodejs.org/en/docs/guides/dont-block-the-event-loop/#blocking-the-event-loop-json-dos" style="color: rgb(5, 99, 193); text-decoration: underline;" class="">https://nodejs.org/en/docs/guides/dont-block-the-event-loop/#blocking-the-event-loop-json-dos</a>(mitigated
by the fact there are more than 1 native thread to run vthreads in general).<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">Wouldn’t vthread preemptive scheduling fix that?<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">Otherwise, I guess we can insert manually Thread.yield in the slow CPU code but it’s not ideal:<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">For (int i=0; i<99999999; i++) {<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""> Thread.yield();<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""> // CPU-only stuff, or maybe old sync file IO<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">}<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class="">Arnaud<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 10pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div>
</div>
</div>
</blockquote>
<br class="">
</div>
</div>
<div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">First, virtual thread scheduling is already preemptive — the JDK decides when to preempt a virtual thread without any explicit cooperation from user code. What you’re talking about
is making the decision to preempt based on some expired time-slice on the CPU. This is called time-sharing.</span>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">Second, no, this isn’t like Node.JS because the scheduler uses all cores. To saturate the scheduler you’d need to hog the CPU on *all* cores at the same time, i.e. reach 100% CPU utilisation.
It is true that the kernel scheduler will employ time-sharing at 100% CPU while the virtual threads scheduler currently doesn’t, but servers don’t normally run at 100% CPU, and when they do I don’t think people are happy with how well they behave under that
condition. I.e. the time-sharing offered by the OS is definitely useful for some things, but making servers work well at 100% CPU doesn’t appear to be one of them — as far as we know.</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">Time-sharing does help when you have a small number of low-priority background processing jobs running on the server, but unlike Erlang or Go, Java easily offers that without adding time-sharing
to the virtual thread scheduler.</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">So the question is, can time-sharing for virtual threads actually improve real workloads, and, if so, what should be the scheduling algorithm that would achieve that? To know the answer to
this question, we’ll need to see real workloads that could potentially be helped by time-sharing. I haven’t seen one yet, but if anyone finds any, we’d certainly take a close look at that.</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">As for file IO, we’re currently working on employing io_uring (where available) to make that properly block the virtual thread rather than the OS thread. By the way, the current implementation
will actually temporarily increase the number of OS threads used by the scheduler when doing long filesystem operations.</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">— Ron</div>
</div>
</body>
</html>