<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 6 Jan 2023, at 11:49, Arnaud Masson <<a href="mailto:arnaud.masson@fr.ibm.com" class="">arnaud.masson@fr.ibm.com</a>> wrote:</div>
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
</div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">As I said, it would not be as bad as the NodeJS example but still a similar problem: a native thread pinned by CPU-bound request.</span></div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>It is a phenomenon. Whether or not it’s a problem is still to be determined.</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">If I have a Kub pod with CPU limit 2, I send 4 CPU-bound http requests concurrently, I really expect them to run concurrently, not 2 then 2 – it’s rather real world scenario.</span></div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>And in most situations they will run in parallel, and in those they won’t, chances are that you won’t notice or it doesn’t matter as your server is already sputtering.</div>
<blockquote type="cite" class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""></o:p></span></div>
</div>
</blockquote>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">Sure, if you have 100% CPU usage all the time there is a problem, but that doesn’t mean incoming requests should be fully stuck as soon as 100% CPU usage is hit on a pod.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">(Note that 100% usage on Pod doesn’t mean 100% on the underlying worker node VM.)<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">The problem here is not raw performance but scheduling fairness.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">In my opinion, for CPU-bound requests on vthreads, that would be even better to have slightly worse perf overall but maintain correct fairness.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">Otherwise, we will have to be concerned each time about running enough carrier/native threads to support possible worst-case concurrent CPU-bound requests – compared with the awesome simplification Loom gives regarding blocked IO
handling.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class="">From a different perspective: I think Loom cheap stack/thread parking could be useful also for CPU-bound tasks. It could allow more concurrent CPU-bound tasks to make progress without improving perf overall.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
</div>
</div>
</blockquote>
<br class="">
</div>
<div>What you’re showing is not a problem. It’s a scenario that you hypothesise could be a problem, and then you further hypothesise that a certain mechanism would fix the hypothetical problem. But we can’t fix something until we see an actual problem and know
what it is that requires fixing.</div>
<div><br class="">
</div>
<div>In particular, I don’t know why you have to be preemptively concerned about a problem you have not seen, and that platform threads aren’t known to address either. In fact, virtual threads already give you better fairness than both platform thread pools
and async code. So are you concerned that virtual threads will improve things but not to the extent you wish there was something that could?</div>
<div><br class="">
</div>
<div>— Ron</div>
<br class="">
</body>
</html>