<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Sure, a system could provision more resources before they’re exhausted to maintain stability.
<div class=""><br class="">
</div>
<div class="">— Ron<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 14 Jul 2022, at 18:29, <a href="mailto:eric@kolotyluk.net" class="">
eric@kolotyluk.net</a> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">Thanks for that clarification, Ron…<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">Would it be fair to say a system is stable, in the case where when the Load Balancer predicts the current resources will soon become unstable, it launches more instances/resources before instability sets in? That is, appropriate feedback loops
can keep the underlying systems stable? Of course, keeping a system stable under load bursts could be challenging, but manageable.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">My past experience with Akka/Scala shows that we can provision systems such that we get better CPU utilization because such systems can distribute tasks more effectively over limited Platform Threads. I am hopeful that with Loom, Virtual Threads,
Structured Concurrency, etc. we also can provision systems with better resource utilization. For example, the load balance spawns new instances at 75% utilization rather than 50% under conventional systems.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">Also, my hope is that Loom will let us design and implement this with a lower cognitive load than using reactive programming techniques. Once Loom becomes more available, such as Java 19, it will be interesting to see what impact this has on
the Reactive Community, Scala, Akka, Kotlin and Kotlin Coroutines, etc.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">This whole Loom Project would make a fascinating University course as a lot of important historical lessons are apparent. There has been a lot of progress from Fibers to Virtual Threads…<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">I took two<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://www.coursera.org/__;!!ACWV5N9M2RV99hQ!K2Wz_OIAW0hHORIeequh1EH5i6b88e87fyd04pvMt_658W6zd9Y46_VIVD2bT2CG6jgwSdRalxuVhJEKOQ$" style="color: blue; text-decoration: underline;" class="">Coursera</a><span class="Apple-converted-space"> </span>courses,
one on Functional Programming in Scala, and the other on Reactive Programming in Scala. They were great courses. It would be fantastic for Coursera to have a course on Concurrent Programming in Loom.<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class="">Cheers, Eric<o:p class=""></o:p></span></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span class=""><o:p class=""> </o:p></span></div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""><span lang="EN-US" class="">From:</span></b><span lang="EN-US" class=""><span class="Apple-converted-space"> </span>Ron Pressler <<a href="mailto:ron.pressler@oracle.com" class="">ron.pressler@oracle.com</a>><span class="Apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>July 14, 2022 3:35 AM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Eric Kolotyluk <<a href="mailto:eric@kolotyluk.net" class="">eric@kolotyluk.net</a>><br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span>Alex Otenko <<a href="mailto:oleksandr.otenko@gmail.com" class="">oleksandr.otenko@gmail.com</a>>; Rob Bygrave <<a href="mailto:robin.bygrave@gmail.com" class="">robin.bygrave@gmail.com</a>>; Egor
Ushakov <<a href="mailto:egor.ushakov@jetbrains.com" class="">egor.ushakov@jetbrains.com</a>>;
<a href="mailto:loom-dev@openjdk.org" class="">loom-dev@openjdk.org</a><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [External] : Re: jstack, profilers and other tools<o:p class=""></o:p></span></div>
</div>
</div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
First, there is no such thing as more or less stable. Stability is binary. Either the rate at which requests are completed is equal to the rate at which they arrive (the system is stable), or it is lower (in which case requests pile up and the system is unstable).
Although, I guess you could talk about how quickly requests pile up and your server starts dropping them.<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Second, if your system is stable, Little’s law tells you how many requests are being concurrently served. Obviously, if you’re serving L concurrent requests in a stable system, then you have sufficient resources to serve them concurrently. Every request might
consume a little or a lot of some resources — CPU, memory, networking — and so those resources' availability imposes upper bounds on your concurrency. But (assuming you use threads as your units of concurrency) every concurrent request must consume at least
one thread, or it won’t be able to make progress at all. So threads are also an upper bound on concurrency, and we know empirically that in a great many server systems OS threads become the most constraining upper bound on concurrency well before other resources.
Virtual threads remove that particular limitation, which helps all those systems, and now the concurrency of your system is only limited by the other resources I mentioned.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
If every request consumes 1/10 of your available CPU over its entire duration, then your CPU puts a limit of 10 on your concurrency and threads are not your bottleneck, but if you’re using virtual threads — meaning you want a much higher number of threads —
then that’s not your circumstance. Clearly, when your CPU, or any other resource consumed by the requests you serve, is at 100% (for any non-instantaneous duration) then your system is not stable.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
— Ron<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
<br class="">
<o:p class=""></o:p></div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="" type="cite">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On 13 Jul 2022, at 19:26,<span class="Apple-converted-space"> </span><a href="mailto:eric@kolotyluk.net" style="color: blue; text-decoration: underline;" class="">eric@kolotyluk.net</a><span class="Apple-converted-space"> </span>wrote:<o:p class=""></o:p></div>
</div>
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Just testing my intuition here… because reading what Ron says is often eye-opening… and changes my intuition<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<ol start="1" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Loom improves concurrency via Virtual Threads<o:p class=""></o:p></li></ol>
<ol start="1" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="1" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
And consequently, potentially improves throughput<o:p class=""></o:p></li></ol>
</ol>
<ol start="2" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
A key aspect of concurrency is blocking, where blocked tasks enable resources to be applied to unblocked tasks (where Fork-Join is highly effective)<o:p class=""></o:p></li></ol>
<ol start="2" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="1" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Pre-Loom, resources such as Threads could be applied to unblocked tasks, but<o:p class=""></o:p></li></ol>
</ol>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>i.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>Platform
Threads are heavy, expensive, etc. such that the number of Platform Threads puts a bound on concurrency<o:p class=""></o:p></div>
</div>
<ol start="2" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="2" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Post-Loom, resources such as Virtual Threads can now be applied to unblocked tasks, such that<o:p class=""></o:p></li></ol>
</ol>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>i.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>Light,
cheap, etc. Virtual Threads enable a much higher bound on concurrency<o:p class=""></o:p></div>
</div>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>ii.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>According
to Little’s Law, throughput<span class="apple-converted-space"> </span><b class=""><i class="">can</i></b><span class="apple-converted-space"> </span>rise because the number of threads<span class="apple-converted-space"> </span><b class=""><i class="">can</i></b><span class="apple-converted-space"> </span>rise.<o:p class=""></o:p></div>
</div>
<ol start="3" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Little’s Law also says “The only requirements are that the system be stable and non-preemptive;”<o:p class=""></o:p></li></ol>
<ol start="3" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="1" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
While the underlying O/S may be preemptive, the JVM is not, so this requirement is met.<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
But, Ron says, “While it is true that the rate of arrival might rise without bound, if the number of threads is insufficient to meet it, then the system is no longer stable (normally that means that queues are growing without bound).”<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Which I take to imply, that increasing the number of Virtual Threads increases the stability… ?<o:p class=""></o:p></li></ol>
</ol>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>i.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>Even
in Loom, there is an upper bound on Virtual Threads created, albeit a much higher upper bound.<o:p class=""></o:p></div>
</div>
<ol start="4" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
Where I am still confused is<o:p class=""></o:p></li></ol>
<ol start="4" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="1" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
In Loom, I would expect that even when all our CPU Cores are at 100%, 100% throughput, the system is still stable?<o:p class=""></o:p></li></ol>
</ol>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>i.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>Or
maybe I am misinterpreting what Ron said?<o:p class=""></o:p></div>
</div>
<ol start="4" type="1" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<ol start="2" type="a" style="margin-bottom: 0cm; margin-top: 0cm;" class="">
<li class="MsoListParagraph" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">
However, latency will suffer, unless<o:p class=""></o:p></li></ol>
</ol>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>i.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>more
CPU Cores are added to the overall load, via some load balancer<o:p class=""></o:p></div>
</div>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>ii.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>flow
control, such as backpressure, is added such that queues do not grow without bound (a topic I would love to explore more)<o:p class=""></o:p></div>
</div>
<div style="margin-left: 108pt;" class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -108pt;" class="">
<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>iii.<span style="font-size: 7pt; font-family: "Times New Roman", serif;" class=""> <span class="apple-converted-space"> </span></span>Or,
does an increase in latency mean a loss of stability?<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Cheers, Eric<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""><span lang="EN-US" class="">From:</span></b><span class="apple-converted-space"><span lang="EN-US" class=""> </span></span><span lang="EN-US" class="">loom-dev <<a href="mailto:loom-dev-retn@openjdk.org" style="color: blue; text-decoration: underline;" class="">loom-dev-retn@openjdk.org</a>><span class="apple-converted-space"> </span><b class="">On
Behalf Of<span class="apple-converted-space"> </span></b>Ron Pressler<br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>July 13, 2022 6:30 AM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>Alex Otenko <<a href="mailto:oleksandr.otenko@gmail.com" style="color: blue; text-decoration: underline;" class="">oleksandr.otenko@gmail.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Rob Bygrave <<a href="mailto:robin.bygrave@gmail.com" style="color: blue; text-decoration: underline;" class="">robin.bygrave@gmail.com</a>>; Egor Ushakov <<a href="mailto:egor.ushakov@jetbrains.com" style="color: blue; text-decoration: underline;" class="">egor.ushakov@jetbrains.com</a>>;<span class="Apple-converted-space"> </span><a href="mailto:loom-dev@openjdk.org" style="color: blue; text-decoration: underline;" class="">loom-dev@openjdk.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>Re: [External] : Re: jstack, profilers and other tools</span><o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The application of Little’s law is 100% correct. Little’s law tells us that the number of threads must *necessarily* rise if throughput is to be high. Whether or not that alone is *sufficient* might depend on the concurrency level of other resources as well.
The number of threads is not the only quantity that limits the L in the formula, but L cannot be higher than the number of threads. Obviously, if the system’s level of concurrency is bounded at a very low level — say, 10 — then having more than 10 threads
is unhelpful, but as we’re talking about a program that uses virtual threads, we know that is not the case.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Also, Little’s law describes *stable* systems; i.e. it says that *if* the system is stable, then a certain relationship must hold. While it is true that the rate of arrival might rise without bound, if the number of threads is insufficient to meet it, then
the system is no longer stable (normally that means that queues are growing without bound).<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
— Ron<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
<br class="">
<br class="">
<o:p class=""></o:p></div>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="" type="cite">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On 13 Jul 2022, at 14:00, Alex Otenko <<a href="mailto:oleksandr.otenko@gmail.com" style="color: blue; text-decoration: underline;" class="">oleksandr.otenko@gmail.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This is an incorrect application of Little's Law. The law only posits that there is a connection between quantities. It doesn't specify which variables depend on which. In particular, throughput is not a free variable.<span class="apple-converted-space"> </span><o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Throughput is something outside your control. 100k users open their laptops at 9am and login within 1 second - that's it, you have throughput of 100k ops/sec.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Then based on response time the system is able to deliver, you can tell what concurrency makes sense here. Adding threads is not going to change anything - certainly not if threads are not the bottleneck resource. Threads become the bottleneck when you have
hardware to run them, but not the threads.<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Tue, 12 Jul 2022, 15:47 Ron Pressler, <<a href="mailto:ron.pressler@oracle.com" style="color: blue; text-decoration: underline;" class="">ron.pressler@oracle.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0cm 0cm 0cm 6pt; margin: 5pt 0cm 5pt 4.8pt;" class="" type="cite">
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
<br class="">
<br class="">
<o:p class=""></o:p></div>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="" type="cite">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On 11 Jul 2022, at 22:13, Rob Bygrave <<a href="mailto:robin.bygrave@gmail.com" target="_blank" style="color: blue; text-decoration: underline;" class="">robin.bygrave@gmail.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<i class="">> An existing application that migrates to using virtual threads doesn’t replace its platform threads with virtual threads</i><o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
What I have been confident about to date based on the testing I've done is that we can use Jetty with a Loom based thread pool and that has worked very well. That is replacing current platform threads with virtual threads. I'm suggesting this will frequently
be sub 1000 virtual threads. Ron, are you suggesting this isn't a valid use of virtual threads or am I reading too much into what you've said here?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The throughput advantage to virtual threads comes from one aspect — their *number* — as explained by Little’s law. A web server employing virtual thread would not replace a pool of N platform threads with a pool of N virtual threads, as that does not increase
the number of threads required to increase throughput. Rather, it replaces the pool of N virtual threads with an unpooled ExecutorService that spawns at least one new virtual thread for every HTTP serving task. Only that can increase the number of threads
sufficiently to improve throughput.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
<br class="">
<br class="">
<o:p class=""></o:p></div>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="" type="cite">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
><span class="apple-converted-space"> </span><b class=""><i class="">unusual</i></b><span class="apple-converted-space"> </span>for an application that has any virtual threads to have fewer than, say, 10,000<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
In the case of http server use of virtual thread, I feel the use of<span class="apple-converted-space"> </span><b class=""><i class="">unusual</i></b><span class="apple-converted-space"> </span>is too strong. That is, when we are using virtual threads for application
code handling of http request/response (like Jetty + Loom), I suspect this is frequently going to operate with less than 1000 concurrent requests per server instance. <span class="apple-converted-space"> </span><o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
</blockquote>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
1000 concurrent requests would likely translate to more than 10,000 virtual threads due to fanout (JEPs 425 and 428 cover this). In fact, even without fanout, every HTTP request might wish to spawn more than one thread, for example to have one thread for reading
and one for writing. The number 10,000, however, is just illustrative. Clearly, an application with virtual threads will have some large number of threads (significantly larger than applications with just platform threads), because the ability to have a large
number of threads is what virtual threads are for.<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The important point is that tooling needs to adapt to a high number of threads, which is why we’ve added a tool that’s designed to make sense of many threads, where jstack might not be very useful.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
— Ron</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>