<div dir="ltr">whoops, that should have been<br><br>T' <= ~1000000<br>T <= max(T')<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 28, 2022 at 6:10 PM Daniel Avery <<a href="mailto:danielaveryj@gmail.com">danielaveryj@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">> Or, putting it in yet another way, you need support not for the average number of requests in the system, but for the maximum number of requests in the system.<br><br>That's a fair point, but I think the result (<span style="font-family:"Helvetica Neue";font-size:13px">T = λW <= ~1000000) still holds. </span>If we broke this step<div><br>T <= ~1000000<br><br>into multiple steps<br><br>T' <= ~1000000<br>T <= T'<br><br>Where<br>- T' is the instantaneous number of threads (or, requests)<div>- T is the average number of threads</div><div><br></div><div>Then the other steps still hold, but we've clarified that our system can never exceed ~1000000 instantaneous concurrent requests (or else it will run out of memory / crash / become "unstable" in a way that makes Little's Law inapplicable)</div></div><div><br></div><div>-Daniel</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 28, 2022 at 5:22 PM Alex Otenko <<a href="mailto:oleksandr.otenko@gmail.com" target="_blank">oleksandr.otenko@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Or, putting it in yet another way, you need support not for the average number of requests in the system, but for the maximum number of requests in the system.<div dir="auto"><br></div><div dir="auto">In a system with finite thread count this translates into support of arbitrarily long request queues (which don't depend on request rate, too). In a thread-per-request system it necessarily is arbitrarily large number of threads. (Of course, this is only a model of some ideal system)</div><div dir="auto"><br></div><div dir="auto">Thank you for bearing with me.</div><div dir="auto"><br></div><div dir="auto">Alex</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 29 Jul 2022, 00:11 Alex Otenko, <<a href="mailto:oleksandr.otenko@gmail.com" target="_blank">oleksandr.otenko@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Thanks.<div dir="auto"><br></div><div dir="auto">That works under _assumption_ that the time stays constant, which is also a statement made in the JEP.</div><div dir="auto"><br></div><div dir="auto">But we know that with thread count growing the time goes down. So one needs more reasoning to explain why T goes up - and at the same time keep the assumption that time doesn't go down.</div><div dir="auto"><br></div><div dir="auto">In addition, it makes sense to talk of thread counts when that presents a limit of some sort - eg if N threads are busy and one more request arrives, the request waits for a thread to become available - we have a system with N threads. Thread-per-request does not have that limit: for any number of threads already busy, if one more request arrives, it still gets a thread.</div><div dir="auto"><br></div><div dir="auto">My study concludes that thread-per-request is the case of infinite number of threads (from mathematical point of view). In this case talking about T and its dependency on request rate is meaningless.</div><div dir="auto"><br></div><div dir="auto">Poisson distribution of requests means that for any request rate there is a non-zero probability for any number of requests in the system - ergo, the number of threads. Connecting this number to a rate is meaningless.</div><div dir="auto"><br></div><div dir="auto">Basically, you need to support "a high number of threads" for any request rate, not just a very high request rate.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 28 Jul 2022, 23:35 Daniel Avery, <<a href="mailto:danielaveryj@gmail.com" rel="noreferrer" target="_blank">danielaveryj@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">





<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Maybe this is not helpful, but it is how I understood the JEP</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">This is Little’s Law:</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">L = λW</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Where</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">- L is the average number of requests being processed by a stationary system (aka concurrency)</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">- λ is the average arrival rate of requests (aka throughput)</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">- W is the average time to process a request (aka latency)</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">This is a thread-per-request system:</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T = L</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Where</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">- T is the average number of threads</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">- L is the average number of requests (same L as in Little’s Law)</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Therefore,</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T = L = λW</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T = λW</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Prior to loom, memory footprint of (platform) threads gives a bound on thread count:</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T <= ~1000</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">After loom, reduced memory footprint of (virtual) threads gives a relaxed bound on thread count:</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T <= ~1000000</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">Relating thread count to Little’s Law tells us that virtual threads can support a higher average arrival rate of requests (throughput), or a higher average time to process a request (latency), than platform threads could:</p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue";min-height:15px"><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">T = λW <= ~1000000</p></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>