<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">The main difference between virtual threads and platform threads is that you can have many more virtual threads. Their throughput impact is a result of their number, which allows a server to scale to higher throughputs, and fewer servers to serve
the same workload. An application employing 200 virtual threads will not perform significantly better than one employing 200 platform threads (although it might have some convenience advantages, such as not sharing threads and not risking ThreadLocal leaks),
and the number of servers required cannot drop from one to zero. Obviously, servers that never reach the workload that virtual threads exist to support will not benefit from the scalability they never use.</div>
<div class=""><br class="">
</div>
<div class="">— Ron</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 4 Jan 2023, at 01:22, Rob Bygrave <<a href="mailto:robin.bygrave@gmail.com" class="">robin.bygrave@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">FWIW, as I see it in the case of Nima http 1.1 server case with little to no fanout (which in my experience to date is the majority case) there can be relatively few (less than 200) relatively long lived virtual threads + virtual threads used
for fanout requests. A lot fewer VTs that gets touted often here.<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Nima Http 1.1 case:<br class="">
</div>
<div class="">1 platform thread listens for new connections - lives for server lifetime.</div>
<div class="">1 virtual thread per connection - lives for the lifetime of http connection and handles all the requests/responses for that connection. This can be relatively long lived based on keepalive & client lifetime.</div>
<div class=""><br class="">
</div>
<div class="">If request/response processing requires NO fanout (no requests to other services) but request handling is instead just persisting to and from databases then that's it and Nima will have 1 VT per connection (and imo for many folks this will commonly
be low <i class="">hundreds</i> depending on your server size - we are not all building services for twitter scale etc).
<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">The next common case is limited fanout meaning that a single request/response only incurs 1 or 2 fanout requests to other services (not order of magnitude or anything like this). In this scenario for example, a http 1.1 based service running with
200 concurrent connections is operating with up to 600 virtual threads at any given time.</div>
<div class=""><br class="">
</div>
<div class="">IMO I think it's going to be common for many "common folk services" to be running with less than 600 virtual threads at any given time. My now outdated testing using Jetty was designed to confirm that using
<i class=""><b class="">sub 200 VT </b><b class="">in total</b></i> was still preferable to traditional 200 platform threads - to see if there was some inflexion point where we need to choose between VT and PT.
<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Note that Nima might change to more a VT per request approach but that will not change the number of virtual threads running at any given time.<br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Cheers, Rob.<br class="">
</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 4 Jan 2023 at 11:23, Ron Pressler <<a href="mailto:ron.pressler@oracle.com" class="">ron.pressler@oracle.com</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;" class=""><br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On 3 Jan 2023, at 19:24, thurston N <<a href="mailto:thurston.nomagicsoftware@gmail.com" target="_blank" class="">thurston.nomagicsoftware@gmail.com</a>> wrote:</div>
<br class="">
<div class="">
<div dir="ltr" class="">
<div class="">Hello Ron,</div>
<div class=""><br class="">
</div>
<div class="">Quoting:<br class="">
<pre class="">"Virtual threads replace <b class="">short </b>individual *tasks* in your application, not platform threads."<br class=""><br class=""></pre>
<pre class=""> Why <b class="">short?<br class=""><br class=""></b></pre>
<pre class=""><b class="">Take a TCP server's accept loop. I would have thought that's a natural target for execution in a virtual thread. <br class=""><br class=""></b></pre>
<pre class=""><b class="">And it's more or less intended to run forever.<br class=""></b></pre>
</div>
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
A small number of virtual threads may run for a long time, but the vast majority would be short lived. They would naturally form a hierarchy: you’d have one or a couple of threads accepting connections that live forever, several thousand request-handling threads
that live as long as a single request, and then tens of thousands or more each servicing a single outgoing service request in a fanout, and those would be even shorter-lived.<br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<pre class=""><b class="">
"They are best thought of as a business logic entity representing a task rather than an “execution resource.”"<br class=""><br class=""></b></pre>
<pre class=""><b class="">That's a pretty good definition of an "actor" (as in actors-model). But there's no (even implicit) restriction on the duration of <br class=""></b></pre>
<pre class=""><b class="">an actor's lifetime.<br class=""></b></pre>
<pre class=""><b class="">Is it your intent to proscribe virtual threads as a possible implementation for actor-model type designs?</b></pre>
</div>
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
Threads are sequential task that is done concurrently with others. What protocols are used to communicate among threads (and the actor model is a communication protocol) is up to the programmer. An actor model is a bit too high-level for the JDK, but libraries
may certainly offer actors based on virtual threads, just as others may build other high level constructs.<br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<pre class=""><b class=""> <br class=""></b></pre>
<pre class=""><b class="">I'm thinking of simulations, e.g. modelling a migrating herd, bees at a honeycomb, a very busy traffic intersection, et al.<br class=""></b></pre>
<pre class=""><b class="">It seems natural (even elegant) to represent each of those entities (deer, bee, car) as a virtual thread, and executing them as a <br class=""></b></pre>
<pre class=""><b class="">platform thread isn't an option (because of their sheer number, the essence of the problem that virtual threads is the solution for)<br class=""></b></pre>
<pre class=""><b class="">And I'm sure there are innumerable other circumstances that could benefit from LWP (as Erlang terms them)<br class=""></b></pre>
<pre class=""><b class="">IMO, it's awfully reductive to restrict practical uses of virtual threads to writing sequential single request/response scenarios
</b></pre>
</div>
</div>
</div>
</blockquote>
<br class="">
</div>
<div class="">My intent is not to restrict so much as to focus. Simulations are absolutely a great use-case for lightweight threads, but there are implementation choices that may need to balance some desirable properties with others and could result in virtual
threads being a better fit for some use-cases than others. Because there are significantly more Java servers than Java simulations, and because servers are normally more complex (in terms of failure modes and architectural graph) and their developers are more
mainstream and could benefit from more help, *if* there’s some prioritisation to be made, the server use-case is the one we prioritise over others by addressing it *first*. More people are interested in that use-case, that’s where virtual threads can contribute
the most value, and that’s why it’s our *initial* focus.</div>
<div class=""><br class="">
</div>
<div class="">— Ron</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>