Project Loom VirtualThreads hang

[Rob Bygrave]

FWIW, as I see it in the case of Nima http 1.1 server case with little to
no fanout (which in my experience to date is the majority case) there can
be relatively few (less than 200) relatively long lived virtual threads +
virtual threads used for fanout requests. A lot fewer VTs that gets touted
often here.

Nima Http 1.1 case:
1 platform thread listens for new connections - lives for server lifetime.
1 virtual thread per connection - lives for the lifetime of http connection
and handles all the requests/responses for that connection. This can be
relatively long lived based on keepalive & client lifetime.

If request/response processing requires NO fanout (no requests to other
services) but request handling is instead just persisting to and from
databases then that's it and Nima will have 1 VT per connection (and imo
for many folks this will commonly be low *hundreds* depending on your
server size - we are not all building services for twitter scale etc).

The next common case is limited fanout meaning that a single
request/response only incurs 1 or 2 fanout requests to other services (not
order of magnitude or anything like this). In this scenario for example, a
http 1.1 based service running with 200 concurrent connections is operating
with up to 600 virtual threads at any given time.

IMO I think it's going to be common for many "common folk services" to be
running with less than 600 virtual threads at any given time. My now
outdated testing using Jetty was designed to confirm that using *sub 200 VT
in total* was still preferable to traditional 200 platform threads - to see
if there was some inflexion point where we need to choose between VT and
PT.

Note that Nima might change to more a VT per request approach but that will
not change the number of virtual threads running at any given time.


Cheers, Rob.

On Wed, 4 Jan 2023 at 11:23, Ron Pressler <ron.pressler at oracle.com> wrote:

>
>
> On 3 Jan 2023, at 19:24, thurston N <thurston.nomagicsoftware at gmail.com>
> wrote:
>
> Hello Ron,
>
> Quoting:
>
> "Virtual threads replace *short *individual *tasks* in your application, not platform threads."
>
>  Why
>
> *short?*
>
>
>
> *Take a TCP server's accept loop.  I would have thought that's a natural target for execution in a virtual thread. *
>
>
> *And it's more or less intended to run forever.*
>
>
> A small number of virtual threads may run for a long time, but the vast
> majority would be short lived. They would naturally form a hierarchy: you’d
> have one or a couple of threads accepting connections that live forever,
> several thousand request-handling threads that live as long as a single
> request, and then tens of thousands or more each servicing a single
> outgoing service request in a fanout, and those would be even shorter-lived.
>
>
>
> *
> "They are best thought of as a business logic entity representing a task rather than an “execution resource.”"*
>
>
> *That's a pretty good definition of an "actor" (as in actors-model).  But there's no (even implicit) restriction on the duration of *
>
>
> *an actor's lifetime.*
>
> *Is it your intent to proscribe virtual threads as a possible implementation for actor-model type designs?*
>
>
> Threads are sequential task that is done concurrently with others. What
> protocols are used to communicate among threads (and the actor model is a
> communication protocol) is up to the programmer. An actor model is a bit
> too high-level for the JDK, but libraries may certainly offer actors based
> on virtual threads, just as others may build other high level constructs.
>
>
>
> *I'm thinking of simulations, e.g. modelling a migrating herd, bees at a honeycomb, a very busy traffic intersection, et al.*
>
>
> *It seems natural (even elegant) to represent each of those entities (deer, bee, car) as a virtual thread, and executing them as a *
>
>
> *platform thread isn't an option (because of their sheer number, the essence of the problem that virtual threads is the solution for)*
>
>
> *And I'm sure there are innumerable other circumstances that could benefit from LWP (as Erlang terms them)*
>
> *IMO, it's awfully reductive to restrict practical uses of virtual threads to writing sequential single request/response scenarios
>
> *
>
>
> My intent is not to restrict so much as to focus. Simulations are
> absolutely a great use-case for lightweight threads, but there are
> implementation choices that may need to balance some desirable properties
> with others and could result in virtual threads being a better fit for some
> use-cases than others. Because there are significantly more Java servers
> than Java simulations, and because servers are normally more complex (in
> terms of failure modes and architectural graph) and their developers are
> more mainstream and could benefit from more help, *if* there’s some
> prioritisation to be made, the server use-case is the one we prioritise
> over others by addressing it *first*. More people are interested in that
> use-case, that’s where virtual threads can contribute the most value, and
> that’s why it’s our *initial* focus.
>
> — Ron
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230104/5987bf27/attachment.htm>