[External] : Re: jstack, profilers and other tools

[Alex Otenko]

This is an incorrect application of Little's Law. The law only posits that
there is a connection between quantities. It doesn't specify which
variables depend on which. In particular, throughput is not a free variable.

Throughput is something outside your control. 100k users open their laptops
at 9am and login within 1 second - that's it, you have throughput of 100k
ops/sec.

Then based on response time the system is able to deliver, you can tell
what concurrency makes sense here. Adding threads is not going to change
anything - certainly not if threads are not the bottleneck resource.
Threads become the bottleneck when you have hardware to run them, but not
the threads.

On Tue, 12 Jul 2022, 15:47 Ron Pressler, <ron.pressler at oracle.com> wrote:

>
>
> On 11 Jul 2022, at 22:13, Rob Bygrave <robin.bygrave at gmail.com> wrote:
>
> *> An existing application that migrates to using virtual threads doesn’t
> replace its platform threads with virtual threads*
>
> What I have been confident about to date based on the testing I've done is
> that we can use Jetty with a Loom based thread pool and that has worked
> very well. That is replacing current platform threads with virtual threads.
> I'm suggesting this will frequently be sub 1000 virtual threads.  Ron, are
> you suggesting this isn't a valid use of virtual threads or am I reading
> too much into what you've said here?
>
>
> The throughput advantage to virtual threads comes from one aspect — their
> *number* — as explained by Little’s law. A web server employing virtual
> thread would not replace a pool of N platform threads with a pool of N
> virtual threads, as that does not increase the number of threads required
> to increase throughput. Rather, it replaces the pool of N virtual threads
> with an unpooled ExecutorService that spawns at least one new virtual
> thread for every HTTP serving task. Only that can increase the number of
> threads sufficiently to improve throughput.
>
>
>
> > *unusual* for an application that has any virtual threads to have fewer
> than, say, 10,000
>
> In the case of http server use of virtual thread, I feel the use of
> *unusual* is too strong. That is, when we are using virtual threads for
> application code handling of http request/response (like Jetty + Loom), I
> suspect this is frequently going to operate with less than 1000 concurrent
> requests per server instance.
>
>
> 1000 concurrent requests would likely translate to more than 10,000
> virtual threads due to fanout (JEPs 425 and 428 cover this). In fact, even
> without fanout, every HTTP request might wish to spawn more than one
> thread, for example to have one thread for reading and one for writing. The
> number 10,000, however, is just illustrative. Clearly, an application with
> virtual threads will have some large number of threads (significantly
> larger than applications with just platform threads), because the ability
> to have a large number of threads is what virtual threads are for.
>
> The important point is that tooling needs to adapt to a high number of
> threads, which is why we’ve added a tool that’s designed to make sense of
> many threads, where jstack might not be very useful.
>
> — Ron
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20220713/8e1e8728/attachment-0001.htm>