jstack, profilers and other tools

[Rob Bygrave]

*> An existing application that migrates to using virtual threads doesn’t
replace its platform threads with virtual threads*

I'd be keen to clarify this around the use of virtual threads for http
servers. The Helidon team has stated that they are working on a new loom
based http server. It will be interesting to see how that works compared to
say Jetty with a loom based thread pool and I hope they release that soon
and that provides us with a view of how http servers might work best with
virtual threads.

What I have been confident about to date based on the testing I've done is
that we can use Jetty with a Loom based thread pool and that has worked
very well. That is replacing current platform threads with virtual threads.
I'm suggesting this will frequently be sub 1000 virtual threads.  Ron, are
you suggesting this isn't a valid use of virtual threads or am I reading
too much into what you've said here?



> *unusual* for an application that has any virtual threads to have fewer
than, say, 10,000

In the case of http server use of virtual thread, I feel the use of
*unusual* is too strong. That is, when we are using virtual threads for
application code handling of http request/response (like Jetty + Loom), I
suspect this is frequently going to operate with less than 1000 concurrent
requests per server instance.

For example, by default a Jetty instance today would have a platform thread
based thread pool with default max 200 threads. There is queuing that goes
on for higher amounts of concurrent load, and yes it is easy to change that
pool size but my gut and experience to date says that a big proportion of
http server instances out there are operating at less than 200 concurrent
requests per server instance. As such, operating jetty + loom at sub 200
requests per second is something I've investigated and tested in the past
and I didn't see a problem here  (tldr: loom performs fine with sub 200
concurrent virtual threads).



Ron,
Is there something that means we should not be using Jetty with Loom
threads? (I'm suggesting this use case is frequently going to be sub 1000
virtual threads)
Are you familiar with what the Helidon team is building wrt a loom based
http server? (Does that only target 10K concurrent requests??)



Thanks, Rob.


On Tue, 12 Jul 2022 at 06:58, Ron Pressler <ron.pressler at oracle.com> wrote:

> Hi.
>
> Alan gave you the specifics, but I’d like to make a more general point.
>
> It’s important to remember that the reason to use virtual threads is to
> have lots of them. It is, therefore, unusual for an application that has
> any virtual threads to have fewer than, say, 10,000. An existing
> application that migrates to using virtual threads doesn’t replace its
> platform threads with virtual threads, but spawns a virtual thread for each
> of its *tasks*. As a consequence, virtual threads are very numerous and
> most of them are very short lived, and the simple thread dump format used
> by jstack will not be informative. The set of mounted virtual threads is
> essentially a random sample of less than 1% of threads (8-30 threads out of
> at least 10K), and probably not useful to understanding what the
> application is doing. That is why a new kind of thread dump, designed for
> these circumstances, is introduced.
>
> — Ron
>
> > On 11 Jul 2022, at 18:37, Egor Ushakov <egor.ushakov at jetbrains.com>
> wrote:
> >
> > Hi all,
> >
> > I'm trying to prepare IntelliJ for loom and have some trouble
> understanding how tooling should be modified:
> > 1. with debugger it is more or less ok - virtual threads are separated
> from carrier threads, stack are separate and debugger is responsible for
> showing all of this.
> > 2. thread dumps (jstack as an example) - no virtual threads are shown,
> carrier threads stacks are truncated even if they are doing some work in
> the mounted virtual threads. It is not clear for me how users should
> understand what (even mounted) virtual thread are doing. Should we always
> switch to the new json format? Should user decide on which format to use?
> Previously thread dumps way an easy way to grab "what the app is doing" at
> the moment. Is there a way to achieve this now? Or this way should be
> abandoned?
> > 3. Profilers - with jfr I was not able to see any sampling data for
> virtual threads, where should I find it? With async-profiler (using
> AsyncGetCallTrace) it is still posible to see the sampling data (and merged
> stacks when vthread is mounted) - good. Hopefully this wont break?
> >
> > Could someone please clarify this?
> >
> > Thanks,
> > Egor
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20220712/a6b4820d/attachment.htm>