Fibers and stack traces

[Ron Pressler]

Hi Bela! I've been a fan of JGroups for many years, and I'm happy to see you're
interested in Loom.

You're asking a good question. Serviceability/observability is a top concern for
Loom, and we know we need to come up with a good story for thread dumps, but
haven't yet settled on an approach. Here are two ideas we've considered so far:

1. Have the creator of a virtual thread opt-in to mark a thread as "managed"
(all platform threads would be automatically managed), which would make those
particular ones, but no others, appear in thread dumps and perhaps other
observability mechanisms, like platform MXBeans.

2. Take advantage of structured concurrency, and have those threads created in a
structured scope (via some ExecutorService), but no others, displayed in thread 
dumps in a tree that captures their scope. Such presentation would hopefully make 
it easier to make sense of a great number of threads.

Personally, I prefer the second idea, but neither has been prototyped yet.

— Ron




On 4 July 2020 at 08:20:25, Bela Ban (belaban at mailbox.org(mailto:belaban at mailbox.org)) wrote:

> Hi everyone (from a just-joined member)
>  
> quick introduction: Bela Ban, JGroups lead (jgroups.org), I work for Red
> Hat / IBM. My interest in Loom is to see how/whether fibers affect
> performance, e.g. for blocking RPCs across a cluster. Also, the wet
> dream of continuing down the road of blocking programming without the
> callback hell of async / reactive programming (but still reaping the
> benefits of fewer ctx switches) looks enticing!
>  
> Testing performance of fibers against JGroups looks good, besides the
> occasional JVM (16-ea29) crash :-)
>  
> The question I have is about stack traces, here's an email excerpt I
> sent to Alan earlier this week (I did read through some of the archives,
> but might have overseen this):
>  
> A big issue, however, is that stack dumps don't show anything
> meaningful. If a fiber is in BLOCKED or (TIME_)WAITING state, then its
> stack trace isn't visible.
>  
> If I keep track of all fibers I create, then I can provide a thread dump
> of them myself: see the attached (deadlocking) program.
>  
> But most customers will want to produce this via
> jstack/CTRL-kill/whatever, so the above is not feasible.
>  
> What have you guys planned to tackle this? I know I should probably be
> asking this on the ML...
>  
> I understand that a stack trace of millions of threads is cumbersome, but
> * I assume most apps still use only a few or a few tens of threads, so
> this would still be useful
> * Filtering could be used, e.g. only BLOCKED threads, or threads
> starting with "Invoker*"
>  
> Thoughts?
>  
>  
> --
> Bela Ban | http://www.jgroups.org
>