Jetty and Loom

[Ron Pressler]

That the number of threads is limited by RAM is true though misses the point.
GC and manual memory management are *both* limited by RAM, but the difference
is that with a GC you *generally* “allocate without thinking about management” 
*up to that shared limit*.

That virtual threads are “not suitable for CPU bound tasks” depends on the 
speaker’s assumption that the listener knows what is meant by “not suitable,”
namely, that virtual threads do not *help* perform more CPU-bound tasks than
a thread pool, so, if that’s the only thing your tasks do, you’ll see no 
benefit from virtual threads. If the listener might think it means that virtual
threads somehow *harm* the execution of CPU bound tasks, then it’s misleading.

The “additional load on GC” statement is not, I believe, demonstrated. If it 
turns out to be true, then it’s a problem that should be fixed. It takes the same
amount of work for the GC to processes references on platform thread stacks than
on virtual thread stacks. It shouldn’t even be true for tasks on thread pools. 
The data used by the tasks must be stored somewhere, and the GC doesn’t care
if it’s stored in a virtual thread stack or any other object. Having said that,
the interaction of virtual threads with the GC is still work-in-progress and
can be improved, but it is not a priori true that virtual threads place more
load on the GC than a thread pool.

> ThreadLocals only work as a resource cache because threads are pooled. Yes there 
> are alternatives, but it is still a driver that may push some users to pools 
> rather than virtual threads.

Habit is certainly a driver against changing behaviour but it’s not a good 
reason not to advise people to consider changing their behaviour to gain
more benefits.

> If the amount of concurrency required by the application is only 10s, 100s or 
> 1000s then it is probably not worth dealing with the different limitations of 
> virtual threads. Maybe not true in future as you address some of these limitations, 
> but it is true today.

It is tautologically true that if your use case does not benefit from virtual 
threads then it does not benefit from virtual threads. I don’t think you’ve 
demonstrated that it will be *harmed* by virtual threads either in terms of 
effort or in terms of runtime characteristics, certainly not in the general
case of “I just need a few thousand threads.” Quite the opposite. Could there 
be such cases? Maybe. Are they common? I don’t think so, but if they are — 
show them.

> Totally confused by the messaging from this project.

I’m confused by what you find confusing.

But I’m trying to exploit your desire to “prove our claims wrong” to actually find
situations where our claims/advice/assumption are actually wrong. :)


— Ron


On 5 January 2021 at 11:39:42, Greg Wilkins (gregw at webtide.com (mailto:gregw at webtide.com)) wrote:

>
> Ron & Alan,
>
> I really think we are talking past each other.
>
> I keep describing these limitations and reasons. I've written examples to demonstrate them, yet your response is still:
> > What limitations?
> and
> > What reasons?
>
> So I repeat the limitations and reasons again, only for Alan to say:
> > User mode threads aren't intended for compute bound tasks
>
>
> and
> > A lot of challenges and unknowns to work through
>
> I keep being bounced between that I'm speaking untruths vs that I'm saying the bleeding obvious?!?!? Which is it?
>
>
> The limitations I know of include:
> Not suitable for CPU bound tasks
> Can't yet use synchronize/wait/notify
> Number is limited by available heap memory
> Additional load on GC
> Additional CPU usage is evident in testing so far
> ThreadLocals can be used for resource caching (not necessarily the only way to cache resources, but a common way that it is done, so it is still a limitation even if there are alternatives).
> An additional scheduler is involved, which may need tuning and/or consume extra resources (and yes it may be super smart and make things faster)
>
>
> Some of these limitations are some of the reasons (other than start time) you might want to use pooled kernel threads:
> ThreadLocals only work as a resource cache because threads are pooled. Yes there are alternatives, but it is still a driver that may push some users to pools rather than virtual threads.
> It may be that an app that is already heavy on heap and GC usage works better with preallocated kernel stacks rather than dynamic heap stacks
> CPU bound tasks
>
> But ultimately the fundamental reason to not use virtual threads if frequently going to be just the numbers! If the amount of concurrency required by the application is only 10s, 100s or 1000s then it is probably not worth dealing with the different limitations of virtual threads. Maybe not true in future as you address some of these limitations, but it is true today.
>
> I just don't understand why this is so controversial to say as there is huge scope for virtual threads as everything has some limitations and few things are universally applicable. I really am trying to focus on what virtual threads are good for, yet every time I mention a limitation I get a strong reaction saying that there are no limitation, and we are back talking about what they are not good for....
>
> Totally confused by the messaging from this project.
>
>
> PS. I'm not bothered by GC vs allocation. They are different techniques that have their pros and cons. It is really good that GC has mostly removed the need for java developers to worry about allocation of resources.... exception that is not 100% and most java apps still have pools of manually managed resources (buffer, deflaters, connections, threads etc) . They are different and each has limitations that need to be acknowledged.
>
> PPS. The reason I've pointed out that virtual threads still need memory (ie like saying GC Strings still use memory) is that message is not apparent in the "just spawn" publicity around Loom. It may be obvious to you that "Cheap threads can do expensive things", but don't think that is obvious to everybody.
>
> --
> Greg Wilkins CTO http://webtide.com (https://urldefense.com/v3/__http://webtide.com__;!!GqivPVa7Brio!IcSCP0QTw_LnPwSRqMb467sO1aeJcD5scWbQq2mo3sYiO5pOMsMKt2bxdVvjwQ_jbA$)