Jetty and Loom

Tue Jan 5 12:44:13 UTC 2021

Oh, and one more thing about over-promising vs. over-delivering.

There might be downsides to over-promising, but my personal goal, at this
point in time, isn’t to be proven right, but to be proven wrong, as that 
might expose actual problems early, problems that we can fix and help us
deliver something better. My assumption is that the number of Java developers
exposed to those “promises” is relatively low, and so the gain of you finding
potential real issues is greater than the downside.

But I think there might be some misunderstanding as what it is that we promise.
We do *not* promise that virtual threads somehow create more computational
resources than available. We do “promise” (or hope to) that virtual threads 
provide a superior way to handle the resources that you have. If you can prove
*this* to be wrong, then that would be terrific. It will allow us to fix those
problems, or, at worst, make our advice more precise. Like the implementation,
that, too, is a work-in-progress.

Finally, the team working on OpenJDK in isn’t a hivemind. It is perfectly fine 
for different people to have different communication styles. When the work is 
done, we will converge on a single message in the “official” documentation.
It is also fine for you to disagree with us on what the best messaging is.
I, for one, think it is counterproductive when explaining virtual threads to
*focus* on the fact that CPU cores are fixed just as it is counterproductive
to focus on the rare need for object pooling when explaining what a GC is and
how to work with it *in the general case* to people who are only familiar
with manual memory management. I think it is first important to explain the
general principle; otherwise you’re inviting premature optimisation that 
takes effort, and in the vast majority of cases isn’t necessary, even if 
some people might encounter some issues in some uncommon situations.

— Ron

On 5 January 2021 at 12:19:13, Ron Pressler (ron.pressler at oracle.com (mailto:ron.pressler at oracle.com)) wrote:

> That the number of threads is limited by RAM is true though misses the point.
> GC and manual memory management are *both* limited by RAM, but the difference
> is that with a GC you *generally* “allocate without thinking about management”
> *up to that shared limit*.
>
> That virtual threads are “not suitable for CPU bound tasks” depends on the
> speaker’s assumption that the listener knows what is meant by “not suitable,”
> namely, that virtual threads do not *help* perform more CPU-bound tasks than
> a thread pool, so, if that’s the only thing your tasks do, you’ll see no
> benefit from virtual threads. If the listener might think it means that virtual
> threads somehow *harm* the execution of CPU bound tasks, then it’s misleading.
>
> The “additional load on GC” statement is not, I believe, demonstrated. If it
> turns out to be true, then it’s a problem that should be fixed. It takes the same
> amount of work for the GC to processes references on platform thread stacks than
> on virtual thread stacks. It shouldn’t even be true for tasks on thread pools.
> The data used by the tasks must be stored somewhere, and the GC doesn’t care
> if it’s stored in a virtual thread stack or any other object. Having said that,
> the interaction of virtual threads with the GC is still work-in-progress and
> can be improved, but it is not a priori true that virtual threads place more
> load on the GC than a thread pool.
>
> > ThreadLocals only work as a resource cache because threads are pooled. Yes there
> > are alternatives, but it is still a driver that may push some users to pools
> > rather than virtual threads.
>
> Habit is certainly a driver against changing behaviour but it’s not a good
> reason not to advise people to consider changing their behaviour to gain
> more benefits.
>
> > If the amount of concurrency required by the application is only 10s, 100s or
> > 1000s then it is probably not worth dealing with the different limitations of
> > virtual threads. Maybe not true in future as you address some of these limitations,
> > but it is true today.
>
> It is tautologically true that if your use case does not benefit from virtual
> threads then it does not benefit from virtual threads. I don’t think you’ve
> demonstrated that it will be *harmed* by virtual threads either in terms of
> effort or in terms of runtime characteristics, certainly not in the general
> case of “I just need a few thousand threads.” Quite the opposite. Could there
> be such cases? Maybe. Are they common? I don’t think so, but if they are —
> show them.
>
> > Totally confused by the messaging from this project.
>
> I’m confused by what you find confusing.
>
> But I’m trying to exploit your desire to “prove our claims wrong” to actually find
> situations where our claims/advice/assumption are actually wrong. :)
>
>
> — Ron
>
>
> On 5 January 2021 at 11:39:42, Greg Wilkins (gregw at webtide.com (mailto:gregw at webtide.com)) wrote:
>
> >
> > Ron & Alan,
> >
> > I really think we are talking past each other.
> >
> > I keep describing these limitations and reasons. I've written examples to demonstrate them, yet your response is still:
> > > What limitations?
> > and
> > > What reasons?
> >
> > So I repeat the limitations and reasons again, only for Alan to say:
> > > User mode threads aren't intended for compute bound tasks
> >
> >
> > and
> > > A lot of challenges and unknowns to work through
> >
> > I keep being bounced between that I'm speaking untruths vs that I'm saying the bleeding obvious?!?!? Which is it?
> >
> >
> > The limitations I know of include:
> > Not suitable for CPU bound tasks
> > Can't yet use synchronize/wait/notify
> > Number is limited by available heap memory
> > Additional load on GC
> > Additional CPU usage is evident in testing so far
> > ThreadLocals can be used for resource caching (not necessarily the only way to cache resources, but a common way that it is done, so it is still a limitation even if there are alternatives).
> > An additional scheduler is involved, which may need tuning and/or consume extra resources (and yes it may be super smart and make things faster)
> >
> >
> > Some of these limitations are some of the reasons (other than start time) you might want to use pooled kernel threads:
> > ThreadLocals only work as a resource cache because threads are pooled. Yes there are alternatives, but it is still a driver that may push some users to pools rather than virtual threads.
> > It may be that an app that is already heavy on heap and GC usage works better with preallocated kernel stacks rather than dynamic heap stacks
> > CPU bound tasks
> >
> > But ultimately the fundamental reason to not use virtual threads if frequently going to be just the numbers! If the amount of concurrency required by the application is only 10s, 100s or 1000s then it is probably not worth dealing with the different limitations of virtual threads. Maybe not true in future as you address some of these limitations, but it is true today.
> >
> > I just don't understand why this is so controversial to say as there is huge scope for virtual threads as everything has some limitations and few things are universally applicable. I really am trying to focus on what virtual threads are good for, yet every time I mention a limitation I get a strong reaction saying that there are no limitation, and we are back talking about what they are not good for....
> >
> > Totally confused by the messaging from this project.
> >
> >
> > PS. I'm not bothered by GC vs allocation. They are different techniques that have their pros and cons. It is really good that GC has mostly removed the need for java developers to worry about allocation of resources.... exception that is not 100% and most java apps still have pools of manually managed resources (buffer, deflaters, connections, threads etc) . They are different and each has limitations that need to be acknowledged.
> >
> > PPS. The reason I've pointed out that virtual threads still need memory (ie like saying GC Strings still use memory) is that message is not apparent in the "just spawn" publicity around Loom. It may be obvious to you that "Cheap threads can do expensive things", but don't think that is obvious to everybody.
> >
> > --
> > Greg Wilkins CTO http://webtide.com (https://urldefense.com/v3/__http://webtide.com__;!!GqivPVa7Brio!IcSCP0QTw_LnPwSRqMb467sO1aeJcD5scWbQq2mo3sYiO5pOMsMKt2bxdVvjwQ_jbA$)
>