[External] : Re: Project Loom VirtualThreads hang

[Rob Bygrave]

Well I think this discussion has gone well. Thanks and apologies in equal
measure I reckon :)

To be blunt, I'm really comfortable using VT in low numbers and for myself
I'm squarely in the camp "The default choice is to use VT". I'm in the camp
that says my server of choice going forward is very likely Nima *regardless
of load*. To be explicit, the implication is that I'm also in the camp that
is never willingly choosing to use reactive style so no WebFlux, no R2DBC
and use of Rx Libraries is going to be niche for me.

I might have misunderstood prior comments but frequently wrt VT I read
something that goes like "blah blah ... *but they are designed for cases
with 1000's of VT*" which to me was giving off the wrong vibe / implication
that VT are not designed to be used for cases where there are relatively
low numbers of VT.


*> if they work well at some high throughput X, then they obviously work
well at throughput 0.1X*

It wasn't obvious to me how the costs would play out in the 0.1X range
which was why I did the testing I did with Jetty and JDK HttpClient (where
it's easy to swap between VT and PT). It was this testing that made me
comfortable in the 0.1X range and the outcome for myself having the
expectation that "The default is VT".

To me it wasn't surprising that Nima provides *no option* to "swap in an ES
that uses Platform Threads" - it's an example of a library that is 100%
committed to Virtual Threads regardless of load and regardless of the
number of VT.


Cheers, Rob.

On Thu, 5 Jan 2023 at 14:11, thurston N <thurston.nomagicsoftware at gmail.com>
wrote:

> Yes, I agree.
> The "false" choice is between:
> load is less than some sweetspot N -> use reactive style design
> load is greater than N -> use sequential style with virtual threads
>
> in that sense I disagree with Rob's post (despite my previous post)- why
> would reactive style perform better than sequential style (whether using
> virtual or platform threads) at some small N?  surely it wouldn't
>
> Having written that, speaking for myself, I think the whole "just have to
> change the ExecutorService" is a bit of a fool's errand, and is n't
> important; e.g. because a fixed pool size ES has the side effect (and it is
> only a side effect) as a throttle on your application, where you need to do
> something different (though not difficult) if using a VT ES; I'm sure in
> the end there will be many other examples.
> I would prefer making VirtualThread and Continuation accessible (if
> read-only) to "minimizing surface API", if only for better
> observability/debugging at the user code level, but that's a different
> discussion
>
> The main point you make makes sense: there is no need to choose between
> sequential and reactive style design *because of efficiency/performance*
> reasons - touche
>
>
>
> On Wed, Jan 4, 2023 at 4:54 PM Ron Pressler <ron.pressler at oracle.com>
> wrote:
>
>>
>>
>> > On 5 Jan 2023, at 00:25, thurston N <thurston.nomagicsoftware at gmail.com>
>> wrote:
>> >
>> > " With virtual threads available, you can use that style no matter what
>> the load on your server is, and easily switch between thread implementation
>> if it makes a difference."
>> >
>> > Can you?
>> > even if it's a configuration option, you need to bounce a production
>> server(s) to "switch"
>> >
>> > My experience jibes with Robin's, especially since load is not constant
>> in the real world, as long as virtual threads offer acceptable throughput
>> and latency *even when their true benefits, viz high numbers are not in
>> effect".  If that is the case (and I don't see why it wouldn't be), the
>> question might better be "why would you ever bother to switch (from virtual
>> to platform)"?
>>
>> If the workload *could* rise to the point that it requires more threads,
>> then virtual threads already help: they support the possibly-needed scale,
>> and if they work well at some high throughput X, then they obviously work
>> well at throughput 0.1X. There’s obviously no need to switch.
>>
>> I’m talking about cases where the workload is such that it could never
>> rise to the point where virtual threads would be necessary, and the
>> appropriate thread implementation is picked during development, but
>> whatever it is, the thread-per-request style that the platform is designed
>> for would fit well. So you can always use the right style for Java, no
>> matter what your workload is, and then you can pick an appropriate thread
>> implementation for your application. In the vast majority of cases, the
>> choice in the low-throughput cases doesn’t matter much, but when it does,
>> it’s not an API choice, but a simple implementation choice. So in these
>> cases there’s also no need to switch.
>>
>> My main point is to try and steer the discussion away from cases where,
>> for whatever reason, the developer doesn’t wish to use the
>> thread-per-request model, because however well virtual threads perform such
>> tasks now, that is not their primary focus. Once the ecosystem gains more
>> experience with virtual threads for their primary usage, we can shift our
>> attention to other uses as well. There’s a lot of excitement about virtual
>> threads, and people are experimenting with them for various tasks — and
>> that’s great. But because we must choose what to focus on *first*, we
>> choose the more common use-cases where the performance impact is also much
>> higher (and if virtual threads help for other things too, that’s terrific,
>> but still not our primary focus).
>>
>> — Ron
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230105/03b0880e/attachment.htm>