Native methods and virtual threads

[Cay Horstmann]

I just led a discussion about virtual threads at JCrete, and I was astonished that I got blank stares when I said the following: "Virtual threads have one big advantage: In the code that they execute, blocking is cheap and natural, thus freeing whoever writes that code from the tyranny of nested callbacks. This is the principal reason to use virtual threads."

But that really is the principal reason.

If you implement a framework where other programmers write code for the tasks that you dispatch, you will hear from them soon enough. Once they find out that they no longer have to use a reactive style, they will want virtual threads.

I don't think the server needs to change. It will continue to wait on idle sockets. But the tasks that it dispatches will likely be virtual threads by default, or platform threads for the situations where virtual threads don't work. That is more complex than it was before (unless you can be sure that all tasks can use virtual threads). But the programmers who implement the business logic will thank you.

On 14/07/2023 23.50, Brian S O'Neill wrote:
> The solution you describe (a single thread pool for blocking native tasks) is the same as the message passing solution that I proposed. But if I have to manage different thread pools for different type of operations, it's no different than using platform threads.
> 
> In a world without virtual threads, my server waits on idle sockets by using epoll or a SelectableChannel. Tasks can then be dispatched to a thread pool. In a world _with_ virtual threads, I don't need to directly use epoll or a SelectableChannel. But if I have to dispatch tasks to different thread pools, then my server isn't that much different than it was before. I'm dispatching to something else all the time, so what have I gained? I still have to manage thread pool sizes, etc.
> 
> If I'm using virtual threads to dispatch short lived tasks (via structured concurrency) I now need to be aware of all of the task dependencies. Sometimes I can use virtual threads, and sometimes I need to use a regular FJ pool. This is more complex than what I had before, when I could just use a single FJ pool and not worry about it.
> 
> Am I describing a false dichotomy? No. It's about choosing the right type of threads to use, and virtual threads have unfortunate gotchas. The safe choice is to design a server to use platform threads by default, and later add virtual threads as an option.
> 
> 
> On 2023-07-14 02:13 PM, Attila Kelemen wrote:
>>
>>     If I'm writing a new service, and I have complete control over what
>>     native libraries it depends on, I might conclude that virtual threads
>>     are a good choice. But as the service evolves, I might want to start
>>     depending on native libraries that provide needed functionality. If
>>     they
>>     perform blocking operations, I might need to wrap the library with a
>>     message passing queue, which just adds overhead and complexity. If the
>>     advice is to stop using virtual threads at this point, I could have
>>     made
>>     that decision from day one. And it seems that's the safest option.
>>
>>
>> But it is a false dichotomy to choose between not using virtual threads basically at all, and using only VTs. If that is a serious problem, then you could just create a single thread pool for your "slow, blocking native tasks" and then just wait for the submitted task on the VT. It is unlikely to make things very difficult, because you can easily hide this behind a method which is barely different from marking a method as "blocking, please compensate", and you even have more flexibility (which you might need, if you have such an edge case).

-- 

--

Cay S. Horstmann | http://horstmann.com | mailto:cay at horstmann.com