New candidate JEP: 425: Virtual Threads (Preview)

[Ron Pressler]

Different people mean different things when they say “preemptive” — for some, it means not cooperative; for others it means time sharing. So I’ll avoid it.

Virtual threads are NOT cooperative, either in their API or implementation, regardless of whether the scheduler employs time sharing or not.

Cooperative multitasking — with syntactic coroutines (async/await) or in Windows 3.x programs — means that all possible scheduling points are statically known. They could require an explicit operation, like `await`, or be a encoded in a subroutine’s type and implicitly injected at the call-site, but either way, they are known. Note that this implies transitivity: if subroutine `bar` has a scheduling point in it and `foo` calls `bar`, then the call to `bar` in `foo` will be statically known as a scheduling point, and so will any call to `foo`. The code (at *all* layers), therefore, “cooperates” with the scheduler.

With virtual threads, not only are not all scheduling points known, by design *none* of them are known. Scheduling does not require an explicit operation, and the presence of a scheduling point is not reflected in a type or is statically knowable in advance in any way. Which methods might contain scheduling points could change at any time without impacting the assumptions made by their transitive callers, which is why they’re more composable and abstractable than any cooperative scheduling scheme.

As to time-sharing, whether it actually provides any useful “responsiveness” is unclear (although I’m not saying it doesn’t). When the OS needs to resort to relyng on time-sharing, which is often in extreme circumstances, the machine is not very responsive, and that’s when, say, 20 threads become CPU-bound. With virtual threads there’s the possibility that 2000 threads could become CPU-bound. Will time-sharing actually provide responsiveness?

— Ron

On 9 Apr 2022, at 15:22, Moataz Abdelnasser <moataz.nasser20 at gmail.com<mailto:moataz.nasser20 at gmail.com>> wrote:

If you have a long-running CPU-intensive task available from a request, you may have DDos attak problems whatever the kind of threads you use because in both case you will exhaust the pool of available threads to serve the requests.

Furthermore I’d argue, based on my own experiments, that virtual threads are safer against such attacks. Imagine a thread pool with 1000 platform threads trying to execute 1000 infinite loops. The server would become completely unresponsive and would probably require a hard reset. But the same experiment with 1000 virtual threads would spin a maximum of # of hardware threads. The server would be busy, but not unresponsive.

Good points. I realize virtual threads make throughput their primary
focus, not necessarily responsiveness.

This question was already asked on this list so i'm parroting
"Not in the first release, maybe in the future, will see."

Good to know that's at least considered a possibility :)
For fun, I just learned that Go apparently got a non-cooperative,
preemptive scheduler (only) about 10 years after its initial
appearance (https://go.googlesource.com/proposal/+/master/design/24543-non-cooperative-preemption.md).

Thanks & best regards,
Moataz


On Sat, Apr 9, 2022 at 1:11 PM Dr Heinz M. Kabutz
<heinz at javaspecialists.eu<mailto:heinz at javaspecialists.eu>> wrote:



On Sat, 09 Apr 2022 at 12:34, Remi Forax <forax at univ-mlv.fr<mailto:forax at univ-mlv.fr>> wrote:

----- Original Message -----
From: "Moataz Abdelnasser" <moataz.nasser20 at gmail.com<mailto:moataz.nasser20 at gmail.com>>
To: "loom-dev" <loom-dev at openjdk.java.net<mailto:loom-dev at openjdk.java.net>>
Sent: Saturday, April 9, 2022 7:24:45 AM
Subject: Re: New candidate JEP: 425: Virtual Threads (Preview)


Also, doesn't this make it somewhat dangerous to run a CPU-bound task
in an Executors.newThreadPerTaskExecutor()? I can imagine a DoS attack
where a malicious client makes a server do some long-running
CPU-intensive task in virtual threads enough times that all FJP
parallelism is exhausted. Unlike OS threads, control never goes back
to carriers, denying servicing other clients. If my reasoning is
correct, shouldn't this make it necessary to explicitly caution
against CPU-bound tasks in virtual threads? Perhaps by recommending a
`var result = myFJP.submit(() -> doSomeCpuWork()).get()`.

If you have a long-running CPU-intensive task available from a request, you may have DDos attak problems whatever the kind of threads you use because in both case you will exhaust the pool of available threads to serve the requests. And yes, a way to mitigate that issue is to use another pool of worker threads for those tasks and reject the request if this pool has no more threads.


Furthermore I’d argue, based on my own experiments, that virtual threads are safer against such attacks. Imagine a thread pool with 1000 platform threads trying to execute 1000 infinite loops. The server would become completely unresponsive and would probably require a hard reset. But the same experiment with 1000 virtual threads would spin a maximum of # of hardware threads. The server would be busy, but not unresponsive.


--
Dr Heinz M. Kabutz (PhD CompSci)
Author of "The Java(tm) Specialists' Newsletter"
Sun/Oracle Java Champion
JavaOne Rockstar Speaker
http://www.javaspecialists.eu
Tel: +30 69 75 595 262
Skype: kabutz