: Re: Project Loom VirtualThreads hang

[Ron Pressler]

On 6 Jan 2023, at 09:42, Arnaud Masson <arnaud.masson at fr.ibm.com<mailto:arnaud.masson at fr.ibm.com>> wrote:

A long CPU-bound request (without any IO/yield) on a vthread would virtually “pin” its native thread, which is a thing we don’t generally want, no?
(A server can have both CPU-bound and IO-bound requests at the same time.)

That would be a bit like blocking the even loop on NodeJS: https://nodejs.org/en/docs/guides/dont-block-the-event-loop/#blocking-the-event-loop-json-dos(mitigated by the fact there are more than 1 native thread to run vthreads in general).

Wouldn’t vthread preemptive scheduling fix that?
Otherwise, I guess we can insert manually Thread.yield in the slow CPU code but it’s not ideal:

For (int i=0; i<99999999; i++) {
  Thread.yield();
   // CPU-only stuff, or maybe old sync file IO
}

Arnaud


First, virtual thread scheduling is already preemptive — the JDK decides when to preempt a virtual thread without any explicit cooperation from user code. What you’re talking about is making the decision to preempt based on some expired time-slice on the CPU. This is called time-sharing.

Second, no, this isn’t like Node.JS because the scheduler uses all cores. To saturate the scheduler you’d need to hog the CPU on *all* cores at the same time, i.e. reach 100% CPU utilisation. It is true that the kernel scheduler will employ time-sharing at 100% CPU while the virtual threads scheduler currently doesn’t, but servers don’t normally run at 100% CPU, and when they do I don’t think people are happy with how well they behave under that condition. I.e. the time-sharing offered by the OS is definitely useful for some things, but making servers work well at 100% CPU doesn’t appear to be one of them — as far as we know.

Time-sharing does help when you have a small number of low-priority background processing jobs running on the server, but unlike Erlang or Go, Java easily offers that without adding time-sharing to the virtual thread scheduler.

So the question is, can time-sharing for virtual threads actually improve real workloads, and, if so, what should be the scheduling algorithm that would achieve that? To know the answer to this question, we’ll need to see real workloads that could potentially be helped by time-sharing. I haven’t seen one yet, but if anyone finds any, we’d certainly take a close look at that.

As for file IO, we’re currently working on employing io_uring (where available) to make that properly block the virtual thread rather than the OS thread. By the way, the current implementation will actually temporarily increase the number of OS threads used by the scheduler when doing long filesystem operations.

— Ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230106/d888e8f0/attachment.htm>