Cache topology aware scheduling

[Alan Bateman]

On 02/09/2024 07:23, Danny Thomas wrote:
> Hi folks,
>
> I was giving some thought to our adoption of Xen 4 coinciding with 
> virtual threads being available, and it occurred to me with an 
> increasing number of architectures clustering L3 and L2 caches between 
> groups of cores on a die, that virtual threads scheduling in user 
> space could make them particularly well suited to these architectures, 
> if the scheduler were topology aware.
>
> Have you given any thought to worker CPU affinity and/or locality to 
> an existing worker when a virtual thread is started by another? Would 
> you consider this something to be proved out by custom schedulers, or 
> is this enough of a trend to justify future investment in the default 
> scheduler?

To date, we've put CPU and node affinity into the "custom scheduler" 
topic, which is still TBD on whether to expose. If you have data from 
any experiments with the current EA builds then it would be useful to 
see. The current EA builds allow the the default FJP based scheduler to 
be replaced for experimentation purposes.

In a system with a mix of schedulers then starting a virtual thread will 
"inherit" the scheduler when not configured. That seems a sensible default.

-Alan