Cache topology aware scheduling

[Dave Dice]

It’s possible some of the work done in Oracle Labs on NUMA-aware locks might be applicable in this context.  Briefly,  we reorder the list of threads waiting on a lock to prefer “near” handovers in the short-term, but still preserve long-term fairness.   Most of the keys ideas map cleanly over to schedulers.   The paper appeared in EuroSys 2019, but the link below is to the non-paywalled arXiv version.    Some of the things we needed to do — making all the operations constant-time, and dealing with concurrent arriving threads, lock-free list manipulation — could likely be relaxed in the context of user-level scheduling.

Regards
Dave



<https://arxiv.org/pdf/1810.05600>
[preview.png]
1810<https://arxiv.org/pdf/1810.05600>
PDF Document · 1.3 MB<https://arxiv.org/pdf/1810.05600>

On Sep 5, 2024, at 12:16 PM, Francesco Nigro <nigro.fra at gmail.com> wrote:

Hi @Danny Thomas<mailto:dannyt at netflix.com>

We're working (nudge nudge Andrew Haley) on a custom scheduler API - as mentioned by Alan, which enables (expert) users/framework devs to implement something like this - and more :)

Cheers,
Franz



Il giorno lun 2 set 2024 alle ore 10:41 Alan Bateman <alan.bateman at oracle.com<mailto:alan.bateman at oracle.com>> ha scritto:
On 02/09/2024 07:23, Danny Thomas wrote:
> Hi folks,
>
> I was giving some thought to our adoption of Xen 4 coinciding with
> virtual threads being available, and it occurred to me with an
> increasing number of architectures clustering L3 and L2 caches between
> groups of cores on a die, that virtual threads scheduling in user
> space could make them particularly well suited to these architectures,
> if the scheduler were topology aware.
>
> Have you given any thought to worker CPU affinity and/or locality to
> an existing worker when a virtual thread is started by another? Would
> you consider this something to be proved out by custom schedulers, or
> is this enough of a trend to justify future investment in the default
> scheduler?

To date, we've put CPU and node affinity into the "custom scheduler"
topic, which is still TBD on whether to expose. If you have data from
any experiments with the current EA builds then it would be useful to
see. The current EA builds allow the the default FJP based scheduler to
be replaced for experimentation purposes.

In a system with a mix of schedulers then starting a virtual thread will
"inherit" the scheduler when not configured. That seems a sensible default.

-Alan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20240905/433e484f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: preview.png
Type: image/png
Size: 412800 bytes
Desc: preview.png
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20240905/433e484f/preview-0001.png>