Scheduler API Feedback

[Mengyang Li]

*Thank you for your reply.*

While testing the code, I found an issue when using the custom scheduler
feature.

When I delegate to the default scheduler, I noticed that the nested virtual
thread created inside another virtual thread does *not* actually “use” my
custom scheduler.
``` java

public static void main(String[] args) throws Throwable {

System.getProperties().setProperty("jdk.virtualThreadScheduler.implClass",
"io.github.dreamlike.CustomerVirtualThreadScheduler");
    CompletableFuture<Void> future = new CompletableFuture<>();
    Thread.ofVirtual()
            .name("parent vt")
            .start(() -> {
                Thread.ofVirtual().name("sub vt")
                        .start(() -> {
                            future.complete(null);
                        });
            });

    future.get();
}

// other file io/github/dreamlike/CustomerVirtualThreadScheduler.java
public class CustomerVirtualThreadScheduler implements
Thread.VirtualThreadScheduler  {

    private final Thread.VirtualThreadScheduler defaultScheduler;

    public CustomerVirtualThreadScheduler(Thread.VirtualThreadScheduler
defaultScheduler) {
        this.defaultScheduler = defaultScheduler;
    }

    @Override
    public void execute(Thread vthread, Runnable task) {
        Thread thread = Thread.currentThread();
        System.out.println(thread.getClass());
        System.out.println(vthread.toString() + " " + vthread.getState() +
" currentThread " + thread + " current is virtual " + thread.isVirtual());
        new RuntimeException().printStackTrace();
        defaultScheduler.execute(vthread, task);
    }
}
```

Console output:
``` bash
WARNING: Using custom default scheduler, this is an experimental feature!
class java.lang.Thread
VirtualThread[#27,VirtualThreadA]/runnable RUNNABLE currentThread
Thread[#3,main,5,main] current is virtual false
java.lang.RuntimeException
        at
io.github.dreamlike.CustomerVirtualThreadScheduler.execute(CustomerVirtualThreadScheduler.java:16)
        at
java.base/java.lang.VirtualThread.submitRunContinuation(VirtualThread.java:367)
        at
java.base/java.lang.VirtualThread.externalSubmitRunContinuationOrThrow(VirtualThread.java:456)
        at java.base/java.lang.VirtualThread.start(VirtualThread.java:734)
        at java.base/java.lang.VirtualThread.start(VirtualThread.java:745)
        at
java.base/java.lang.ThreadBuilders$VirtualThreadBuilder.start(ThreadBuilders.java:257)
        at io.github.dreamlike.VTMain.main(VTMain.java:14)
```

After checking the relevant code, it seems that execution goes through
the *first
branch* below.

This behavior doesn’t appear to align with the semantics of a custom
scheduler — the nested virtual thread is submitted directly to
ct.getPool() instead
of going through CustomerVirtualThreadScheduler::execute.
``` java

// VirtualThread.java
private void externalSubmitRunContinuationOrThrow() {
    if (scheduler == DEFAULT_SCHEDULER && currentCarrierThread() instanceof
CarrierThread ct) {
        try {

ct.getPool().externalSubmit(ForkJoinTask.adapt(runContinuation));
        } catch (RejectedExecutionException ree) {
            submitFailed(ree);
            throw ree;
        }
    } else {
        submitRunContinuation(false);
    }
}
```

Best regards,

Mengyang li

Alan Bateman <alan.bateman at oracle.com> 于2025年10月5日周日 00:35写道：

> On 04/10/2025 14:19, dreamlike_ocean lei wrote:
>
> Hello @loom-dev <loom-dev at openjdk.org>,
>
> I have been enjoying the new POLLER_PER_CARRIER design in the Loom repo
> and I really like the direction it is going.
> While building on top of the latest code, I noticed a couple of issues and
> would like to ask for clarification.
>
> Thanks for your mail. It's useful to hear from folks that are trying out
> the experimental support for custom schedulers in the loom repo as this
> will help to inform whether any of the directions prototyped should be
> taken further.
>
> BTW: I assume "dreamlike_ocean lei" isn't your real name. It would be
> helpful to use a real name or affiliation so we have some idea who you are.
>
>
>    1.
>
>    When calling Thread.startVirtualThread, the new virtual thread does
>    not inherit the scheduler of the calling virtual thread, but instead uses
>    DEFAULT_SCHEDULER. What is the reasoning behind this design? Could
>    there be a mechanism to allow implicit inheritance? This would be very
>    helpful for custom schedulers based on the per-core model.
>
> In a system with several virtual thread schedulers in use, which I think
> is what you mean, then it would be unpredictable and problematic to inherit
> the scheduler in many cases. If some library were, on first usage, start a
> virtual thread as a "background thread" that runs indefinitely then
> inheritance would mean it depends on the first usage. Examples where
> inheritance at thread create time is problematic are the thread context
> class loader, and until recently, the inherited access control context.
>
> When you say "custom schedulers based on the per-core model" do you mean
> you are experimenting with a scheduler per core in order to get "carrier
> affinity"  (virtual thread X will always be scheduled on carrier Y). You
> might also be using processor affinity to bind carrier Y to specific sets
> of CPUs.
>
> You might find it simpler to just use one scheduler and keep a mapping of
> virtual thread to carrier. That would remove complications with lifecycle
> that would arise if carriers were to terminate (e.g. idle/keep-alive). It
> might avoid needing to expose APIs to get a virtual thread's scheduler,
> which I think what your second point is about.
>
> -Alan
>

Alan Bateman <alan.bateman at oracle.com> 于2025年10月5日周日 00:35写道：

> On 04/10/2025 14:19, dreamlike_ocean lei wrote:
>
> Hello @loom-dev <loom-dev at openjdk.org>,
>
> I have been enjoying the new POLLER_PER_CARRIER design in the Loom repo
> and I really like the direction it is going.
> While building on top of the latest code, I noticed a couple of issues and
> would like to ask for clarification.
>
> Thanks for your mail. It's useful to hear from folks that are trying out
> the experimental support for custom schedulers in the loom repo as this
> will help to inform whether any of the directions prototyped should be
> taken further.
>
> BTW: I assume "dreamlike_ocean lei" isn't your real name. It would be
> helpful to use a real name or affiliation so we have some idea who you are.
>
>
>    1.
>
>    When calling Thread.startVirtualThread, the new virtual thread does
>    not inherit the scheduler of the calling virtual thread, but instead uses
>    DEFAULT_SCHEDULER. What is the reasoning behind this design? Could
>    there be a mechanism to allow implicit inheritance? This would be very
>    helpful for custom schedulers based on the per-core model.
>
> In a system with several virtual thread schedulers in use, which I think
> is what you mean, then it would be unpredictable and problematic to inherit
> the scheduler in many cases. If some library were, on first usage, start a
> virtual thread as a "background thread" that runs indefinitely then
> inheritance would mean it depends on the first usage. Examples where
> inheritance at thread create time is problematic are the thread context
> class loader, and until recently, the inherited access control context.
>
> When you say "custom schedulers based on the per-core model" do you mean
> you are experimenting with a scheduler per core in order to get "carrier
> affinity"  (virtual thread X will always be scheduled on carrier Y). You
> might also be using processor affinity to bind carrier Y to specific sets
> of CPUs.
>
> You might find it simpler to just use one scheduler and keep a mapping of
> virtual thread to carrier. That would remove complications with lifecycle
> that would arise if carriers were to terminate (e.g. idle/keep-alive). It
> might avoid needing to expose APIs to get a virtual thread's scheduler,
> which I think what your second point is about.
>
> -Alan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20251008/e69b6ca5/attachment.htm>