Continuations before Fibers

Fri Mar 6 07:24:05 UTC 2020

Just to be clear I never suggested (or meant to suggest) that virtual
threads be pooled. Just that the OS threads that execute the virtual
threads be pooled.

I played with the EA build some more. You are reusing existing Thread API
for Fibers. So we have a Thread (virtual) that is run by another Thread
(non-virtual). Bold move. I think you made the right choice of not having
an extra Fiber API. Pretty impressed.

For disk IO. How do you intend to do it? Io_uring?

I have an existing queue and thread pool that does file io. It also keeps
track of read/write speeds. How would I integrate it with virtual threads?
So that I can do in a virtual thread something like byte[] data =
looksLikeBlockingCallButHeavyLiftingIsDoneOnMyCustomDiskIoThreadPool();

Ah, I think I got it. Virtual park/unpark. Basically LockSupport.park() in
the caller and LockSupport.unpark(virtualThread); from the existing thread
pool? I tried the code and it looks to work as expected. Very nice. Might I
add that this feels more natural than working with Continuation directly.

I do have one remark. The IO work is done on an IO thread. And the Virtual
Thread is run on the Executor pool. On multi socketed server. The file IO
might be done on one socket (calls unpark) and the virtual thread resumes
on another socket. There is an unnecessary NUMA memory access. As the IO
data that is on the IO thread CPU cache needs to be copied to the other
socket. Might be faster to copy the virtual thread stack to the IO socket
than to copy the IO data to the other socket. Guess I'd have to do some
benchmarks. Thoughts?

Alen

V V pet., 6. mar. 2020 ob 02:51 je oseba Ron Pressler <
ron.pressler at oracle.com> napisala:

> You don’t need to ever pool virtual threads. In fact, you shouldn't.
> There’s
> a convenient ExecutorService that creates a new thread for each task: [1]
>
> There are no kernel threads blocked for IO operations (except, currently,
> using the file system). The API is synchronous but, when used on virtual
> threads
> the actual IO operations are non-blocking.
>
> I am unclear on why you think you’ll need to rewrite monitoring etc.. Can
> you
> explain?
>
> - Ron
>
>
>
> [1]:
> https://download.java.net/java/early_access/loom/docs/api/java.base/java/util/concurrent/Executors.html#newUnboundedExecutor(java.util.concurrent.ThreadFactory)
>
>
>
>
> On 5 March 2020 at 22:41:32, Alen Vrečko (alen.vrecko at gmail.com(mailto:
> alen.vrecko at gmail.com)) wrote:
>
> > Hi, Ron.
> >
> > Ignore the email that I sent you. Thought I clicked reply to all but was
> reply. No loss as I made some wrong assumptions.
> >
> > I think I get it now.
> >
> > With Virtual Threads by default I'd have just one thread pool (without
> any extra code). At the end of the day system io call would still be called
> and a thread would still be blocked. Minus the needing of having custom
> queues and thread pools and doing the awkward capturing of Continuation.
> What you said. It would make my code example simpler without any loss in
> functionality.
> >
> > But in existing legacy code, the queues and thread pools do more. There
> is monitoring, rate limiting, priority queueing, timeouts, etc. Also I can
> do some single writer principled code and in some cases use ManyToOne
> queues instead of ManyToMany queues.
> >
> > With continuations I'd be able to free some threads right away with just
> awkwardly (yet with little code) integrate into existing code (queues and
> pools). With virtual threads I'd have to redesign and re-facture the
> monitoring, rate limiting, priority queueing, timeouts etc code. Or maybe I
> am not seeing something?
> >
> > Thank you for your time
> > Alen
> >
> > V V čet., 5. mar. 2020 ob 20:32 je oseba Ron Pressler napisala:
> > > Hi.
> > >
> > > How is this use case not even *better* with virtual threads? Seems to
> me
> > > that it will require less code (as the necessary scheduling is already
> in the JDK),
> > > and will be able to run more Java code as it won’t break certain
> invariants such
> > > as the identity of the current thread remaining the same for the
> duration of a
> > > method.
> > >
> > > I *think* that every use case of continuations that depends on
> switching threads
> > > is better served by virtual threads. The only time I would consider
> using another
> > > continuation-based construct is for constructs that are essentially
> single threaded,
> > > e.g. generators. Am I missing something?
> > >
> > > For that reason — virtual threads are easier, less error-prone, and
> better integrated
> > > with the JDK — I don’t see us releasing continuations before virtual
> threads, or even
> > > making continuations public in the first release of virtual threads.
> Unless, of course,
> > > some very compelling use case arises. Do you think your use case
> benefits more
> > > from continuations than from threads? If so, why?
> > >
> > > BTW, in your case, as in the implementation of virtual thread, a
> single static final
> > > continuation scope seems sufficient.
> > >
> > > Ron
> > >
> > >
> > > On 5 March 2020 at 18:53:23, Alen Vrečko (alen.vrecko at gmail.com
> (mailto:alen.vrecko at gmail.com)) wrote:
> > >
> > > > Hello, everyone.
> > > >
> > > > On your website you have this:
> > > >
> > > > Continuations are intended as a low-level API, that application
> authors are
> > > > not intended to use directly. They will use higher-level constructs
> built
> > > > on top of continuations, such as fibers or generators.
> > > >
> > > > I took the current EA build for a spin. There is support for
> Continuations.
> > > > It solves my use case.
> > > >
> > > > I've put an example of how I'd use continuations here:
> > > >
> > > >
> https://gist.github.com/avrecko/a706484e6d7f4963af75a042006f8a4a(https://urldefense.com/v3/__https://gist.github.com/avrecko/a706484e6d7f4963af75a042006f8a4a__;!!GqivPVa7Brio!IDC-OHh3_kwRS_QC25tUbnOKf1agzoOnByLbcvYeYDSXk3CUulhC7CPSCB_NVwYHPg$)
> > > >
> > > > Basically using Continuations with a SEDA style application. The
> current EA
> > > > build implementation looks like it works as expected and solves my
> use case.
> > > >
> > > > Is this something you would _not_ like the users to do? I can see
> myself
> > > > using this kind of approach in production very easily without drastic
> > > > changes to the existing code base and without much surprises.
> > > >
> > > > Briefly looking at the code of Continuation. It looks thread safe.
> So there
> > > > shouldn't be problems from passing it around and being executing in
> part
> > > > from different threads.
> > > >
> > > > I do find the Scope argument a bit annoying. I think you could
> provide an
> > > > implicit default scope. So unless explicitly specified an implicit -
> > > > default scope is assumed.
> > > >
> > > > In my view you should consider releasing Continuations before Fibers
> and
> > > > making Continuations part of public api to be used by developers.
> > > >
> > > > I find the current API pretty intuitive to use. Minus the scopes. I
> like
> > > > that #run() is final and that Continuation takes Runnable. Makes
> sense to
> > > > me.
> > > >
> > > > Best regards
> > > > Alen
>
>