loom-dev Digest, Vol 40, Issue 33

[Arthur Navarro]

Thank you very much for taking the time to answer my long message.

How would you decide to do a blocking syscall?
>

I agree with you on this point, I remember reading about it some time ago
(I believe it was in "The state of Loom" ?) and seeing it in the source
code of jdk-17-loom. However the initial question specifically asked about
blocking syscalls and I wanted to answer it precisely, hence my will to
give some examples using blocking syscalls to illustrate what would happen
*if* we used blocking syscalls. As you pointed out, the old networking
example is outdated and might not be relevant anymore (thanks for
clarifying it by the way), but we can still use it as an illustration I
guess ?
JNIs are a way to access native code and perform blocking syscalls but you
have made it clear from the start that virtual threads are not likely to
enhance the performance of code using JNIs.

Also, the terminology behind "blocking" and "non-blocking" is sometimes a
bit blurry and it is easy to slip from "blocking syscalls" to a larger
meaning of blocking. This is the case with file IO where I don't know where
blocking syscalls end and where plain userspace long-running-CPU-intensive
code begins. But the fact remains that file I/O is often performed using a
threadpool rather than non-blocking tools (libuv does it this way for
instance if I remember correctly) and the use of virtual threads in this
context was not clear to me. Thankfully your answer made me look again and
you tackled it here
<https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html#all-your-blocking-are-belong-to-us>,
explaining that it is indeed problematic for now and that io_uring should
be helpful.

As to virtual thread overhead, creating a virtual thread costs just as much
> as
> creating any other object. The mechanism has other overheads compared, to
> say, hand-writing an equivalent state machine and reifying its state in
> some class,
> but also some optimisations compared to it, so whether the contribution
> will be
> positive or negative depends on specifics, and in the vast majority of
> cases
> shouldn’t matter.
>

I agree with you too. I should have been clearer than just throwing the
word "negligible" but I was afraid to make my answer even longer... And
more generally I can imagine that the need to provide a programmer-friendly
API in the form of virtual threads might lead to some tradeoffs now and
then.

Again, thank you very much for your answer.

On Mon, Apr 26, 2021 at 4:55 PM <loom-dev-request at openjdk.java.net> wrote:

> Send loom-dev mailing list submissions to
>         loom-dev at openjdk.java.net
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.openjdk.java.net/mailman/listinfo/loom-dev
> or, via email, send a message with subject or body 'help' to
>         loom-dev-request at openjdk.java.net
>
> You can reach the person managing the list at
>         loom-dev-owner at openjdk.java.net
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of loom-dev digest..."
> Today's Topics:
>
>    1. Re : Question on Project Loom (Arthur Navarro) (Arthur Navarro)
>    2. Re: Question on Project Loom (Arthur Navarro) (Ron Pressler)
>
>
>
> ---------- Forwarded message ----------
> From: Arthur Navarro <arnavarr at redhat.com>
> To: loom-dev at openjdk.java.net
> Cc:
> Bcc:
> Date: Mon, 26 Apr 2021 15:31:57 +0200
> Subject: Re : Question on Project Loom (Arthur Navarro)
> Thank you for your work on Project Loom and thank you for taking the time
> to answer the questions we have on this mailing list. I too am really
> excited to see what will come out of it.
>
> I would like to follow up on this question to share some of my thoughts
> (and thank you in advance for correcting me where I am wrong).
>
> >     1.  Current implementation of  Java Threads are wrappers around
> Native
> > >     Threads. In linux, threads are Process sharing Virtual Address
> Space
> > >     allocated to the parent process. If a Java thread makes a blocking
> > >     system call like Read() on a file descriptor, scheduler adds it to
> > the Wait
> > >     Queue, to be awakened later. When running in virtual threads.
> > blocking it
> > >     could be detrimental to the entire JVM. Is my understanding
> > correct?. If
> > >     it's correct then the only IO allowed from Virtual Thread will be
> > >     non-blocking syscalls, any traditional API that uses blocking under
> > the
> > >     hood will need to be invoked from a non virtual thread.
> >
>
> I am sorry if my answer is a little verbose, I can't really express it in a
> shorter way. There is a short summary at the end.
>
> Also my answer is based on the ea build of jdk 17-loom of April, I have
> very little knowledge of previous versions of the jdk and since I will talk
> about some lower-level classes it is worth mentioning that I know close to
> nothing about implementations for platforms other than modern linux.
>
> My understanding is that virtual threads love the *semantics* of blocking
> I/O since they are synchronous and programmers like synchronous code (for
> reasons that are either bound to habits and the current teaching of
> programming or to the wiring of our human brains, that I don't think we
> know), however virtual threads do not love blocking syscalls at all.
>
> I find the example of the networking socket quite good in this regard :
> internally the NioSocketImpl java class is used (a legacy version
> PlainSocketImpl exists, I don't know how many systems rely on that one),
> and it leverages *non-blocking facilities provided by the OS*. On modern
> Linux it uses epoll, on older ones it uses select or poll and on BSD it
> uses kqueue (I guess IOCP is used on windows ?).
> In Linux when we perform a non-blocking read operation on a networking
> socket we don't actually wait for it to complete, we merely register an
> event to the epoll instance and ask it to watch the event. The syscall
> returns almost immediately and we continue our computation. The thread is
> not put in any wait queue.
> In Java there is a class (named something like Poller) that communicates
> with an epoll instance and maintains a table matching epoll events and java
> callbacks.
> Now in Loom (finally) when we want to read on a socket we interact with
> this class and create an entry containing an epoll event and a java
> callback : *the resumption of the virtual thread* (ie. adding the virtual
> thread in the run queues consumed by java carrier threads), then we park
> the virtual thread (ie. store it in memory and allow its carrier thread to
> process the next virtual thread).
> When the OS is told that the socket is ready for IO it will signal it to
> the epoll instance. The Java Poller will see it and execute the java
> callback bound to the epoll event : adding the virtual thread in the run
> queues consumed by the carrier threads.
> We program *as if *the operation is blocking and synchronous, the Loom
> project conceals all the details to make non-blocking asynchronous code
> look synchronous while still being non-blocking.NioSocketImpl
>
> Now if we perform a blocking syscall it doesn't matter if we are running a
> virtual thread or not. As you said, we switch to kernel mode and the OS
> puts the thread (the native linux process on which is mapped the
> "traditional" Java thread) in a wait queue. A virtual thread is basically a
> java function executed by a java traditional thread so it will not make it
> better or worse than blocking a traditional thread.
> Typically if we use *PlainSocketImpl* instead of *NioSocketImpl* or any
> third-party library using blocking syscalls virtual threads should not be
> of any help.
> Also, disks I/O on linux are a difficult subject and although io_uring
> seems to be a major step towards non-blocking disks I/O with good API
> (starting linux 5.1 I believe ?) I don't think that non-blocking tools are
> widely used today (linux aio and posix aio for disks don't seem to get much
> tractions due to certain limitations), so I wonder if virtual threads would
> be of any use for disk IO currently, for Linux at least.
>
> *TL; DR*
> If I got it right, the Loom project aims at concealing the asynchronous
> nature of the non-blocking IO facilities provided by different OSs, but it
> is powerless if you decide to use blocking syscalls.
> My understanding is that your question assumed that performing blocking
> syscalls with virtual threads was inefficient (which it is if my
> understanding is correct), but you wanted to know if it was worse than
> doing so with a "traditional" java thread ?
> My current understanding of the Loom project tells me that it might add a
> negligible overhead due to the creation of the virtual thread but that is
> all.
>
>
> --
> *Arthur Navarro*
>
> Research and Development associate
>
> He/him/his
>
>
>
>
> ---------- Forwarded message ----------
> From: Ron Pressler <ron.pressler at oracle.com>
> To: Arthur Navarro <arnavarr at redhat.com>
> Cc: "loom-dev at openjdk.java.net" <loom-dev at openjdk.java.net>
> Bcc:
> Date: Mon, 26 Apr 2021 14:55:24 +0000
> Subject: Re: Question on Project Loom (Arthur Navarro)
> Maybe Alan would like to give a longer answer, but here’s my short one:
>
> How would you decide to do a blocking syscall? All IO operations go through
> the JDK. If you tell the JDK you want, say, a synchronous read, and the
> runtime
> sees you’re running in a virtual thread, it will emit a non-blocking
> syscall. To emit
> a blocking syscall you’d need to call native code that does that (or the
> runtime
> might do that in special cases where the underlying OS does not have a
> nonblocking operation). As to legacy networking, that implementation has
> been
> replaced with a Java one that wraps NIO, so the same thing would happen.
>
> As to virtual thread overhead, creating a virtual thread costs just as
> much as
> creating any other object. The mechanism has other overheads compared, to
> say, hand-writing an equivalent state machine and reifying its state in
> some class,
> but also some optimisations compared to it, so whether the contribution
> will be
> positive or negative depends on specifics, and in the vast majority of
> cases
> shouldn’t matter.
>
> — Ron
>
> > On 26 Apr 2021, at 14:31, Arthur Navarro <arnavarr at redhat.com> wrote:
> >
> > Thank you for your work on Project Loom and thank you for taking the time
> > to answer the questions we have on this mailing list. I too am really
> > excited to see what will come out of it.
> >
> > I would like to follow up on this question to share some of my thoughts
> > (and thank you in advance for correcting me where I am wrong).
> >
> >>    1.  Current implementation of  Java Threads are wrappers around
> Native
> >>>    Threads. In linux, threads are Process sharing Virtual Address Space
> >>>    allocated to the parent process. If a Java thread makes a blocking
> >>>    system call like Read() on a file descriptor, scheduler adds it to
> >> the Wait
> >>>    Queue, to be awakened later. When running in virtual threads.
> >> blocking it
> >>>    could be detrimental to the entire JVM. Is my understanding
> >> correct?. If
> >>>    it's correct then the only IO allowed from Virtual Thread will be
> >>>    non-blocking syscalls, any traditional API that uses blocking under
> >> the
> >>>    hood will need to be invoked from a non virtual thread.
> >>
> >
> > I am sorry if my answer is a little verbose, I can't really express it
> in a
> > shorter way. There is a short summary at the end.
> >
> > Also my answer is based on the ea build of jdk 17-loom of April, I have
> > very little knowledge of previous versions of the jdk and since I will
> talk
> > about some lower-level classes it is worth mentioning that I know close
> to
> > nothing about implementations for platforms other than modern linux.
> >
> > My understanding is that virtual threads love the *semantics* of blocking
> > I/O since they are synchronous and programmers like synchronous code (for
> > reasons that are either bound to habits and the current teaching of
> > programming or to the wiring of our human brains, that I don't think we
> > know), however virtual threads do not love blocking syscalls at all.
> >
> > I find the example of the networking socket quite good in this regard :
> > internally the NioSocketImpl java class is used (a legacy version
> > PlainSocketImpl exists, I don't know how many systems rely on that one),
> > and it leverages *non-blocking facilities provided by the OS*. On modern
> > Linux it uses epoll, on older ones it uses select or poll and on BSD it
> > uses kqueue (I guess IOCP is used on windows ?).
> > In Linux when we perform a non-blocking read operation on a networking
> > socket we don't actually wait for it to complete, we merely register an
> > event to the epoll instance and ask it to watch the event. The syscall
> > returns almost immediately and we continue our computation. The thread is
> > not put in any wait queue.
> > In Java there is a class (named something like Poller) that communicates
> > with an epoll instance and maintains a table matching epoll events and
> java
> > callbacks.
> > Now in Loom (finally) when we want to read on a socket we interact with
> > this class and create an entry containing an epoll event and a java
> > callback : *the resumption of the virtual thread* (ie. adding the virtual
> > thread in the run queues consumed by java carrier threads), then we park
> > the virtual thread (ie. store it in memory and allow its carrier thread
> to
> > process the next virtual thread).
> > When the OS is told that the socket is ready for IO it will signal it to
> > the epoll instance. The Java Poller will see it and execute the java
> > callback bound to the epoll event : adding the virtual thread in the run
> > queues consumed by the carrier threads.
> > We program *as if *the operation is blocking and synchronous, the Loom
> > project conceals all the details to make non-blocking asynchronous code
> > look synchronous while still being non-blocking.NioSocketImpl
> >
> > Now if we perform a blocking syscall it doesn't matter if we are running
> a
> > virtual thread or not. As you said, we switch to kernel mode and the OS
> > puts the thread (the native linux process on which is mapped the
> > "traditional" Java thread) in a wait queue. A virtual thread is
> basically a
> > java function executed by a java traditional thread so it will not make
> it
> > better or worse than blocking a traditional thread.
> > Typically if we use *PlainSocketImpl* instead of *NioSocketImpl* or any
> > third-party library using blocking syscalls virtual threads should not be
> > of any help.
> > Also, disks I/O on linux are a difficult subject and although io_uring
> > seems to be a major step towards non-blocking disks I/O with good API
> > (starting linux 5.1 I believe ?) I don't think that non-blocking tools
> are
> > widely used today (linux aio and posix aio for disks don't seem to get
> much
> > tractions due to certain limitations), so I wonder if virtual threads
> would
> > be of any use for disk IO currently, for Linux at least.
> >
> > *TL; DR*
> > If I got it right, the Loom project aims at concealing the asynchronous
> > nature of the non-blocking IO facilities provided by different OSs, but
> it
> > is powerless if you decide to use blocking syscalls.
> > My understanding is that your question assumed that performing blocking
> > syscalls with virtual threads was inefficient (which it is if my
> > understanding is correct), but you wanted to know if it was worse than
> > doing so with a "traditional" java thread ?
> > My current understanding of the Loom project tells me that it might add a
> > negligible overhead due to the creation of the virtual thread but that is
> > all.
> >
> >
> > --
> > *Arthur Navarro*
> >
> > Research and Development associate
> >
> > He/him/his
>
>

-- 
*Arthur Navarro*

Research and Development associate

He/him/his