Comment on fibers and async library interfaces

[Ron Pressler]

There are two different issues here: async IO and async APIs (or programming style).

Fibers do async IO automatically given a synchronous API. I don’t think I understand your concern about deemphasizing async APIs.
Is a server that uses blocking APIs with fibers considered a “pure async server” or not? If not, why not? 
Under the covers, only async IO is used. This is the same as in Erlang and Go.

You also write that "The opposite requires threads to be spawned and that defeats the purpose of async for scalability/throughput”,
but the whole point of lightweight threads is that spawning (and blocking) them is cheap so that it does harm scalability/throughput.

In what cases will fibers not suffice?

Ron


On May 24, 2019 at 10:11:32 AM, Nils Henrik Lorentzen (nils.lorentzen at gmail.com) wrote:

Hi,  


I am a longtime Java developer just becoming aware of project Loom and its  
lightweight threads. It seems like an idea well worth implementing in the  
core JVM/libraries for easier making scalable server applications.  



From reading the proposal at  
https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html, I do have a  
few concerns though, if I have understood fibers correctly.  


Not subscribed to the list so sending this as food for thought.  


These are mainly related to that there probably will be corner cases where  
one has to write async code, and fibers just will not suffice. These cases  
might be not be known yet but will surface in the future.  


What is of concern is the statement "In addition to making concurrent  
applications simpler and/or more scalable, this will make life easier for  
library authors, as there will no longer be a need to provide both  
synchronous and asynchronous APIs for a different simplicity/performance  
tradeoff."  


I understand this is written with the best intentions, who wouldn't want to  
make life easier for library writers, and I have no intention to criticise  
the author on this.  

What I am wary of here is that this might discourage providing async APIs,  
even at the low level, which will then make it way more difficult to write  
pure async servers if need be. Or one just prefers that way of programming  
(better logs solve much of the no-proper-stacktrace issues, and better log  
capabilities are also a plus in prodution deployed systems)  


Consider JDBC as an example, one is now at long last working on providing  
async JDBC drivers that can be useful for high throughput processing and  
reactive/async apps.  


When it comes to network communication, similarily to what the proposal  
states that async/await can be easily implemented by continuations, so can  
a synchronous network driver API easily be made on top of an asynchronous  
driver. The opposite requires threads to be spawned and that defeats the  
purpose of async for scalability/throughput.  

Keep in mind that even for request/response protocols, the base  
communication is always async by nature. There is no blocking operation on  
an ethernet card :) Thus async operation on top of a sync driver means  
async network => sync API => thread to simulate asynchronousity => async  
application, which is a long chain for something that was asynchronous in  
the first place.  


I would argue that for essential drivers (especially proprietary ones like  
JDBC), one should always implement an async API at the base using NIO and  
then just have a generic sync wrapper on top.  


Async driver at the core does imply either spawning a thread in the driver  
for its own select() mainloop or an API for integrating NIO Selectors into  
another mainloop (eg. of an application server) but should be manageable.  


An example of this architecture is Erlang. From what I can tell, socket  
communication is non-blocking and done via message passing between  
processes. The trick (and elegance) of Erlang is that it has a "selective  
receive for messages" and from what I can tell, 'receive' is pretty much  
the only place in all of Erlang that it would suspend lightweight threads  
(probably a setjmp()/longjmp() libc call at that place in its VM).  


For an async network driver in Java would be the blocking API doing  
Object.wait()/notify() for threads. For suspend in fibers, the underlying  
sync/async wrapper implementation could continue the fiber when there is  
input (or on writeability for writes).  



Just raising a flag here a bit because even if it is not such now, it could  
become a classic case of group think where async becomes discouraged, and  
then at some point one figures one needs it anyways. Except that all APIs  
have adopted synchronous functioning and it would be even more difficult to  
convince someone provide async network drivers as they would argue that  
fibers should solve it, so no need for it.  

Lightweight threads have a bright future but hopefully not at the expense  
of tried and proven patterns for high throughput servers :)  


Kind regards,  

Nils Henrik Lorentzen