Proposal for Hybrid Threading Model and simpler Async IO

[Joel Richard]

Hi David,

The Java API would not have to be changed. Let me explain it with an
example:

Right now, InputStream.read(byte b[]) is a blocking method. Hence the
native thread waits until the byte array got filled. With my proposal, the
underlaying blocking native method (for example
java.net.SocketInputStream#socketRead0) would not block the native thread
anymore. Instead, it would call the C function with the _async suffix and
continue to process another task. As soon the async operation has
completed, it can then continue with the first task (maybe even in another
native thread). So the Java code pauses while waiting for IO (as so far),
but it doesn't block a native thread for that anymore.

So yes, new native non-blocking code would have to be written, but for the
developers the API does not change. Does that make sense?

Regards, Joel


On Sun, May 4, 2014 at 1:14 PM, David Holmes <david.holmes at oracle.com>wrote:

> Hi Joel,
>
>
> On 4/05/2014 9:08 PM, Joel Richard wrote:
>
>> Hi David,
>>
>> Thank you for your response. I have started a new thread here:
>> http://cs.oswego.edu/pipermail/concurrency-interest/2014-May/012638.html
>>
>> I am aware that the required changes would be complicated, but I think
>> it is worth a discussion since concurrency, non-blocking IO and high
>> scalability will be even more important in the future. This feature
>> would make it much easier to fulfill such requirements. For example, it
>> would not be necessary anymore to create a second version of each so far
>> blocking API.
>>
>
> What you are proposing already requires creating a second version. The
> bulk of the I/O code lies in the JDK libraries not the VM. If you want to
> make them non-blocking you will have to replace them all with non-blocking
> versions.
>
> David
> -----
>
>  Also all standardized APIs like JDBC or JPA would suddenly
>> all have a non-blocking implementation. The amount of work which could
>> be saved would be enormous.
>>
>> Regards, Joel
>>
>>
>> On Sun, May 4, 2014 at 2:46 AM, David Holmes <david.holmes at oracle.com
>> <mailto:david.holmes at oracle.com>> wrote:
>>
>>     Hi Joel,
>>
>>     The best place to raise threading issues/discussions is on Doug
>>     Lea's concurrency-interest mailing list.
>>
>>     http://altair.cs.oswego.edu/__mailman/listinfo/concurrency-__interest<
>> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest>
>>
>>
>>     However I don't think this proposal will get much traction. There is
>>     already async I/O in the core Java libraries. And making the
>>     synchronization primitives non-blocking would require a completely
>>     different implementation within the VM - something which would be
>>     far too destabilizing to ever consider without some very significant
>>     benefit.
>>
>>     Further fork-join tasks already go some way towards this, though
>>     with a more limited programming model.
>>
>>     David Holmes
>>
>>
>>     On 3/05/2014 2:34 AM, Joel Richard wrote:
>>
>>         Hi,
>>
>>         Recently I have researched a little about how different
>>         languages implement
>>         async IO because I wanted to implement a high scalability web
>>         server. When
>>         I looked at Google Go, I realized that the common callback
>>         approach is
>>         actually just a language-level workaround to fix a VM issue.
>>
>>         So I wondered how it could be possible to implement something
>>         similar to
>>         goroutines for the JVM. Here are my thoughts: There should be a
>>         class
>>         ThreadRoutine which implements the Runnable interface. If you
>>         use any (so
>>         far) blocking methods in it (like Thread.sleep, network/file
>>         system IO,
>>         Object.wait and synchronization) it uses however not the
>>         blocking native
>>         implementation, but an async version of it. While it waits for
>>         the method
>>         to complete, it can run another ThreadRoutine in the same native
>>         thread.
>>         This means that all ThreadRoutines can share a small native
>>         thread pool. If
>>         longer computations have to be done, the developer can either call
>>         Thread.yield() from time to time or just moves the computation to
>> a
>>         classical thread.
>>
>>         With such an approach it would be much easier to write highly
>>         scalable
>>         server software* and it would make it even possible to run most
>>         existing
>>         libraries with minimal changes as asynchronous IO libraries.
>>         Basically,
>>         they would just have to make sure that they call Thread.yield()
>>         in longer
>>         computationally intensive loops and don't start additional
>>         normal threads.
>>         Last but not least, the behavior of existing threads wouldn't be
>>         changed at
>>         all.
>>
>>         * The advantage of async IO compared to blocking IO is that it
>>         doesn’t
>>         require expensive thread context switches and that it needs less
>>         memory
>>         (because fewer threads allocate less memory for stacks).
>>
>>         As mentioned above, this would require implementing an
>>         asynchronous version
>>         of all blocking native methods. I am not that familiar with JNI
>>         nor C, but
>>         I think this could by achieved by allowing the developer to
>>         define a second
>>         C function with the _async suffix which calls a callback instead
>> of
>>         returning a value. If the Java native method is called in a
>>         normal thread,
>>         the old C function is called and otherwise the _async version of
>>         it. Since
>>         Java already supports async IO with NIO, I think it should be
>>         technically
>>         possible to implement asynchronous versions for almost all of
>>         the currently
>>         blocking native functions. If an operating system doesn't
>>         support certain
>>         non-blocking operations, it could just block a special thread
>>         instead (as
>>         far as I know Go does it similar).
>>
>>         What do you think about this idea? Has something like that been
>>         discussed
>>         before?
>>
>>         Regards, Joel
>>
>>         P.S. I wasn't sure which mailing list is the correct for such an
>>         idea. If
>>         it is completely wrong here, please forward it.
>>
>>
>>