coroutine support

[Charles Oliver Nutter]

On Fri, Nov 13, 2009 at 11:09 AM, Jochen Theodorou <blackdrag at gmx.org> wrote:
> the alternative would be to generate for run something like:
>
> private Object run$co(Context context) {
>   l0:
>   switch (context.jumpToLabel) {
>      case 1: jmp l1
>   }
>   context.l0=0;
>   for (;context.l0<10; context.l0++) {
>     System.out.println(context.l0);
>     l1:
>     context.jumpToLabel=1;
>     return null;
>   }
>   context.jumpToLabel=-1;
>   return null;
> }
>
> jumpToLabel is used to identify the entry/exit point, context will hold
> all local variables (l*)
>
> Where would such an approach be not as good as yours?

This is basically what all the bytecode-weaving coroutine/continuation
libraries do. Jython also does this for their
lambda/generator/coroutine stuff so they can jump in and out. The
primary issue is that you can only jump in and out of the methods you
have performed this transformation on. That means you can't call
through any untransformed code and expect to be able to resume it.
This is exactly the problem we face in JRuby, where coroutines could
have arbitrarily complex call stacks attached to them, across multiple
layers of logic, potentially through code we do not control or
can't/won't modify in this way. Here's a simple example:

class MyList
  def each
    [1,2,3].each {|i| yield i}
  end
end

enum = MyList.each
enum.next
enum.next
enum.next

The "each" in this code does a second internal iteration over a normal
Ruby Array, yielding each of its items in turn. But because we're
using the "next" method on Enumerator, this acts like a generator,
with yield returning control to the caller of "next" after each item.
In order to resume the array iteration when "next" is called again, we
would have to make all of the following resumable:

* The MyList#each method itself
* Array#each
* The block passed to Array#each
* All call logic and protocols for methods and blocks

So even for this simple example, the switch-based approach falls down
immediately. Full coroutine support means we can just bounce in and
out of that stack without having to manipulate any bytecode at all,
even across code we don't control.

- Charlie