LLVM Talks Related to Coroutines

[John Rose]

On Feb 12, 2019, at 11:41 AM, Ron Pressler <ron.pressler at oracle.com> wrote:
> 
> In addition, the JVM has its own peculiar
> characteristics that can both help or hurt. E.g., heap allocations are very
> cheap, we have world-class GCs (so stack resizing is not much of an issue),
> debuggers/profilers are under our control, and use of native code is uncommon
> (outside of built-in JDK usage, which we also control) so we don't need to worry
> much about interop with native code.

The JVM also supports mixed-mode execution of the same
code via multiple back-ends, including transitions between
modes on the fly, based on dynamic conditions such as
profile measurements.  The interpreter and JITs (all of them)
use two different stack frame formats, and the VM can transition
between those formats, any number of times, within a single
method activation.  The transition from interpreter to JIT code
is called "on-stack replacement", while the other way is called
"deoptimization".  The former (OSR) involves jumping into a
specially-compiled JIT code block that has no proper LIFO entry
point for the running method, but rather a continuation point at
the bytecode which is selected as the transition site.  This
continuation point is entered with a parameter array which
display's the interpreter's current state (at that bytecode);
the JIT code code loads up registers and frame from that
state and off it goes to finish the method call.

The JIT code is optimized specifically for the transition point,
and typically doesn't even contain code to execute method actions
that have already happened (unless the method can loop back
to them).  The JIT code is fully customized to the method
code starting from a non-LIFO method entry point.
In short, method code is versioned according to how it is
expected to be called, though the front door or the side door.

The OSR technique is expensive (in both engineering and
execution) but it helps us optimize a narrow but important set
of use cases—a seldom-called, long-running method with
a hot loop.  The JVM's OSR infrastructure is fully paid for,
and perhaps useful for other use cases that involve jumping
into the middle of a hot computation, that is not already
already optimized by the usual means (starting with inlining).

I think there might be variations of the OSR technique,
using some of our existing infrastructure, which could
be useful to optimize lateral (non-LIFO) control flow patterns
between continuations that would otherwise be hard to optimize
as combinations of inlined methods calls and stack frame
mount/unmount operations.  I don't see a clear use case
yet (except maybe generators, which aren't our focus),
but we might find occasions where, assisted by profile
information, we can detect where a hot loop of non-LIFO
method transitions can be inlined into a custom JIT
compilation, and relaunched through an OSR-style transition.

Such customized code could be viewed as a compromise
between a pure backend and a pure frontend implementation
of continuations.  It would act like a frontend implementation,
in that the transition-hot code would be recompiled as if it
were initially marked for that treatment.  It would act like
a backend implementation, in that the code would interoperate
with the standard mixed VM modes for method execution.
The mixed-mode trick means that frontend compilation
decisions can be revisited retroactively.

I'm not saying we will need to do any of this, just that today's JVM
provides a rich set of options for tweaking the rules of backend
execution.

— John