Crash from "recursive" method handle chain

[Charles Oliver Nutter]

On Wed, Jul 27, 2011 at 2:27 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
> Presumably that's a valid implementation of selectAlternative which seems to disallow these kinds of shenanigans at least until Java requires tail call elimination.

That was my expectation as well. It's a cute little case, since it's a
circular chain of method handles at its core, but the invocation
behavior ends up acting recursive.

I discovered this intentionally. Ola explained the call site rebinding
trick to me, so I wanted a simple example to play with. After getting
it to work, I realized that the algorithm is recursive, even if it
feels loopy, because the call site acts like a call boundary. The
crash was unexpected, but some sort of failure was expected. I was
hopeful, though :)

> Running against b147 also gives StackOverflowError, at least sometimes, but it's more mysterious:
>
> /java/re/jdk/1.7.0/promoted/all/b147/binaries/solaris-amd64/fastdebug/bin/java -server -XX:+PrintCompilation -Xint IndyLoop
> Exception in thread "main" java.lang.StackOverflowError
>        at IndyLoop.add(IndyLoop.java:64)
>        at IndyLoop.main(IndyLoop.java:13)
>
> In this case I believe that we're running out space in the frame because of how we are doing the argument swizzling for MH chains.  We often do expansion without cleaning up since we assume the amount of expansion is bounded.  Making a loop greatly extends the amount of expansion.  I believe the crashes are caused by this happening in I2C adapter code which isn't prepared to deal with the overflow.  In general it appears that the adapter code needs some stack overflow checking.  It may also need fixes to keep from over expanding the argument area.  That might allow this to run to completion in interpreted mode but compiles are a different story.

Yes, this occurred to me too. I'm not really dropping and inserting
args, I'm really shifting args to the left with every iteration. And
that also bothered me about this implementation.

>> I'd expect to happen if everything compiles and inlines together; the
>> MH chain would see that the call site is bound to an
>> already-encountered handle higher up in the chain and treat it as a
>> branching operation.
>
> The compile could conceivably do something like this but it certainly doesn't right now.  It also seems pretty far outside the design center of 292 to make it behave like this, or at least the spec should give some guidance on whether method handle cycles are required to behave as you describe.  Anyway, John would probably have the last word here.  I'm really just talking about what happens as opposed to what should happen.
>
> Also note that this loop will never get compiled in a normal fashion since there are no invocation counters on method handles.
>
> I filed 7071915 for this since we shouldn't simply crash, though a StackOverflowError seems permissible to me.

It seems permissable to me too, but I'm also sad that I again have no
way to express a looping construct (or a general-purpose backward
branch) within a method handle chain.

- Charlie