Request for reviews (M): 7090904: JSR 292: JRuby junit test crashes in PSScavengeRootsClosure::do_oop

[Christian Thalinger]

http://cr.openjdk.java.net/~twisti/7090904/

7090904: JSR 292: JRuby junit test crashes in PSScavengeRootsClosure::do_oop
Reviewed-by:

A couple of JRuby junit tests crash in various forms (mostly GC
related) when running on 64-bit Linux.  Although it looks like a GC
related problem it actually is a deoptimization bug.

Tom already had a fix ready but we were unable to write a test case to
prove the correctness of the fix.  An assert added in
AbstractInterpreter::layout_activation shows that the JRuby crashes
are indeed an incarnation of this deoptimization bug.

The bug itself is in the deoptimization logic that calculates the size
of parameter space in the caller frame.  Method handle invokes may
involve fairly arbitrary chains of calls so it's impossible to know
how much actual space the caller has for locals.

The fix is to always assume zero parameters for method handle call
sites.

Extensive testing with JRuby junit tests.

src/share/vm/runtime/deoptimization.cpp
src/cpu/x86/vm/templateInterpreter_x86_32.cpp
src/cpu/x86/vm/templateInterpreter_x86_64.cpp
src/share/vm/interpreter/bytecodeTracer.cpp
src/share/vm/runtime/frame.cpp
src/share/vm/runtime/frame.hpp
src/share/vm/runtime/thread.cpp