[9] RFR(S): 8173699: Crash during deoptimization with "assert(result == __null || result->is_oop()) failed: must be oop"

[Tobias Hartmann]

Hi,

please review the following patch:
https://bugs.openjdk.java.net/browse/JDK-8173699
http://cr.openjdk.java.net/~thartmann/8173699/webrev.00/

The Graal compiled method java.lang.invoke.MemberName$Factory::resolve() calls into the method handle runtime via MethodHandleNatives::resolve() which throws a NoSuchMethodError because method resolution failed (see methodHandles.cpp, line 1234). We then call into JVMCIRuntime::exception_handler_for_pc() -> SharedRuntime::compute_compiled_exc_handler() to determine the appropriate exception handler. Because the ExceptionHandlerTable has no entry for this pc, we deoptimize and return to the DeoptimizationBlob at offset _unpack_with_exception_in_tls which calls Deoptimization::fetch_unroll_info(). Since the callee returns a MemberName object, the ScopeDesc is marked as return_oop() and the re-allocation code expects the return register (eax) to contain the oop of this returned object. We fail when trying to save the oop, because eax contains not an oop but the address of SharedRuntime::deopt_blob()->unpack_with_exception_in_tls() which was returned from JVMCIRuntime::exception_handler_for_pc() right before and is therefore still in eax.

As Tom suggested in the bug comments, we should ignore the return_oop() when dispatching an exception and only try to retrieve the oop when performing re-allocation during a normal deoptimization (if exec_mode == Unpack_deopt). This code should be refactored with JDK 10. I filed JDK-8173823.

This problem only affects JVMCI compiled code. C1 does not set return_oop() because it does not eliminate allocations (see IRScopeDebugInfo::record_debug_info()) and therefore does not need to re-allocate objects on deoptimization. C2 computes the exception handler via OptoRuntime::handle_exception_C() and uses DeoptimizationBlob::_unpack_with_exception as handler in case the nmethod was deoptimized. When calling Deoptimization::fetch_unroll_info() from the DeoptimizationBlob, eax still contains the exception oop and therefore the code works "by accident" because the exception oop is treated as returned oop.

Tested with JVMCI test and RBT (running).

Thanks to Tom and Dean for the help!

Thanks,
Tobias