RFR: 8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

[Claes Redestad]

On Thu, 26 Nov 2020 13:10:02 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Method linkage of `invokehandle` instructions involve an upcall into Java (`MethodHandleNatives::linkMethod`), but the result is not cached. Unless the upcall behaves idempotently (which is highly desirable, but not guaranteed), repeated invokehandle resolution attempts enter a vicious cycle in tiered mode: switching to a higher tier involves call site re-resolution in generated code, but re-resolution installs a fresh method which puts execution back into interpreter.
> 
> (Another prerequisite is no inlining through a `invokehandle` which doesn't normally happen in practice - relevant methods are marked w/ `@ForceInline` - except some testing modes, `-Xcomp` in particular.)
> 
> Proposed fix is to inspect `ConstantPoolCache` first. Previous resolution attempts from interpreter and C1 records their results there and it stabilises the execution.         
> 
> Testing:
>  - failing test
>  - tier1-4

Nice!

-------------

Marked as reviewed by redestad (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/1453