RFR: 8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

[Coleen Phillimore]

On Thu, 26 Nov 2020 13:10:02 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Method linkage of `invokehandle` instructions involve an upcall into Java (`MethodHandleNatives::linkMethod`), but the result is not cached. Unless the upcall behaves idempotently (which is highly desirable, but not guaranteed), repeated invokehandle resolution attempts enter a vicious cycle in tiered mode: switching to a higher tier involves call site re-resolution in generated code, but re-resolution installs a fresh method which puts execution back into interpreter.
> 
> (Another prerequisite is no inlining through a `invokehandle` which doesn't normally happen in practice - relevant methods are marked w/ `@ForceInline` - except some testing modes, `-Xcomp` in particular.)
> 
> Proposed fix is to inspect `ConstantPoolCache` first. Previous resolution attempts from interpreter and C1 records their results there and it stabilises the execution.         
> 
> Testing:
>  - failing test
>  - tier1-4

Looks great.  I had one question in the code.

src/hotspot/share/interpreter/linkResolver.cpp line 1705:

> 1703:   int cache_index = ConstantPool::decode_cpcache_index(index, true);
> 1704:   ConstantPoolCacheEntry* cpce = pool->cache()->entry_at(cache_index);
> 1705:   if (!cpce->is_f1_null()) {

If f1 is non-null any racing resolution is complete.  Can you double check if that's also the case for indy_resolution_failed?

-------------

Marked as reviewed by coleenp (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/1453