RFR: 8256368: Avoid repeated upcalls into Java to re-resolve MH/VH linkers/invokers

[Claes Redestad]

On Fri, 27 Nov 2020 19:02:04 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Method linkage of `invokehandle` instructions involve an upcall into Java (`MethodHandleNatives::linkMethod`), but the result is not cached. Unless the upcall behaves idempotently (which is highly desirable, but not guaranteed), repeated invokehandle resolution attempts enter a vicious cycle in tiered mode: switching to a higher tier involves call site re-resolution in generated code, but re-resolution installs a fresh method which puts execution back into interpreter.
>> 
>> (Another prerequisite is no inlining through a `invokehandle` which doesn't normally happen in practice - relevant methods are marked w/ `@ForceInline` - except some testing modes, `-Xcomp` in particular.)
>> 
>> Proposed fix is to inspect `ConstantPoolCache` first. Previous resolution attempts from interpreter and C1 records their results there and it stabilises the execution.         
>> 
>> Testing:
>>  - failing test
>>  - tier1-4
>
> src/hotspot/share/interpreter/linkResolver.cpp line 1705:
> 
>> 1703:   int cache_index = ConstantPool::decode_cpcache_index(index, true);
>> 1704:   ConstantPoolCacheEntry* cpce = pool->cache()->entry_at(cache_index);
>> 1705:   if (!cpce->is_f1_null()) {
> 
> If f1 is non-null any racing resolution is complete.  Can you double check if that's also the case for indy_resolution_failed?

My reading is that a non-null f1 means any resolution races is complete and writes to indy_resolution_failed (part of flags) is done before writing f1, see comments in and logic in  [ConstantPoolCacheEntry::set_method_handle_common](https://github.com/openjdk/jdk/blob/7ed591cc9a1c46f7c0b10a4c4ad3c181333bf5b6/src/hotspot/share/oops/cpCache.cpp#L364). It thus seem consistent and correct to resolve successfully here, no?

-------------

PR: https://git.openjdk.java.net/jdk/pull/1453