A bug in the AOT runtime

[Vladimir Kozlov]

CCing to hotspot-compiler because we are responsible for AOT.

Andrew, it looks like next bug which I unfortunately did not have time 
to look on:

https://bugs.openjdk.java.net/browse/JDK-8146029

It did not show up on Linux because usually CodeCache and and AOTed code 
are located near each other on Linux.

Currently I am planning to look on it after JDK 11.

Regards,
Vladimir

On 6/4/18 7:51 AM, Andrew Haley wrote:
> I saw an extreme slowdown when running AOT-compiled code.
> 
> It's a situation where we get stuck perpetually calling the IC miss
> code every time a method is invoked, and it never gets fixed.  This
> happens when the AOT library is loaded more than 2G away from the code
> buffer, so calls cannot reach from compiled to AOT-compiled code.
> 
> The symptom looks like this:
> 
> FALSE IC miss (invokeinterface) converting to compiled call to jdk.internal.jrtfs.JrtPath::toString code: 0x00007f58bca9b570
> IC miss (invokeinterface) call to java.util.ArrayList$ArrayListSpliterator::tryAdvance code: 0x00007f58bcaa9680
> FALSE IC miss (invokeinterface) converting to compiled call to java.util.ArrayList$ArrayListSpliterator::tryAdvance code: 0x00007f58bcaa9680
> IC miss (invokeinterface) call to jdk.internal.jrtfs.JrtPath::getFileName code: 0x00007f58bca9e6f0
> FALSE IC miss (invokeinterface) converting to compiled call to jdk.internal.jrtfs.JrtPath::getFileName code: 0x00007f58bca9e6f0
> IC miss (invokeinterface) call to jdk.internal.jrtfs.JrtPath::getFileName code: 0x00007f58bca9e6f0
> FALSE IC miss (invokeinterface) converting to compiled call to jdk.internal.jrtfs.JrtPath::getFileName code: 0x00007f58bca9e6f0
> IC miss (invokeinterface) call to jdk.internal.jrtfs.JrtPath::toString code: 0x00007f58bca9b570
> FALSE IC miss (invokeinterface) converting to compiled call to jdk.internal.jrtfs.JrtPath::toString code: 0x00007f58bca9b570
> IC miss (invokeinterface) call to java.util.ArrayList$ArrayListSpliterator::tryAdvance code: 0x00007f58bcaa9680
> 
> ... and the call misses every time, never to be fixed up.
> 
> This is how it happens:
> 
> We have a call to the IC miss stub which is caused not by a true IC
> miss (i.e. different classes) but by AOT-compiled code becoming
> available.  We hit a transition stub, and that diverts us to the IC
> miss handler.
> 
> This is the logic in the transition stub.  The relevant part for us is
> the second part:
> 
>      __ load_klass(temp, receiver);
>      __ cmpptr(temp, Address(holder, CompiledICHolder::holder_klass_offset()));
>      __ movptr(rbx, Address(holder, CompiledICHolder::holder_metadata_offset()));
>      __ jcc(Assembler::equal, ok);
>      __ jump(RuntimeAddress(SharedRuntime::get_ic_miss_stub()));
> 
>      __ bind(ok);
>      // Method might have been compiled since the call site was patched to
>      // interpreted if that is the case treat it as a miss so we can get
>      // the call site corrected.
>      __ cmpptr(Address(rbx, in_bytes(Method::code_offset())), (int32_t)NULL_WORD);
>      __ jcc(Assembler::equal, skip_fixup);
>      __ jump(RuntimeAddress(SharedRuntime::get_ic_miss_stub()));
> 
> Because we have AOT-compiled code, the cmpptr with NULL failes and we
> enter SharedRuntime::handle_wrong_method_ic_miss().  If the call from
> compiled to AOT code cannot reach, we're going to need a stub.  We now
> execute this code:
> 
>    if (entry != NULL && !far_c2a) {
>      // Call to near compiled code (nmethod or aot).
> 
> ** We can't do this because the branch is too far
> 
>      info.set_compiled_entry(entry, is_optimized ? NULL : receiver_klass, is_optimized);
>    } else {
>      if (is_optimized) {
>        if (far_c2a) {
>          // Call to aot code from nmethod.
>          info.set_aot_entry(entry, method());
>        } else {
>          // Use stub entry
>          info.set_interpreter_entry(method()->get_c2i_entry(), method());
>        }
>      } else {
> 
> ** We do this
> 
>        // Use icholder entry
>        assert(method_code == NULL || method_code->is_compiled(), "must be compiled");
>        CompiledICHolder* holder = new CompiledICHolder(method(), receiver_klass);
>        info.set_icholder_entry(method()->get_c2i_unverified_entry(), holder);
>      }
>    }
> 
> So, we still have a transition stub, and it will do the same thing as
> last time.
> 
> When we compare Method::_code with NULL, we will find that it is not
> NULL: it is the AOT-compiled entry point.  So we will call the IC miss
> stub.  Every time, around and around.
> 
> What we actually need, I think, is a different kind of transition stub
> that does not check that Method::_code is zero but instead jumps
> straight to the AOT-compiled code.
>