RFR: 8320302: compiler/arguments/TestC1Globals.java hits SIGSEGV in ContinuationEntry::set_enter_code [v3]

[David Holmes]

On Tue, 6 Feb 2024 20:11:55 GMT, Patricio Chilano Mateo <pchilanomate at openjdk.org> wrote:

>> src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp line 1747:
>> 
>>> 1745:                                               oop_maps,
>>> 1746:                                               exception_offset);
>>> 1747:     if (nm == nullptr) vm_exit_during_initialization("Initial size of CodeCache is too small");
>> 
>> IIUC `AdapterHandlerLibrary::create_native_wrapper`  is the only caller of `generate_native_wrapper` so it is only called during VM init now. But not sure it wouldn't be better/cleaner to simply return nullptr here and have `AdapterHandlerLibrary::create_native_wrapper` do the `vm_exit_during_initialization`?
>
> We could do that too. We have to check for nullptr && method->is_continuation_native_intrinsic() since AdapterHandlerLibrary::create_native_wrapper() itself is called from 2 other places which should not fail with vm_exit_during_initialization() if creation of the native nmethod failed. I don't have a preference, both ways are fine with me.

Sorry for the delay. I've been looking at the proposed initialization sequence:

Threads::create_vm
  initialize_class(vmSymbols::jdk_internal_vm_Continuation(), CHECK
    // causes linking of Continuation
    InstanceKlass::link_class_impl(TRAPS)
      InstanceKlass::link_methods(TRAPS)
        Method::link_method(mh, TRAPS)
          AdapterHandlerLibrary::create_native_wrapper
            SharedRuntime::generate_native_wrapper

and it seems to me that if `generate_native_wrapper` returns null then `create_native_wrapper` can check for that and also return early. Then `link_method` can do what other callers of `create_native_wrapper` do and simply check the resulting method and throw an exception:

if (!h_method->has_compiled_code()) {
  // throw OOME
}

that exception will propagate up and terminate the VM the same as any other exception during vm init would.

What do you think?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17455#discussion_r1482462165