RFR: 8353216: Improve VerifyMethodHandles for method handle linkers

[Vladimir Kozlov]

On Fri, 7 Mar 2025 20:58:15 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Add extra verification logic into `MethodHandle::invokeBasic/linkTo*` to ensure that holder classes are properly initialized.
> 
> The patch covers x86 and aarch64 platforms.
> 
> There are some differences in expectations between invocation modes.
> While `invokeStatic` assumes a clinit barrier (and `invokeBasic` just requires the holder class to be fully initialized), other invocation modes can only expect that class initialization has been initiated (due to class initialization failures and premature publication, instances of partially initialized classes can be observed).   
> 
> Testing: hs-tier1 - hs-tier4

src/hotspot/cpu/x86/methodHandles_x86.cpp line 131:

> 129:     Label L_ok;
> 130: 
> 131:     const Register method_holder = temp;

`assert_different_registers` missing. Or you don't need it?

src/hotspot/cpu/x86/methodHandles_x86.cpp line 139:

> 137:         // Require compiled LambdaForm class to be fully initialized.
> 138:         __ cmpb(Address(method_holder, InstanceKlass::init_state_offset()), InstanceKlass::fully_initialized);
> 139:         __ jccb(Assembler::equal, L_ok);

Should this also be long jump?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23950#discussion_r2019548630
PR Review Comment: https://git.openjdk.org/jdk/pull/23950#discussion_r2019549790