RFR: 8343377: Performance regression in reflective invocation of native methods

[Mandy Chung]

On Fri, 15 Nov 2024 22:17:10 GMT, Chen Liang <liach at openjdk.org> wrote:

> When core reflection was migrated to be implemented by Method Handles, somehow, the method handles are not used for native methods, which are generally linkable by method handles.  This causes significant performance regressions when reflecting native methods, even if their overrides may be non-native methods.  This is evident in `Object.clone` and `Object.hashCode` as shown in the original report.
> 
> I believe the blanket restriction previously placed on the native methods was because of signature polymorphic methods ([JLS 15.12.3](https://docs.oracle.com/javase/specs/jls/se23/html/jls-15.html#jls-15.12.3), [JVMS 2.9.3](https://docs.oracle.com/javase/specs/jvms/se23/html/jvms-2.html#jvms-2.9.3)) for MethodHandle and VarHandle; method handles do not link to the backing implementation that throws UOE while core reflection is required to do so.  I have narrowed the restrictions to be specifically against these methods.
> 
> Additionally, I cleaned up another check for invalid varargs flag.  Together, I clarified the scenarios where native method accessors are used - all to bypass restrictions of java.lang.invoke.
> 
> Testing: tier 1-5 green

Good performance improvement.  Indeed we can use method handles for native methods except during early VM startup before `java.lang.invoke` is fully initialized and the signature polymorphic methods as UOE is thrown if invoked via core reflection.

src/java.base/share/classes/jdk/internal/reflect/MethodHandleAccessorFactory.java line 376:

> 374:         // trailing array,  but core reflection ignores ACC_VARARGS flag like the JVM does.
> 375:         // Fall back to use the native implementation instead.
> 376:         if (isInvalidVarArgs(member))

I prefer to keep the check inlined here rather than an utility method as it's explicit what to check.

src/java.base/share/classes/jdk/internal/reflect/MethodHandleAccessorFactory.java line 459:

> 457: 
> 458:     private static final Unsafe UNSAFE = Unsafe.getUnsafe();
> 459:     private static final ReflectionFactory reflectionFactory = ReflectionFactory.getReflectionFactory();

As this now accesses shared parameter types, we can change this file to avoid cloning the array of parameter types in `slotCount` and creating a method type from a member.  This can be done in a separate PR.

test/jdk/java/lang/invoke/MethodHandleInvokeUOE.java line 26:

> 24: /*
> 25:  * @test
> 26:  * @bug 8343377

I'm surprised adding `@bug` in this test.  As no change made to this test, I think no need to add the bug ID.

-------------

PR Review: https://git.openjdk.org/jdk/pull/22169#pullrequestreview-2462621492
PR Review Comment: https://git.openjdk.org/jdk/pull/22169#discussion_r1859191509
PR Review Comment: https://git.openjdk.org/jdk/pull/22169#discussion_r1859197828
PR Review Comment: https://git.openjdk.org/jdk/pull/22169#discussion_r1859199051