Understanding the performance of my FFI-based API

[Maurizio Cimadamore]

On 14/03/2023 11:06, Alan Paxton wrote:

> You might be able to point me at something that explains what goes on 
> under the cover of invocation, and why exact matters ? My overall 
> takeaway is that there are a number of rules of thumb for making use 
> of FFI fast, if you follow them you get equivalent performance to JNI, 
> with safety for free.

Hi Alan,

thanks for coming back to us - it’s great that you have been able to 
reproduce the results.

As for why “exact” matter - that has to do with the method handle 
machinery (upon which var handles are also built). Both 
VarHandle::get/set and MethodHandle::invokeXYZ are “polymorphic 
signature” methods. This means that the java static compiler will look 
at these calls, and emit a /specialized/ method descriptor for the call. 
So, if you have:

|String s = (String)methodHandle.invokeExact(10); |

The compiler will emit a descriptor for that call with type 
|(int)String|. (A similar thing happens for var handle calls).

Now, the “exact” bit has to do with whether the JVM can “trust” that the 
symbolic description emitted by javac is the same as the type of the 
invoked method handle. If that’s the case, we can go straight to the 
method handle, and most (all) of the method handle machinery goop 
disappears when the call is inlined.

If, however, we use an “inexact” call (e.g. MethodHandle::invoke), we 
are telling the JVM that it needs to be prepared to perform some 
adaptations - that is, the symbolic description might be different from 
the method handle type. For instance, our static descriptor might say:

|(int)String |

but our underlying method handle might be like this:

|(Integer)String |

Method handles allow these adaptation (boxing, and widening, mostly) by 
wrapping the target method handle in a so called “asType” adaptation. 
This returns a /new/ method handle which has a signature that matches 
that of the incoming arguments, converts them to the right values, and 
then forwards them to the original method handle.

So, there’s one extra level of indirection and, worse, since the same 
method handle can be adapted to many different signatures (e.g. both 
|(int)String| and |(Object)int| are valid adaptation for 
|(Integer)String|), the method handle maintains a “type cache”, so that 
if we keep adapting the same method handle to the same type we end up 
creating another adapter method handle (as we already have one). This 
explains why type adaptation typically ends up hurting performance quite 
a bit (even though the JVM have come a long way to deal with those in 
the last few years :-) ).

(Sidebar: the Java 19 FFM API, with its separation between 
MemorySegment, MemoryAddress and Addressable makes it really hard to 
write /exact/ method handle invocations, as the downcall method handle 
type says Addressable, while the user will probably just have a 
MemorySegment. This means that, in order to have javac write the correct 
symbolic description, users should cast their segment to Addressable, 
which seems counterintuitive. This was one of the main reasons as to why 
in 20 we have simplified the API to remove the split between 
MemorySegment and MemoryAddress).

Var handles work pretty much in the same way - except that, for var 
handles, we don’t have inexact and exact version of get/set (that would 
have resulted in way too many methods). Instead, the var handle will act 
as “exact” or “inexact” based on whether the symbolic description 
matches the var handle type. If you get the exact path, all is good, and 
no type adaptation is required. If you go the inexact path, then you go 
through some form of MethodHandle::asType, which ends up hurting 
performance.

Detecting performance issues due to lack of exact var handle invocation 
has been historically tricky. The “withInvokeExactBehavior” was added 
precisely to allow developers to “express their intentions” more clearly 
(kudos to Jorn!).

As for the need for |final|, the JVM can only truly apply maximum 
inlining of var handles and method handles if these are “true constants” 
- which, for the JVM this means /both/ static /and/ final.

Maurizio

​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230314/12b26a01/attachment-0001.htm>