jextract generation for functional interfaces

[Maurizio Cimadamore]

Hi Duncan,
we took a better look at this, and noticed that, in reality, there’s not 
much we can do at the moment about this, for two reasons.

First, the idea behind your approach was to move some of this shared 
functionality (such as the “allocate” method) to the Upcall class. But 
this changes the client code in ways that IMHO are not optimal — e.g.

|compar.allocate(...) |

becomes

|compar$func.UPCALL.allocate(...) |

So I think that, to make this work, we would have to replicate the 
|allocate| method (and possibly other accessors) in the toplevel class — 
which reduces the gain.

A better strategy would be one where the toplevel class “extends” from 
Upcall — then all the various methods would be inherited — and clients 
would be good again. But if we do that, we lose “constant-ness” of the 
various method handles involved, which will make methods like “invoke” 
slower.

Another difference between the two approaches is that you have merged 
the functional pointer wrapper with the functional interface. This is a 
move we have considered when we transitioned to this new code generation 
strategy, but we ended up rejecting because there’s no way to make 
interface fields non-public. Which is why we opted for an enclosing 
/class/ with a nested interface.

For these reasons, I believe that, for the time being, it’s better to 
keep the generated code as is. It is possible that the StableValue API 
[1] will allow us to change this — at some point we considered making 
fields of type StableValue implicitly trusted finals. While we have 
removed that trick in the current PR, something like that might allow 
for the code generation strategy I was alluding to earlier — where the 
function pointer class extends from some Upcall class that has a pair of 
stable value fields, one for the upcall MH, the other for the downcall 
MH. (In general, we’re confident that StableValue should result in 
jextract bindings become much more compact, as SV will essentially 
remove the need for all the holder classes that are spread throughout 
the generated code).

Thanks for the exploration!
Maurizio

[1] - https://github.com/openjdk/jdk/pull/23972

On 07/04/2025 16:09, Maurizio Cimadamore wrote:

> I like this. The use of record components should probably preserve the 
> fact that the upcall method handle is treated as a constant.
>
> This seems related to this PR under review:
>
> https://git.openjdk.org/jextract/pull/279
>
> So we should make sure to decide what code we want to generate (as 
> that PR will make function pointer support even more verbose).
>
> Cheers
> Maurizio
>
> On 07/04/2025 13:42, Duncan Gittins wrote:
>> I've been using jextract successfully for some time now and it has 
>> made it very easy to call into various Windows. Great work.
>>
>> One area which seems a bit verbose is the handling of the functional 
>> interface callbacks for upcalls. For example, the qsort compare 
>> callback is as follows (I've shortened the jextract version here):
>>
>> publicclassQsortCompareFunc {
>>
>> QsortCompareFunc() { }
>>
>> /** The function pointer signature, expressed as a functional 
>> interface */
>>
>> publicinterfaceFunction {
>>
>> intapply(MemorySegment _x0, MemorySegment _x1);
>>
>> }
>>
>> privatestaticfinalFunctionDescriptor $DESC= FunctionDescriptor.of(
>>
>> C_h.C_INT,
>>
>> C_h.C_POINTER,
>>
>> C_h.C_POINTER
>>
>> );
>>
>> /** The descriptor of this function pointer */
>>
>> publicstaticFunctionDescriptor descriptor() {
>>
>> return$DESC;
>>
>> }
>>
>> privatestaticfinalMethodHandle UP$MH= 
>> C_h.upcallHandle(QsortCompareFunc.Function.class, "apply", $DESC);
>>
>> /**
>>
>> * Allocates a new upcall stub, whose implementation is defined by 
>> {@code fi}.
>>
>> * The lifetime of the returned segment is managed by {@code arena}
>>
>> */
>>
>> publicstaticMemorySegment allocate(QsortCompareFunc.Function fi, 
>> Arena arena) {
>>
>> returnLinker.nativeLinker().upcallStub(UP$MH.bindTo(fi), $DESC, arena);
>>
>> }
>>
>> privatestaticfinalMethodHandle DOWN$MH= 
>> Linker.nativeLinker().downcallHandle($DESC);
>>
>> /**
>>
>> * Invoke the upcall stub {@code funcPtr}, with given parameters
>>
>> */
>>
>> publicstaticintinvoke(MemorySegment funcPtr,MemorySegment _x0, 
>> MemorySegment _x1) {
>>
>> try{
>>
>> return(int) DOWN$MH.invokeExact(funcPtr, _x0, _x1);
>>
>> } catch(Throwable ex$) {
>>
>> thrownewAssertionError("should not reach here", ex$);
>>
>> }
>>
>> }
>>
>> }
>>
>>
>> I made a few changes in jextract to make the callbacks generate 
>> interface rather than a class, adding a helper record Upcall to do 
>> most of the actions:
>>
>> publicinterfaceQsortCompareFunc {
>>
>> /** The function pointer signature, expressed as a functional 
>> interface */
>>
>> intapply(MemorySegment _x0, MemorySegment _x1);
>>
>> staticfinalC_h.Upcall<QsortCompareFunc> UPCALL= 
>> newC_h.Upcall<>(QsortCompareFunc.class, "apply", FunctionDescriptor.of(
>>
>> C_h.C_INT,
>>
>> C_h.C_POINTER,
>>
>> C_h.C_POINTER
>>
>> ));
>>
>> // A new upcall stub is allocated by:
>>
>> // MemorySegment funcPtr = QsortCompareFunc.UPCALL.allocate(fi,arena)
>>
>> /**
>>
>> * Invoke the upcall stub {@code funcPtr}, with given parameters
>>
>> */
>>
>> publicstaticintinvoke(MemorySegment funcPtr,MemorySegment _x0, 
>> MemorySegment _x1) {
>>
>> try{
>>
>> return(int) UPCALL.downcall().invokeExact(funcPtr, _x0, _x1);
>>
>> } catch(Throwable ex$) {
>>
>> thrownewAssertionError("should not reach here", ex$);
>>
>> }
>>
>> }
>>
>> }
>>
>>
>> This reduces the definition somewhat, as long as the top level header 
>> has a record to handle all callback interfaces:
>>
>> publicrecordUpcall<FunctionInterface>(FunctionDescriptor descriptor, 
>> MethodHandle upcall, MethodHandle downcall) {
>>
>> publicUpcall(Class<FunctionInterface> fi, String name, 
>> FunctionDescriptor descriptor) {
>>
>> this(descriptor, upcallHandle(fi, name, descriptor), 
>> Linker.nativeLinker().downcallHandle(descriptor));
>>
>> }
>>
>> /** Find Java method handle of interface for upcall from native ->Java */
>>
>> publicstaticMethodHandle upcallHandle(Class<?> fi, String name, 
>> FunctionDescriptor fdesc) {
>>
>> try{
>>
>> returnMethodHandles.lookup().findVirtual(fi, name, fdesc.toMethodType());
>>
>> } catch(ReflectiveOperationException ex) {
>>
>> thrownewAssertionError(ex);
>>
>> }
>>
>> }
>>
>> /// Allocate java stub for upcall
>>
>> publicMemorySegment allocate(FunctionInterface javaImpl, Arena scope) {
>>
>> returnLinker.nativeLinker().upcallStub(upcall.bindTo(javaImpl), 
>> descriptor, scope);
>>
>> }
>>
>> }
>>
>> Worth considering?
>>
>> Duncan
>>
>>
>>
>>
>>
​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20250411/86959431/attachment-0001.htm>