jextract generation for functional interfaces

Fri Apr 11 16:09:48 UTC 2025

Thanks for looking at this. I did this out of interest really - I use a
monster Windows jar for prototyping, this Upcall change cut 5MB from a 31MB
jar. The only client code changes are to add ".UPCALL" or remove
".Function" so are easy to apply (not sure where your "$func" part comes
from).

However I agree the use of StableValue API will be best time to reconsider
jextract code structure especially for the holder classes.

Kind regards

Duncan

On Fri, 11 Apr 2025 at 10:24, Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

> Hi Duncan,
> we took a better look at this, and noticed that, in reality, there’s not
> much we can do at the moment about this, for two reasons.
>
> First, the idea behind your approach was to move some of this shared
> functionality (such as the “allocate” method) to the Upcall class. But this
> changes the client code in ways that IMHO are not optimal — e.g.
>
> compar.allocate(...)
>
> becomes
>
> compar$func.UPCALL.allocate(...)
>
> So I think that, to make this work, we would have to replicate the
> allocate method (and possibly other accessors) in the toplevel class —
> which reduces the gain.
>
> A better strategy would be one where the toplevel class “extends” from
> Upcall — then all the various methods would be inherited — and clients
> would be good again. But if we do that, we lose “constant-ness” of the
> various method handles involved, which will make methods like “invoke”
> slower.
>
> Another difference between the two approaches is that you have merged the
> functional pointer wrapper with the functional interface. This is a move we
> have considered when we transitioned to this new code generation strategy,
> but we ended up rejecting because there’s no way to make interface fields
> non-public. Which is why we opted for an enclosing *class* with a nested
> interface.
>
> For these reasons, I believe that, for the time being, it’s better to keep
> the generated code as is. It is possible that the StableValue API [1] will
> allow us to change this — at some point we considered making fields of type
> StableValue implicitly trusted finals. While we have removed that trick in
> the current PR, something like that might allow for the code generation
> strategy I was alluding to earlier — where the function pointer class
> extends from some Upcall class that has a pair of stable value fields, one
> for the upcall MH, the other for the downcall MH. (In general, we’re
> confident that StableValue should result in jextract bindings become much
> more compact, as SV will essentially remove the need for all the holder
> classes that are spread throughout the generated code).
>
> Thanks for the exploration!
> Maurizio
>
> [1] - https://github.com/openjdk/jdk/pull/23972
>
> On 07/04/2025 16:09, Maurizio Cimadamore wrote:
>
> I like this. The use of record components should probably preserve the
> fact that the upcall method handle is treated as a constant.
>
> This seems related to this PR under review:
>
> https://git.openjdk.org/jextract/pull/279
>
> So we should make sure to decide what code we want to generate (as that PR
> will make function pointer support even more verbose).
>
> Cheers
> Maurizio
> On 07/04/2025 13:42, Duncan Gittins wrote:
>
> I've been using jextract successfully for some time now and it has made it
> very easy to call into various Windows. Great work.
>
> One area which seems a bit verbose is the handling of the functional
> interface callbacks for upcalls. For example, the qsort compare callback is
> as follows (I've shortened the jextract version here):
>
> public class QsortCompareFunc {
>
> QsortCompareFunc() { }
>
> /** The function pointer signature, expressed as a functional interface */
>
> public interface Function {
>
> int apply(MemorySegment _x0, MemorySegment _x1);
>
> }
>
> private static final FunctionDescriptor $DESC = FunctionDescriptor.of(
>
> C_h.C_INT,
>
> C_h.C_POINTER,
>
> C_h.C_POINTER
>
> );
>
> /** The descriptor of this function pointer */
>
> public static FunctionDescriptor descriptor() {
>
> return $DESC;
>
> }
>
> private static final MethodHandle UP$MH =
> C_h.upcallHandle(QsortCompareFunc.Function.class, "apply", $DESC);
>
> /**
>
> * Allocates a new upcall stub, whose implementation is defined by {@code
> fi}.
>
> * The lifetime of the returned segment is managed by {@code arena}
>
> */
>
> public static MemorySegment allocate(QsortCompareFunc.Function fi, Arena
> arena) {
>
> return Linker.nativeLinker().upcallStub(UP$MH.bindTo(fi), $DESC, arena);
>
> }
>
> private static final MethodHandle DOWN$MH = Linker.nativeLinker
> ().downcallHandle($DESC);
>
> /**
>
> * Invoke the upcall stub {@code funcPtr}, with given parameters
>
> */
>
> public static int invoke(MemorySegment funcPtr,MemorySegment _x0,
> MemorySegment _x1) {
>
> try {
>
> return (int) DOWN$MH.invokeExact(funcPtr, _x0, _x1);
>
> } catch (Throwable ex$) {
>
> throw new AssertionError("should not reach here", ex$);
>
> }
>
> }
>
> }
>
> I made a few changes in jextract to make the callbacks generate interface
> rather than a class, adding a helper record Upcall to do most of the
> actions:
>
> public interface QsortCompareFunc {
>
> /** The function pointer signature, expressed as a functional interface */
>
> int apply(MemorySegment _x0, MemorySegment _x1);
>
> static final C_h.Upcall<QsortCompareFunc> UPCALL = new
> C_h.Upcall<>(QsortCompareFunc.class, "apply", FunctionDescriptor.of(
>
> C_h.C_INT,
>
> C_h.C_POINTER,
>
> C_h.C_POINTER
>
> ));
>
> // A new upcall stub is allocated by:
>
> // MemorySegment funcPtr = QsortCompareFunc.UPCALL.allocate(fi,arena)
>
> /**
>
> * Invoke the upcall stub {@code funcPtr}, with given parameters
>
> */
>
> public static int invoke(MemorySegment funcPtr,MemorySegment _x0,
> MemorySegment _x1) {
>
> try {
>
> return (int) UPCALL.downcall().invokeExact(funcPtr, _x0, _x1);
>
> } catch (Throwable ex$) {
>
> throw new AssertionError("should not reach here", ex$);
>
> }
>
> }
>
> }
>
> This reduces the definition somewhat, as long as the top level header has
> a record to handle all callback interfaces:
>
> public record Upcall<FunctionInterface>(FunctionDescriptor descriptor,
> MethodHandle upcall, MethodHandle downcall) {
>
> public Upcall(Class<FunctionInterface> fi, String name,
> FunctionDescriptor descriptor) {
>
> this(descriptor, upcallHandle(fi, name, descriptor), Linker.nativeLinker
> ().downcallHandle(descriptor));
>
> }
>
> /** Find Java method handle of interface for upcall from native -> Java */
>
> public static MethodHandle upcallHandle(Class<?> fi, String name,
> FunctionDescriptor fdesc) {
>
> try {
>
> return MethodHandles.lookup().findVirtual(fi, name, fdesc.toMethodType());
>
> } catch (ReflectiveOperationException ex) {
>
> throw new AssertionError(ex);
>
> }
>
> }
>
> /// Allocate java stub for upcall
>
> public MemorySegment allocate(FunctionInterface javaImpl, Arena scope) {
>
> return Linker.nativeLinker().upcallStub(upcall.bindTo(javaImpl),
> descriptor, scope);
>
> }
>
> }
>
> Worth considering?
>
> Duncan
>
>
>
>
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20250411/0c73cfae/attachment-0001.htm>