Windows OLE/COM with jextract code generation

[Duncan Gittins]

Thanks Maurizio.

I've built my OLE/COM based examples with new calls, and they all run as 
before. The code is much neater with one line lookups, now all virtual 
functions come from jextract. Example for QueryInterface:

      IUnknownVtbl.QueryInterface qi = 
IUnknownVtbl.QueryInterface.ofAddressRestricted(IUnknownVtbl.QueryInterface$get(vtable));

Is there a performance / other reason behind why you implemented 
ofAddressRestricted(addr) as a new anonymous class rather than as a 
lambda? This is the generated code in IUnknownVtbl.QueryInterface:

             static QueryInterface ofAddressRestricted(MemoryAddress addr) {
                 return new QueryInterface() {
                     public int 
apply(jdk.incubator.foreign.MemoryAddress x0, 
jdk.incubator.foreign.MemoryAddress x1, 
jdk.incubator.foreign.MemoryAddress x2) {
                         try {
                             return 
(int)constants$2.QueryInterface$MH.invokeExact((Addressable)addr, x0, 
x1, x2);
                         } catch (Throwable ex$) {
                             throw new AssertionError("should not reach 
here", ex$);
                         }
                     }
                 };
             }

A lambda as below does same action but avoids extra .class files (in my 
case 330 x ~1,200 bytes) in the compiled output:

            static QueryInterface ofAddressRestricted(MemoryAddress addr) {
                 return (jdk.incubator.foreign.MemoryAddress x0, 
jdk.incubator.foreign.MemoryAddress x1, 
jdk.incubator.foreign.MemoryAddress x2) -> {
                         try {
                             return 
(int)constants$73.QueryInterface$MH.invokeExact((Addressable)addr, x0, 
x1, x2);
                         } catch (Throwable ex$) {
                             throw new AssertionError("should not reach 
here", ex$);
                         }
                 };
             }

This was the first time I pulled from the "--release 17" Panama jextract 
codebase and it caused a compile issue - related to upcallStub for 
WNDENUMPROC to enumerate Windows handles into a Java instance method. 
I'll respond separately with the example if I can't resolve what the 
change is.

Kind regards

Duncan

On 05/03/2021 21:44, Maurizio Cimadamore wrote:
> Hi Duncan,
> I've just integrated a fix for this:
>
> https://github.com/openjdk/panama-foreign/pull/456
>
> Now, if you have a global variable, or a struct field whose type is a 
> function pointer type, an extra accessor will be generated which 
> returns an instance of the corresponding functional interface.
>
> Let us know if this helps (we had a couple of possible strategies to 
> go through, described in the PR - in the end we opted for the 
> functional interface accessor).
>
> Cheers
> Maurizio
>
>
> On 17/02/2021 17:59, Duncan Gittins wrote:
>> On 17/02/2021 12:58, Jorn Vernee wrote:
>>> Hi Duncan,
>>>
>>> Thanks for bringing this up. Virtual calls are an interesting 
>>> use-case, and a while ago we added support for linking virtual calls 
>>> to CLinker, where you get back a method handle that takes an 
>>> additional leading Addressable parameter that represents the 
>>> function that should be called.
>>>
>>> On the jextract front though, there is a relatively low-hanging 
>>> fruit that we could also grab for this. IUnknownVtbl is essentially 
>>> a struct with a bunch of function pointers. A simplified example 
>>> would be:
>>>
>>>     struct Foo {
>>>         void (*cb)(int);
>>>     };
>>>
>>> Now maybe you get one of these structs back from a native call, and 
>>> then you want to call 'cb'.
>>>
>>> Jextract currently generates getters and setters for those kinds of 
>>> function pointer fields, which would allow you to manually get the 
>>> function address and then manually link a MethodHandle to call it, 
>>> as you've found out. But, we could, in the case of function pointer 
>>> types, also add another kind of accessor to call the function 
>>> directly, and wire up the right MethodHandle behind the scenes.
>>>
>>> So, your case could theoretically be turned from:
>>>
>>>     Addressable vtFuncPtr = (Addressable) 
>>> Ole32_h.IUnknownVtbl.QueryInterface$VH().get(vtable);
>>>     MethodType mt= MethodType.methodType(int.class, 
>>> MemoryAddress.class, MemoryAddress.class, MemoryAddress.class);
>>>     FunctionDescriptor fd = FunctionDescriptor.of(CLinker.C_LONG, 
>>> CLinker.C_POINTER, CLinker.C_POINTER, CLinker.C_POINTER);
>>>     MethodHandle queryInterfaceMH = 
>>> CLinker.getInstance().downcallHandle(vtFuncPtr, mt, fd);
>>>
>>> Into something like:
>>>
>>>     MemoryAddress hresult = 
>>> Ole32_h.IUnknownVtbl.QueryInterface(vtable);
>>>
>> That extra code generation would be helpful. I used the method handle 
>> from vtable in above code to create version of the interface added by 
>> jextract:
>>
>>       Ole32_h.IUnknownVtbl.QueryInterface queryInterface = 
>> (This,riid,ppvObject) -> {
>>             var mh$ = queryInterfaceMH;
>>             try {
>>                 return (int)mh$.invokeExact(This, riid, ppvObject);
>>             } catch (Throwable ex$) {
>>                 throw new AssertionError("should not reach here", ex$);
>>             }
>>         };
>>
>> ... and then call QI was simple:
>>
>>         int hRes = queryInterface.apply(comObj.address(), 
>> riid.address(), queryRes.address());
>>
>> Duncan
>>
>>> Maurizio has filed an issue for this [1].
>>>
>>> Thanks,
>>> Jorn
>>>
>>> [1] : https://bugs.openjdk.java.net/browse/JDK-8261906
>>>
>>> On 16/02/2021 19:13, Duncan Gittins wrote:
>>>> This may be outside the scope of jextract (too Windows specific), 
>>>> or a dumb question (then I'll send it to the usual place for dumb 
>>>> questions: stackoverflow).
>>>>
>>>> I'm converting my handwritten Panama Java application over to use 
>>>> the corresponding data structures generated by jextract, but can't 
>>>> find definitions which could simplify the MethodHandle lookups for 
>>>> referencing Windows OLE/COM API calls. The jextract output of 
>>>> Ole32.h (which contains only "#include <objbase.h>") used is 
>>>> generated from:
>>>>
>>>>      set "WINKIT=c:\Program Files (x86)\Windows 
>>>> Kits\10\Include\10.0.18362.0"
>>>>      jextract -source -lole32    -t duncan.win -d java -I 
>>>> "%WINKIT%\um" Ole32.h
>>>>
>>>> Retrieving any COM object IUnknown is easy with appropriate values 
>>>> for class+iid guids as follows:
>>>>
>>>>      MemoryAddress comObj = 
>>>> Ole32_h.CoCreateInstance(rclsid.address(), pUnkOuter, dwClsContext, 
>>>> riid.address(), ptrComObj.address());
>>>>
>>>> ... and read the COM object vTable by following IUnknown.lpVtbl 
>>>> pointer to IUnknownVTbl:
>>>>
>>>>     MemorySegment memseg = 
>>>> Ole32_h.IUnknown.ofAddressRestricted(comObj);
>>>>     MemoryAddress pVTableAddr = Ole32_h.IUnknown.lpVtbl$get(memseg);
>>>>     MemorySegment vtable = 
>>>> Ole32_h.IUnknownVtbl.ofAddressRestricted(pVTableAddr);
>>>>
>>>> At this point I need downcallHandle for every COM method pointer in 
>>>> vtable - QueryInterface / AddRef... I have this for 
>>>> "QueryInterface" but is there a way to avoid my hardcoded mt/fd 
>>>> declarations for all each method handle?
>>>>
>>>>     Addressable vtFuncPtr = (Addressable) 
>>>> Ole32_h.IUnknownVtbl.QueryInterface$VH().get(vtable);
>>>>     MethodType mt= MethodType.methodType(int.class, 
>>>> MemoryAddress.class, MemoryAddress.class, MemoryAddress.class);
>>>>     FunctionDescriptor fd = FunctionDescriptor.of(CLinker.C_LONG, 
>>>> CLinker.C_POINTER, CLinker.C_POINTER, CLinker.C_POINTER);
>>>>     MethodHandle queryInterfaceMH = 
>>>> CLinker.getInstance().downcallHandle(vtFuncPtr, mt, fd);
>>>>
>>>> The interface IUnknownVtbl.QueryInterface has support for upcalls 
>>>> (- so maybe I could implement QueryInterface for a COM object in 
>>>> Java), and has "fd" internally as xxx_constants.QueryInterface 
>>>> $FUNC, but no equivalent to "mt" or downcalls of this specific 
>>>> comObj - say as "IUnknownVtbl.QueryInterface$MH(vtFuncPtr)".
>>>>
>>>> Kind regards
>>>>
>>>> Duncan Gittins
>>
>>