Windows OLE/COM with jextract code generation

[Duncan Gittins]

On 17/02/2021 12:58, Jorn Vernee wrote:
> Hi Duncan,
>
> Thanks for bringing this up. Virtual calls are an interesting 
> use-case, and a while ago we added support for linking virtual calls 
> to CLinker, where you get back a method handle that takes an 
> additional leading Addressable parameter that represents the function 
> that should be called.
>
> On the jextract front though, there is a relatively low-hanging fruit 
> that we could also grab for this. IUnknownVtbl is essentially a struct 
> with a bunch of function pointers. A simplified example would be:
>
>     struct Foo {
>         void (*cb)(int);
>     };
>
> Now maybe you get one of these structs back from a native call, and 
> then you want to call 'cb'.
>
> Jextract currently generates getters and setters for those kinds of 
> function pointer fields, which would allow you to manually get the 
> function address and then manually link a MethodHandle to call it, as 
> you've found out. But, we could, in the case of function pointer 
> types, also add another kind of accessor to call the function 
> directly, and wire up the right MethodHandle behind the scenes.
>
> So, your case could theoretically be turned from:
>
>     Addressable vtFuncPtr = (Addressable) 
> Ole32_h.IUnknownVtbl.QueryInterface$VH().get(vtable);
>     MethodType mt= MethodType.methodType(int.class, 
> MemoryAddress.class, MemoryAddress.class, MemoryAddress.class);
>     FunctionDescriptor fd = FunctionDescriptor.of(CLinker.C_LONG, 
> CLinker.C_POINTER, CLinker.C_POINTER, CLinker.C_POINTER);
>     MethodHandle queryInterfaceMH = 
> CLinker.getInstance().downcallHandle(vtFuncPtr, mt, fd);
>
> Into something like:
>
>     MemoryAddress hresult = Ole32_h.IUnknownVtbl.QueryInterface(vtable);
>
That extra code generation would be helpful. I used the method handle 
from vtable in above code to create version of the interface added by 
jextract:

       Ole32_h.IUnknownVtbl.QueryInterface queryInterface = 
(This,riid,ppvObject) -> {
             var mh$ = queryInterfaceMH;
             try {
                 return (int)mh$.invokeExact(This, riid, ppvObject);
             } catch (Throwable ex$) {
                 throw new AssertionError("should not reach here", ex$);
             }
         };

... and then call QI was simple:

         int hRes = queryInterface.apply(comObj.address(), 
riid.address(), queryRes.address());

Duncan

> Maurizio has filed an issue for this [1].
>
> Thanks,
> Jorn
>
> [1] : https://bugs.openjdk.java.net/browse/JDK-8261906
>
> On 16/02/2021 19:13, Duncan Gittins wrote:
>> This may be outside the scope of jextract (too Windows specific), or 
>> a dumb question (then I'll send it to the usual place for dumb 
>> questions: stackoverflow).
>>
>> I'm converting my handwritten Panama Java application over to use the 
>> corresponding data structures generated by jextract, but can't find 
>> definitions which could simplify the MethodHandle lookups for 
>> referencing Windows OLE/COM API calls. The jextract output of Ole32.h 
>> (which contains only "#include <objbase.h>") used is generated from:
>>
>>      set "WINKIT=c:\Program Files (x86)\Windows 
>> Kits\10\Include\10.0.18362.0"
>>      jextract -source -lole32    -t duncan.win -d java -I 
>> "%WINKIT%\um" Ole32.h
>>
>> Retrieving any COM object IUnknown is easy with appropriate values 
>> for class+iid guids as follows:
>>
>>      MemoryAddress comObj = 
>> Ole32_h.CoCreateInstance(rclsid.address(), pUnkOuter, dwClsContext, 
>> riid.address(), ptrComObj.address());
>>
>> ... and read the COM object vTable by following IUnknown.lpVtbl 
>> pointer to IUnknownVTbl:
>>
>>     MemorySegment memseg = Ole32_h.IUnknown.ofAddressRestricted(comObj);
>>     MemoryAddress pVTableAddr = Ole32_h.IUnknown.lpVtbl$get(memseg);
>>     MemorySegment vtable = 
>> Ole32_h.IUnknownVtbl.ofAddressRestricted(pVTableAddr);
>>
>> At this point I need downcallHandle for every COM method pointer in 
>> vtable - QueryInterface / AddRef... I have this for "QueryInterface" 
>> but is there a way to avoid my hardcoded mt/fd declarations for all 
>> each method handle?
>>
>>     Addressable vtFuncPtr = (Addressable) 
>> Ole32_h.IUnknownVtbl.QueryInterface$VH().get(vtable);
>>     MethodType mt= MethodType.methodType(int.class, 
>> MemoryAddress.class, MemoryAddress.class, MemoryAddress.class);
>>     FunctionDescriptor fd = FunctionDescriptor.of(CLinker.C_LONG, 
>> CLinker.C_POINTER, CLinker.C_POINTER, CLinker.C_POINTER);
>>     MethodHandle queryInterfaceMH = 
>> CLinker.getInstance().downcallHandle(vtFuncPtr, mt, fd);
>>
>> The interface IUnknownVtbl.QueryInterface has support for upcalls (- 
>> so maybe I could implement QueryInterface for a COM object in Java), 
>> and has "fd" internally as xxx_constants.QueryInterface $FUNC, but no 
>> equivalent to "mt" or downcalls of this specific comObj - say as 
>> "IUnknownVtbl.QueryInterface$MH(vtFuncPtr)".
>>
>> Kind regards
>>
>> Duncan Gittins