Win32 / OLE issues
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Fri Oct 29 15:15:25 UTC 2021
This should have been fixed. Sorry again for the delay.
Cheers
Maurizio
On 28/10/2021 22:06, Maurizio Cimadamore wrote:
> Thanks for the detailed report, I think I know where to look, and
> sorry for the late reply.
>
> This is a minimal reproducer which shows all the issues:
>
> ```
> typedef int*(*foo)(int*);
> ```
>
> Extracting this reveals two problems:
>
> * the type of the functional interface is wrong - e.g.
> (MemoryAddress)MemoryAddress, instead of (MemoryAddress)Addressable
> * as you noted, the arguments (and the return types) are missing some
> casts (from MemoryAddress to Addressable and back) which doesn't sit
> well with invokeExact
>
> I've filed this:
>
> https://bugs.openjdk.java.net/browse/JDK-8276136
>
> Thanks for your patience.
>
> Maurizio
>
>
>
> On 28/10/2021 21:05, Duncan Gittins wrote:
>> Apologies for the long email. I've refactored / inlined my Windows
>> OLE code
>> to a single class test case showing the bug in jextract.
>>
>> All this code sample does is load a Window COM object by the string
>> version
>> of its CLSID GUID, and retrieves a requested interface IID GUID (plus
>> any
>> other IIDs specified. This ought to work for any COM / IID interface,
>> but
>> it defaults to CLSID_ShellLink with IID_IShellLinkW / IID_Persist COM
>> interface lookup using only the IUnknown callbacks - QueryInterface /
>> AddRef / Release. Perhaps you could refactor into a test case for
>> jextract
>> in future Windows builds.
>>
>> The error is:
>>
>> java.lang.AssertionError: should not reach here
>> at
>> duncan.win.ole.IUnknownVtbl$Release.lambda$ofAddress$0(IUnknownVtbl.java:123)
>>
>> at duncan.panama.test.TestGuid$JUnknown.Release(Unknown Source)
>> at duncan.panama.test.TestGuid.main(Unknown Source)
>> at
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> at
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:76)
>>
>> at
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:51)
>>
>> at java.base/java.lang.reflect.Method.invoke(Method.java:569)
>> at duncan.launch.Launch.main(Unknown Source)
>> at duncan.launch.Launch.run(Unknown Source)
>> at duncan.launch.Launch.main(Unknown Source)
>> Caused by: java.lang.invoke.WrongMethodTypeException: expected
>> (NativeSymbol,Addressable)int but found (NativeSymbol,MemoryAddress)int
>> at
>> java.base/java.lang.invoke.Invokers.newWrongMethodTypeException(Invokers.java:523)
>>
>> at
>> java.base/java.lang.invoke.Invokers.checkExactType(Invokers.java:532)
>> at
>> duncan.win.ole.IUnknownVtbl$Release.lambda$ofAddress$0(IUnknownVtbl.java:121)
>>
>>
>> Class is below
>>
>> Kind regards
>>
>> Duncan
>>
>> import static duncan.win.ole.Ole32_h.S_OK;
>> import static jdk.incubator.foreign.ValueLayout.ADDRESS;
>> import static jdk.incubator.foreign.ValueLayout.JAVA_BYTE;
>>
>> import java.nio.charset.StandardCharsets;
>> import java.util.Arrays;
>> import java.util.HexFormat;
>> import java.util.concurrent.TimeUnit;
>> import java.util.function.Supplier;
>>
>> import org.junit.jupiter.api.Test;
>> import org.junit.jupiter.api.condition.EnabledOnOs;
>> import org.junit.jupiter.api.condition.OS;
>>
>> import duncan.win.ole.GUID;
>> import duncan.win.ole.IUnknown;
>> import duncan.win.ole.IUnknownVtbl;
>> import duncan.win.ole.Ole32_h;
>>
>> import jdk.incubator.foreign.MemoryAddress;
>> import jdk.incubator.foreign.MemorySegment;
>> import jdk.incubator.foreign.ResourceScope;
>> import jdk.incubator.foreign.SegmentAllocator;
>>
>> /**
>> * Test app to run through Windows COM IUnknown API calls.
>> * A JUNIT test runs main() - easily removed
>> *
>> * Usage:
>> java ... TestGuid GUID_CLSID GUID_IID+
>> *
>> * GUIDs are defined in Windows format eg.
>> "{00021401-0000-0000-C000-000000000046}"
>> *
>> * Note: have kept to Windows naming conventions when calling Windows
>> definitions.
>> *
>> * This instantiates the class with GUID_CLSID, and returns the
>> requested
>> IID interface.
>> * <p>If additional IIDs are provided these are requested
>> * <p>This requires jextract where files are:
>> * Ole32.h:
>> * #include <objbase.h>
>> *
>> * jextract -source -lole32 -t duncan.win.ole -d duncan.win\src Ole32.h
>> */
>> public class TestGuid
>> {
>> // Junit testcase
>> @EnabledOnOs(OS.WINDOWS)
>> @Test void coverageTestGuid() throws Exception
>> {
>> TestGuid.main();
>> }
>>
>> // This uses slimmed down version of my JUnknown, inlined here
>> // along with all the dependent calls on other my other win32
>> classes
>> // Not called IUnknown to avoid name classes with jextract
>> generated
>> code.
>> static class JUnknown
>> {
>> protected final MemoryAddress comObj;
>>
>> private final IUnknownVtbl.QueryInterface QueryInterface;
>> private final IUnknownVtbl.AddRef AddRef;
>> private final IUnknownVtbl.Release Release;
>>
>> /**
>> * Derived classes can use this after chaining to the protected
>> constructor
>> */
>> protected final MemorySegment vtable;
>> protected final SegmentAllocator allocator;
>>
>> /**
>> * Derived classes must not use this constructor as it
>> * only reserves only a small memory segment for the VTABLE
>> */
>> public JUnknown(ResourceScope scope, SegmentAllocator
>> allocator,
>> MemoryAddress comObj)
>> {
>> this(scope, allocator, comObj,
>> IUnknownVtbl.ofAddress(vtable(scope, comObj), scope));
>> }
>>
>> /**
>> * This version ensures a larger vTABLE can be read:
>> * A constructor for a derived class - say IShellLink
>> * would need to extract its own callbacks from vtable like
>> this:
>> * <code>
>> public class JShellLink extends JUnknown { ...
>>
>> public JShellLink(ResourceScope scope, SegmentAllocator
>> allocator, MemoryAddress comObj)
>> {
>> super(scope, allocator, comObj,
>> IShellLinkWVtbl.ofAddress(vtable(scope, comObj), scope));
>> GetPath = IShellLinkWVtbl.GetPath(vtable, scope);
>> SetPath = IShellLinkWVtbl.SetPath(vtable, scope);
>> SetArguments = IShellLinkWVtbl.SetArguments(vtable,
>> scope);
>> GetArguments = IShellLinkWVtbl.GetArguments(vtable,
>> scope);
>> GetDescription =
>> IShellLinkWVtbl.GetDescription(vtable,
>> scope);
>> GetIDList = IShellLinkWVtbl.GetIDList(vtable,
>> scope);
>> }
>> ...
>> }
>> * </code>
>> * @param scope
>> * @param allocator
>> * @param comObj
>> * @param vtable as provided by a call to
>> Ole32_h.IxxxxxxVtbl.ofAddressRestricted(vtableA(comObj))
>> */
>> protected JUnknown(ResourceScope scope, SegmentAllocator
>> allocator,
>> MemoryAddress comObj, MemorySegment vtable)
>> {
>> this.allocator = allocator;
>> this.comObj = comObj;
>> this.vtable = vtable;
>>
>> QueryInterface = IUnknownVtbl.QueryInterface(vtable,
>> scope);
>> AddRef = IUnknownVtbl.AddRef(vtable, scope);
>> Release = IUnknownVtbl.Release(vtable, scope);
>> }
>>
>> public String toString()
>> {
>> return
>> getClass().getSimpleName()+"["+toHex(comObj.toRawLongValue())+"]";
>> }
>>
>> /**
>> * Reads the vTABLE for this COM object, returning the address
>> * which must be dereferenced by the outermost derived class
>> with
>> the jextract
>> * call specific for the derived classes VTABLE:
>> * IClassName.ofAddressRestricted(vtable(comObj));
>> */
>> public static MemoryAddress vtable(ResourceScope scope,
>> MemoryAddress comObj)
>> {
>> // Only need to retrieve a pointer at index 0 so size of
>> one
>> pointer is OK:
>> MemorySegment memseg = IUnknown.ofAddress(comObj, scope);
>>
>> // The address of QueryInterface is the first entry in the
>> vtable:
>> MemoryAddress pVT = IUnknown.lpVtbl$get(memseg);
>>
>> // This is only the address of the vTable, see
>> IClassName.ofAddressRestricted(pVT);
>> return pVT;
>> }
>>
>> public int AddRef()
>> {
>> int refcount = AddRef.apply(comObj);
>> System.out.println(this+".AddRef => "+refcount);
>>
>> return refcount;
>> }
>>
>> /**
>> * IUnknown::Release method (unknwn.h)
>> *
>> https://docs.microsoft.com/en-us/windows/win32/api/unknwn/nf-unknwn-iunknown-release
>>
>> * ULONG Release()
>> */
>> public int Release()
>> {
>> int refcount = Release.apply(comObj);
>> System.out.println(this+".Release => "+refcount);
>>
>> return refcount;
>> }
>>
>> /**
>> *
>> https://docs.microsoft.com/en-us/windows/win32/api/unknwn/nf-unknwn-iunknown-queryinterface(refiid_void)
>>
>> * IUnknown::QueryInterface(REFIID,void) method (unknwn.h)
>> * HRESULT IUnknown::QueryInterface(REFIID iid, void** ppv);
>> * <P>
>> * https://osm.hpi.de/LV/Components04/VL5/MSDN/DrGUI-on-COM
>> */
>> public MemoryAddress QueryInterface(MemorySegment riid)
>> {
>> System.out.println(this+".QueryInterface
>> riid="+toHex(riid.toArray(JAVA_BYTE)));
>>
>> // This is allocated each time in case more than one
>> call made
>> to QueryInterface
>> MemorySegment queryRes = allocator.allocate(ADDRESS,
>> MemoryAddress.NULL);
>>
>> int hRes = QueryInterface.apply(comObj, riid.address(),
>> queryRes.address());
>>
>> MemoryAddress address = queryRes.get(ADDRESS, 0);
>> check0("QueryInterface", hRes);
>> System.out.println(" =>
>> IID="+toHex(address.toRawLongValue()));
>>
>> return address;
>> }
>> }
>>
>> public static MemorySegment toWideString(String s, SegmentAllocator
>> allocator)
>> {
>> return allocator.allocateArray(JAVA_BYTE,
>> (s+"\0").getBytes(StandardCharsets.UTF_16LE));
>> }
>> public static void check0(String func, int hRes)
>> {
>> checkThat(hRes == S_OK(), () -> "Error: "+func+" NOT OK,
>> result was
>> "+hRes+" / 0x"+Integer.toHexString(hRes));
>> System.out.println(func+" OK result => "+hRes);
>> }
>>
>> public static void checkThat(boolean condition, Supplier<String>
>> error)
>> {
>> if (!condition)
>> {
>> String message = error.get();
>> System.err.println(message);
>> throw new RuntimeException(message);
>> }
>> }
>>
>> public static MemorySegment CLSIDFromString(SegmentAllocator
>> allocator,
>> String guid)
>> {
>> System.out.println("CLSIDFromString clsid="+guid);
>> // Falls through to do the actual conversion:
>> MemorySegment pGUID = GUID.allocate(allocator);
>>
>> MemorySegment wide = toWideString(guid, allocator);
>>
>> int hRes = Ole32_h.CLSIDFromString(wide, pGUID);
>> check0("CLSIDFromString", hRes);
>> System.out.println(" =>
>> GUID="+toHex(pGUID.toArray(JAVA_BYTE)));
>>
>> return pGUID;
>> }
>> public static String toHex(long value)
>> {
>> return "0x"+Long.toHexString(value).toUpperCase();
>> }
>> private static String toHex(byte[] arr) {
>> HexFormat hex = HexFormat.ofDelimiter(",
>> ").withPrefix("0x").withUpperCase();
>> return "new byte["+arr.length+"] {"+hex.formatHex(arr)+"}";
>> }
>> public static MemoryAddress CoCreateInstance(SegmentAllocator
>> allocator, MemorySegment rclsid, int dwClsContext, MemorySegment riid)
>> {
>> MemoryAddress pUnkOuter = MemoryAddress.NULL;
>> MemorySegment ptrComObj = allocator.allocate(ADDRESS,
>> MemoryAddress.NULL);
>> // https://www.purebasic.fr/english/viewtopic.php?f=13&t=45583
>>
>> System.out.println("CoCreateInstance
>> rclsid="+toHex(rclsid.toArray(JAVA_BYTE)));
>> System.out.println("CoCreateInstance
>> pUnkOuter="+pUnkOuter.address());
>> System.out.println("CoCreateInstance
>> dwClsContext="+dwClsContext);
>> System.out.println("CoCreateInstance
>> riid="+toHex(riid.toArray(JAVA_BYTE)));
>>
>> int hRes = Ole32_h.CoCreateInstance(rclsid, pUnkOuter,
>> dwClsContext, riid, ptrComObj);
>> check0("CoCreateInstance", hRes);
>> MemoryAddress comObj = ptrComObj.get(ADDRESS, 0);
>> System.out.println("CoCreateInstance =>
>> address="+toHex(comObj.toRawLongValue()));
>>
>> return comObj;
>> }
>> public static void checkResult(String func, int hRes, int ...
>> okcodes)
>> {
>> for (int ok : okcodes)
>> {
>> if (ok == hRes)
>> {
>> System.out.println(func+" OK result => "+hRes+"
>> ok="+Arrays.toString(okcodes));
>> return;
>> }
>> }
>> checkThat(false, () -> "Error: "+func+" NOT OK, result code
>> "+hRes+" / 0x"+Integer.toHexString(hRes)+"
>> ok="+Arrays.toString(okcodes));
>> }
>> public static AutoCloseable CoInitialize()
>> {
>> int hRes = Ole32_h.CoInitialize(MemoryAddress.NULL);
>> checkResult("CoInitialize", hRes, Ole32_h.S_OK(),
>> Ole32_h.S_FALSE());
>>
>> return Ole32_h::CoUninitialize;
>> }
>>
>> public static void main(String ... args) throws Exception
>> {
>> final long t0 = System.nanoTime();
>>
>> // Default run instantiates COM object CLSID_ShellLink and
>> asks for
>> its IShellLinkW and IID_Persist interfaces
>> String[] defaults = {
>> /*GUID_CLSID_ShellLink_str*/
>> "{00021401-0000-0000-C000-000000000046}",
>> /*GUID_IID_IShellLinkW_str*/
>> "{000214F9-0000-0000-C000-000000000046}",
>> /*GUID_IID_Persist_str*/
>> "{0000010B-0000-0000-C000-000000000046}",
>> };
>>
>> if (args.length < 2) args = defaults;
>>
>> String clsidStr = args[0];
>> String iidStr = args[1];
>> System.out.println("lookup GUID CLSID "+clsidStr+" IID
>> "+iidStr);
>>
>> try(ResourceScope scope = ResourceScope.newConfinedScope())
>> {
>> SegmentAllocator allocator =
>> SegmentAllocator.newNativeArena(scope);
>>
>> // Lookup GUIDs for class and interfaces
>> MemorySegment clsid = CLSIDFromString(allocator,
>> clsidStr);
>> MemorySegment iid = CLSIDFromString(allocator, iidStr);
>>
>> // Setup COM:
>> try(var autoC = CoInitialize())
>> {
>> // Get a pointer to the COM instance.
>> MemoryAddress comObj = CoCreateInstance(allocator,
>> clsid,
>> Ole32_h.CLSCTX_INPROC_SERVER(), iid);
>>
>> // JUnknown ignores the complete vtable, only
>> initialises
>> callbacks for IUnknown interface
>> JUnknown iUnknown = new JUnknown(scope, allocator,
>> comObj);
>> try
>> {
>> // test add/release on this iUnknown
>> int ar = iUnknown.AddRef();
>> int re = iUnknown.Release();
>> checkResult("AddRef/Release()", ar, re+1);
>>
>> // Access other interfaces via the first one:
>> for (int ii = 2; ii < args.length; ii++)
>> {
>> // Use Query interface on another interface
>> from
>> the CLSID
>> MemorySegment iidextra =
>> CLSIDFromString(allocator,
>> args[ii]);
>>
>> // Query CLS for this other interface
>> definition
>> MemoryAddress iOtherUnknown =
>> iUnknown.QueryInterface(iidextra);
>> JUnknown iOther = new JUnknown(scope,
>> allocator,
>> iOtherUnknown);
>> try
>> {
>> // test add/release on this other
>> iUnknown (NB
>> same instance but with different vtable)
>> int add = iOther.AddRef();
>> checkResult("AddRef()", add, 3);
>> int rel = iOther.Release();
>> checkResult("Release()", rel, 2);
>> }
>> finally
>> {
>> // Signal end of use of iOther
>> // Note that the ref count includes the
>> value
>> on iUnknown
>> int refC = iOther.Release();
>> checkResult("Release()", refC, 1);
>> }
>> }
>> }
>> finally
>> {
>> // Signal end of use of iUnknown
>> int refC = iUnknown.Release();
>> // Note that the ref count should now be zero
>> checkResult("Release()", refC, 0);
>> }
>> }
>> final long now = System.nanoTime();
>> System.out.println("Ended
>> ms="+TimeUnit.NANOSECONDS.toMillis(now - t0)+" "+clsidStr);
>> }
>> }
>> }
>>
>> On Mon, 27 Sept 2021 at 16:45, Duncan Gittins <duncan.gittins at gmail.com>
>> wrote:
>>
>>> The jextract parameters which generate the broken Windows OLE API
>>> code I
>>> outlined below is:
>>>
>>> jextract -source -lole32 -t duncan.win.ole -d
>>> source\duncan.win\java
>>> headers\Ole32.h
>>>
>>> where headers\Ole32.h contains:
>>>
>>> #include <objbase.h>
>>>
>>> A temporary workaround for this jextract issue is to edit
>>> FunctionalInterfaceBuilder.java / emitFunctionalFactoryForPointer() to
>>> insert the (Addressable) casts for MemoryAddress parameters:
>>>
>>> ---
>>> a/src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/FunctionalInterfaceBuilder.java
>>>
>>> +++
>>> b/src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/FunctionalInterfaceBuilder.java
>>>
>>> @@ -123,7 +123,7 @@ public class FunctionalInterfaceBuilder extends
>>> ClassSourceBuilder {
>>> append(mhConstant.accessExpression() +
>>> ".invokeExact((Addressable)addr");
>>> if (fiType.parameterCount() > 0) {
>>> String params = IntStream.range(0,
>>> fiType.parameterCount())
>>> - .mapToObj(i -> "x" + i)
>>> + .mapToObj(i ->
>>> (fiType.parameterType(i).getName().endsWith(".MemoryAddress") ?
>>> "(Addressable)":"")+"x" + i)
>>> .collect(Collectors.joining(", "));
>>> append(", " + params);
>>> }
>>>
>>> this fixes the ofAddress callbacks, for example see
>>> IPersistFileVtbl.java:
>>>
>>> public interface Load {
>>> ....
>>> static Load ofAddress(MemoryAddress addr) {
>>> return (jdk.incubator.foreign.MemoryAddress x0,
>>> jdk.incubator.foreign.MemoryAddress x1, int x2) -> {
>>> try {
>>> // WAS: return
>>> (int)IPersistFileVtbl.Load$MH.invokeExact((Addressable)addr, x0, x1,
>>> x2);
>>> return
>>> (int)IPersistFileVtbl.Load$MH.invokeExact((Addressable)addr,
>>> (Addressable)x0, (Addressable)x1, x2);
>>> } catch (Throwable ex$) {
>>> throw new AssertionError("should not reach
>>> here", ex$);
>>> }
>>> };
>>> }
>>>
>>> Kind regards
>>>
>>> Duncan
>>>
>>>
>>> On Fri, 24 Sept 2021 at 16:36, Duncan Gittins
>>> <duncan.gittins at gmail.com>
>>> wrote:
>>>
>>>> I've pulled latest panama-foreign which has the changes outlined in
>>>>
>>>> https://inside.java/2021/09/16/finalizing-the-foreign-apis/
>>>>
>>>> I've a few problems to resolve to match up, one is related to jextract
>>>> which generates interfaces for Windows OLE APIs. The invokeExact
>>>> params
>>>> are missing some (Addressable) casts (I think?) eg this is IUnknown
>>>> Release:
>>>>
>>>> public interface Release {
>>>>
>>>> int apply(jdk.incubator.foreign.MemoryAddress x0);
>>>> static CLinker.UpcallStub allocate(Release fi) {
>>>> return RuntimeHelper.upcallStub(Release.class, fi,
>>>> IUnknownVtbl.Release$FUNC,
>>>> "(Ljdk/incubator/foreign/MemoryAddress;)I");
>>>> }
>>>> static CLinker.UpcallStub allocate(Release fi, ResourceScope
>>>> scope) {
>>>> return RuntimeHelper.upcallStub(Release.class, fi,
>>>> IUnknownVtbl.Release$FUNC, "(Ljdk/incubator/foreign/MemoryAddress;)I",
>>>> scope);
>>>> }
>>>> static Release ofAddress(MemoryAddress addr) {
>>>> return (jdk.incubator.foreign.MemoryAddress x0) -> {
>>>> try {
>>>> return
>>>> (int)IUnknownVtbl.Release$MH.invokeExact((Addressable)addr, x0);
>>>> } catch (Throwable ex$) {
>>>> throw new AssertionError("should not reach here",
>>>> ex$);
>>>> }
>>>> };
>>>> }
>>>> }
>>>>
>>>> Stack traces show
>>>>
>>>> Caused by: java.lang.invoke.WrongMethodTypeException: expected
>>>> (Addressable,Addressable)int but found (Addressable,MemoryAddress)int
>>>>
>>>> Kind regards
>>>>
>>>> Duncan
>>>>
>>>>
>>>>
More information about the panama-dev
mailing list