Win32 / OLE issues
Duncan Gittins
duncan.gittins at gmail.com
Fri Oct 29 18:28:13 UTC 2021
Thanks, those changes have worked
Kind regards
Duncan
On Fri, 29 Oct 2021 at 16:15, Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:
> This should have been fixed. Sorry again for the delay.
>
> Cheers
> Maurizio
>
> On 28/10/2021 22:06, Maurizio Cimadamore wrote:
> > Thanks for the detailed report, I think I know where to look, and
> > sorry for the late reply.
> >
> > This is a minimal reproducer which shows all the issues:
> >
> > ```
> > typedef int*(*foo)(int*);
> > ```
> >
> > Extracting this reveals two problems:
> >
> > * the type of the functional interface is wrong - e.g.
> > (MemoryAddress)MemoryAddress, instead of (MemoryAddress)Addressable
> > * as you noted, the arguments (and the return types) are missing some
> > casts (from MemoryAddress to Addressable and back) which doesn't sit
> > well with invokeExact
> >
> > I've filed this:
> >
> > https://bugs.openjdk.java.net/browse/JDK-8276136
> >
> > Thanks for your patience.
> >
> > Maurizio
> >
> >
> >
> > On 28/10/2021 21:05, Duncan Gittins wrote:
> >> Apologies for the long email. I've refactored / inlined my Windows
> >> OLE code
> >> to a single class test case showing the bug in jextract.
> >>
> >> All this code sample does is load a Window COM object by the string
> >> version
> >> of its CLSID GUID, and retrieves a requested interface IID GUID (plus
> >> any
> >> other IIDs specified. This ought to work for any COM / IID interface,
> >> but
> >> it defaults to CLSID_ShellLink with IID_IShellLinkW / IID_Persist COM
> >> interface lookup using only the IUnknown callbacks - QueryInterface /
> >> AddRef / Release. Perhaps you could refactor into a test case for
> >> jextract
> >> in future Windows builds.
> >>
> >> The error is:
> >>
> >> java.lang.AssertionError: should not reach here
> >> at
> >>
> duncan.win.ole.IUnknownVtbl$Release.lambda$ofAddress$0(IUnknownVtbl.java:123)
>
> >>
> >> at duncan.panama.test.TestGuid$JUnknown.Release(Unknown Source)
> >> at duncan.panama.test.TestGuid.main(Unknown Source)
> >> at
> >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> Method)
> >> at
> >>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:76)
>
> >>
> >> at
> >>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:51)
>
> >>
> >> at java.base/java.lang.reflect.Method.invoke(Method.java:569)
> >> at duncan.launch.Launch.main(Unknown Source)
> >> at duncan.launch.Launch.run(Unknown Source)
> >> at duncan.launch.Launch.main(Unknown Source)
> >> Caused by: java.lang.invoke.WrongMethodTypeException: expected
> >> (NativeSymbol,Addressable)int but found (NativeSymbol,MemoryAddress)int
> >> at
> >>
> java.base/java.lang.invoke.Invokers.newWrongMethodTypeException(Invokers.java:523)
>
> >>
> >> at
> >> java.base/java.lang.invoke.Invokers.checkExactType(Invokers.java:532)
> >> at
> >>
> duncan.win.ole.IUnknownVtbl$Release.lambda$ofAddress$0(IUnknownVtbl.java:121)
>
> >>
> >>
> >> Class is below
> >>
> >> Kind regards
> >>
> >> Duncan
> >>
> >> import static duncan.win.ole.Ole32_h.S_OK;
> >> import static jdk.incubator.foreign.ValueLayout.ADDRESS;
> >> import static jdk.incubator.foreign.ValueLayout.JAVA_BYTE;
> >>
> >> import java.nio.charset.StandardCharsets;
> >> import java.util.Arrays;
> >> import java.util.HexFormat;
> >> import java.util.concurrent.TimeUnit;
> >> import java.util.function.Supplier;
> >>
> >> import org.junit.jupiter.api.Test;
> >> import org.junit.jupiter.api.condition.EnabledOnOs;
> >> import org.junit.jupiter.api.condition.OS;
> >>
> >> import duncan.win.ole.GUID;
> >> import duncan.win.ole.IUnknown;
> >> import duncan.win.ole.IUnknownVtbl;
> >> import duncan.win.ole.Ole32_h;
> >>
> >> import jdk.incubator.foreign.MemoryAddress;
> >> import jdk.incubator.foreign.MemorySegment;
> >> import jdk.incubator.foreign.ResourceScope;
> >> import jdk.incubator.foreign.SegmentAllocator;
> >>
> >> /**
> >> * Test app to run through Windows COM IUnknown API calls.
> >> * A JUNIT test runs main() - easily removed
> >> *
> >> * Usage:
> >> java ... TestGuid GUID_CLSID GUID_IID+
> >> *
> >> * GUIDs are defined in Windows format eg.
> >> "{00021401-0000-0000-C000-000000000046}"
> >> *
> >> * Note: have kept to Windows naming conventions when calling Windows
> >> definitions.
> >> *
> >> * This instantiates the class with GUID_CLSID, and returns the
> >> requested
> >> IID interface.
> >> * <p>If additional IIDs are provided these are requested
> >> * <p>This requires jextract where files are:
> >> * Ole32.h:
> >> * #include <objbase.h>
> >> *
> >> * jextract -source -lole32 -t duncan.win.ole -d duncan.win\src Ole32.h
> >> */
> >> public class TestGuid
> >> {
> >> // Junit testcase
> >> @EnabledOnOs(OS.WINDOWS)
> >> @Test void coverageTestGuid() throws Exception
> >> {
> >> TestGuid.main();
> >> }
> >>
> >> // This uses slimmed down version of my JUnknown, inlined here
> >> // along with all the dependent calls on other my other win32
> >> classes
> >> // Not called IUnknown to avoid name classes with jextract
> >> generated
> >> code.
> >> static class JUnknown
> >> {
> >> protected final MemoryAddress comObj;
> >>
> >> private final IUnknownVtbl.QueryInterface QueryInterface;
> >> private final IUnknownVtbl.AddRef AddRef;
> >> private final IUnknownVtbl.Release Release;
> >>
> >> /**
> >> * Derived classes can use this after chaining to the protected
> >> constructor
> >> */
> >> protected final MemorySegment vtable;
> >> protected final SegmentAllocator allocator;
> >>
> >> /**
> >> * Derived classes must not use this constructor as it
> >> * only reserves only a small memory segment for the VTABLE
> >> */
> >> public JUnknown(ResourceScope scope, SegmentAllocator
> >> allocator,
> >> MemoryAddress comObj)
> >> {
> >> this(scope, allocator, comObj,
> >> IUnknownVtbl.ofAddress(vtable(scope, comObj), scope));
> >> }
> >>
> >> /**
> >> * This version ensures a larger vTABLE can be read:
> >> * A constructor for a derived class - say IShellLink
> >> * would need to extract its own callbacks from vtable like
> >> this:
> >> * <code>
> >> public class JShellLink extends JUnknown { ...
> >>
> >> public JShellLink(ResourceScope scope, SegmentAllocator
> >> allocator, MemoryAddress comObj)
> >> {
> >> super(scope, allocator, comObj,
> >> IShellLinkWVtbl.ofAddress(vtable(scope, comObj), scope));
> >> GetPath = IShellLinkWVtbl.GetPath(vtable, scope);
> >> SetPath = IShellLinkWVtbl.SetPath(vtable, scope);
> >> SetArguments = IShellLinkWVtbl.SetArguments(vtable,
> >> scope);
> >> GetArguments = IShellLinkWVtbl.GetArguments(vtable,
> >> scope);
> >> GetDescription =
> >> IShellLinkWVtbl.GetDescription(vtable,
> >> scope);
> >> GetIDList = IShellLinkWVtbl.GetIDList(vtable,
> >> scope);
> >> }
> >> ...
> >> }
> >> * </code>
> >> * @param scope
> >> * @param allocator
> >> * @param comObj
> >> * @param vtable as provided by a call to
> >> Ole32_h.IxxxxxxVtbl.ofAddressRestricted(vtableA(comObj))
> >> */
> >> protected JUnknown(ResourceScope scope, SegmentAllocator
> >> allocator,
> >> MemoryAddress comObj, MemorySegment vtable)
> >> {
> >> this.allocator = allocator;
> >> this.comObj = comObj;
> >> this.vtable = vtable;
> >>
> >> QueryInterface = IUnknownVtbl.QueryInterface(vtable,
> >> scope);
> >> AddRef = IUnknownVtbl.AddRef(vtable, scope);
> >> Release = IUnknownVtbl.Release(vtable, scope);
> >> }
> >>
> >> public String toString()
> >> {
> >> return
> >> getClass().getSimpleName()+"["+toHex(comObj.toRawLongValue())+"]";
> >> }
> >>
> >> /**
> >> * Reads the vTABLE for this COM object, returning the address
> >> * which must be dereferenced by the outermost derived class
> >> with
> >> the jextract
> >> * call specific for the derived classes VTABLE:
> >> * IClassName.ofAddressRestricted(vtable(comObj));
> >> */
> >> public static MemoryAddress vtable(ResourceScope scope,
> >> MemoryAddress comObj)
> >> {
> >> // Only need to retrieve a pointer at index 0 so size of
> >> one
> >> pointer is OK:
> >> MemorySegment memseg = IUnknown.ofAddress(comObj, scope);
> >>
> >> // The address of QueryInterface is the first entry in the
> >> vtable:
> >> MemoryAddress pVT = IUnknown.lpVtbl$get(memseg);
> >>
> >> // This is only the address of the vTable, see
> >> IClassName.ofAddressRestricted(pVT);
> >> return pVT;
> >> }
> >>
> >> public int AddRef()
> >> {
> >> int refcount = AddRef.apply(comObj);
> >> System.out.println(this+".AddRef => "+refcount);
> >>
> >> return refcount;
> >> }
> >>
> >> /**
> >> * IUnknown::Release method (unknwn.h)
> >> *
> >>
> https://docs.microsoft.com/en-us/windows/win32/api/unknwn/nf-unknwn-iunknown-release
> >>
> >> * ULONG Release()
> >> */
> >> public int Release()
> >> {
> >> int refcount = Release.apply(comObj);
> >> System.out.println(this+".Release => "+refcount);
> >>
> >> return refcount;
> >> }
> >>
> >> /**
> >> *
> >>
> https://docs.microsoft.com/en-us/windows/win32/api/unknwn/nf-unknwn-iunknown-queryinterface(refiid_void)
> >>
> >> * IUnknown::QueryInterface(REFIID,void) method (unknwn.h)
> >> * HRESULT IUnknown::QueryInterface(REFIID iid, void** ppv);
> >> * <P>
> >> * https://osm.hpi.de/LV/Components04/VL5/MSDN/DrGUI-on-COM
> >> */
> >> public MemoryAddress QueryInterface(MemorySegment riid)
> >> {
> >> System.out.println(this+".QueryInterface
> >> riid="+toHex(riid.toArray(JAVA_BYTE)));
> >>
> >> // This is allocated each time in case more than one
> >> call made
> >> to QueryInterface
> >> MemorySegment queryRes = allocator.allocate(ADDRESS,
> >> MemoryAddress.NULL);
> >>
> >> int hRes = QueryInterface.apply(comObj, riid.address(),
> >> queryRes.address());
> >>
> >> MemoryAddress address = queryRes.get(ADDRESS, 0);
> >> check0("QueryInterface", hRes);
> >> System.out.println(" =>
> >> IID="+toHex(address.toRawLongValue()));
> >>
> >> return address;
> >> }
> >> }
> >>
> >> public static MemorySegment toWideString(String s, SegmentAllocator
> >> allocator)
> >> {
> >> return allocator.allocateArray(JAVA_BYTE,
> >> (s+"\0").getBytes(StandardCharsets.UTF_16LE));
> >> }
> >> public static void check0(String func, int hRes)
> >> {
> >> checkThat(hRes == S_OK(), () -> "Error: "+func+" NOT OK,
> >> result was
> >> "+hRes+" / 0x"+Integer.toHexString(hRes));
> >> System.out.println(func+" OK result => "+hRes);
> >> }
> >>
> >> public static void checkThat(boolean condition, Supplier<String>
> >> error)
> >> {
> >> if (!condition)
> >> {
> >> String message = error.get();
> >> System.err.println(message);
> >> throw new RuntimeException(message);
> >> }
> >> }
> >>
> >> public static MemorySegment CLSIDFromString(SegmentAllocator
> >> allocator,
> >> String guid)
> >> {
> >> System.out.println("CLSIDFromString clsid="+guid);
> >> // Falls through to do the actual conversion:
> >> MemorySegment pGUID = GUID.allocate(allocator);
> >>
> >> MemorySegment wide = toWideString(guid, allocator);
> >>
> >> int hRes = Ole32_h.CLSIDFromString(wide, pGUID);
> >> check0("CLSIDFromString", hRes);
> >> System.out.println(" =>
> >> GUID="+toHex(pGUID.toArray(JAVA_BYTE)));
> >>
> >> return pGUID;
> >> }
> >> public static String toHex(long value)
> >> {
> >> return "0x"+Long.toHexString(value).toUpperCase();
> >> }
> >> private static String toHex(byte[] arr) {
> >> HexFormat hex = HexFormat.ofDelimiter(",
> >> ").withPrefix("0x").withUpperCase();
> >> return "new byte["+arr.length+"] {"+hex.formatHex(arr)+"}";
> >> }
> >> public static MemoryAddress CoCreateInstance(SegmentAllocator
> >> allocator, MemorySegment rclsid, int dwClsContext, MemorySegment riid)
> >> {
> >> MemoryAddress pUnkOuter = MemoryAddress.NULL;
> >> MemorySegment ptrComObj = allocator.allocate(ADDRESS,
> >> MemoryAddress.NULL);
> >> // https://www.purebasic.fr/english/viewtopic.php?f=13&t=45583
> >>
> >> System.out.println("CoCreateInstance
> >> rclsid="+toHex(rclsid.toArray(JAVA_BYTE)));
> >> System.out.println("CoCreateInstance
> >> pUnkOuter="+pUnkOuter.address());
> >> System.out.println("CoCreateInstance
> >> dwClsContext="+dwClsContext);
> >> System.out.println("CoCreateInstance
> >> riid="+toHex(riid.toArray(JAVA_BYTE)));
> >>
> >> int hRes = Ole32_h.CoCreateInstance(rclsid, pUnkOuter,
> >> dwClsContext, riid, ptrComObj);
> >> check0("CoCreateInstance", hRes);
> >> MemoryAddress comObj = ptrComObj.get(ADDRESS, 0);
> >> System.out.println("CoCreateInstance =>
> >> address="+toHex(comObj.toRawLongValue()));
> >>
> >> return comObj;
> >> }
> >> public static void checkResult(String func, int hRes, int ...
> >> okcodes)
> >> {
> >> for (int ok : okcodes)
> >> {
> >> if (ok == hRes)
> >> {
> >> System.out.println(func+" OK result => "+hRes+"
> >> ok="+Arrays.toString(okcodes));
> >> return;
> >> }
> >> }
> >> checkThat(false, () -> "Error: "+func+" NOT OK, result code
> >> "+hRes+" / 0x"+Integer.toHexString(hRes)+"
> >> ok="+Arrays.toString(okcodes));
> >> }
> >> public static AutoCloseable CoInitialize()
> >> {
> >> int hRes = Ole32_h.CoInitialize(MemoryAddress.NULL);
> >> checkResult("CoInitialize", hRes, Ole32_h.S_OK(),
> >> Ole32_h.S_FALSE());
> >>
> >> return Ole32_h::CoUninitialize;
> >> }
> >>
> >> public static void main(String ... args) throws Exception
> >> {
> >> final long t0 = System.nanoTime();
> >>
> >> // Default run instantiates COM object CLSID_ShellLink and
> >> asks for
> >> its IShellLinkW and IID_Persist interfaces
> >> String[] defaults = {
> >> /*GUID_CLSID_ShellLink_str*/
> >> "{00021401-0000-0000-C000-000000000046}",
> >> /*GUID_IID_IShellLinkW_str*/
> >> "{000214F9-0000-0000-C000-000000000046}",
> >> /*GUID_IID_Persist_str*/
> >> "{0000010B-0000-0000-C000-000000000046}",
> >> };
> >>
> >> if (args.length < 2) args = defaults;
> >>
> >> String clsidStr = args[0];
> >> String iidStr = args[1];
> >> System.out.println("lookup GUID CLSID "+clsidStr+" IID
> >> "+iidStr);
> >>
> >> try(ResourceScope scope = ResourceScope.newConfinedScope())
> >> {
> >> SegmentAllocator allocator =
> >> SegmentAllocator.newNativeArena(scope);
> >>
> >> // Lookup GUIDs for class and interfaces
> >> MemorySegment clsid = CLSIDFromString(allocator,
> >> clsidStr);
> >> MemorySegment iid = CLSIDFromString(allocator, iidStr);
> >>
> >> // Setup COM:
> >> try(var autoC = CoInitialize())
> >> {
> >> // Get a pointer to the COM instance.
> >> MemoryAddress comObj = CoCreateInstance(allocator,
> >> clsid,
> >> Ole32_h.CLSCTX_INPROC_SERVER(), iid);
> >>
> >> // JUnknown ignores the complete vtable, only
> >> initialises
> >> callbacks for IUnknown interface
> >> JUnknown iUnknown = new JUnknown(scope, allocator,
> >> comObj);
> >> try
> >> {
> >> // test add/release on this iUnknown
> >> int ar = iUnknown.AddRef();
> >> int re = iUnknown.Release();
> >> checkResult("AddRef/Release()", ar, re+1);
> >>
> >> // Access other interfaces via the first one:
> >> for (int ii = 2; ii < args.length; ii++)
> >> {
> >> // Use Query interface on another interface
> >> from
> >> the CLSID
> >> MemorySegment iidextra =
> >> CLSIDFromString(allocator,
> >> args[ii]);
> >>
> >> // Query CLS for this other interface
> >> definition
> >> MemoryAddress iOtherUnknown =
> >> iUnknown.QueryInterface(iidextra);
> >> JUnknown iOther = new JUnknown(scope,
> >> allocator,
> >> iOtherUnknown);
> >> try
> >> {
> >> // test add/release on this other
> >> iUnknown (NB
> >> same instance but with different vtable)
> >> int add = iOther.AddRef();
> >> checkResult("AddRef()", add, 3);
> >> int rel = iOther.Release();
> >> checkResult("Release()", rel, 2);
> >> }
> >> finally
> >> {
> >> // Signal end of use of iOther
> >> // Note that the ref count includes the
> >> value
> >> on iUnknown
> >> int refC = iOther.Release();
> >> checkResult("Release()", refC, 1);
> >> }
> >> }
> >> }
> >> finally
> >> {
> >> // Signal end of use of iUnknown
> >> int refC = iUnknown.Release();
> >> // Note that the ref count should now be zero
> >> checkResult("Release()", refC, 0);
> >> }
> >> }
> >> final long now = System.nanoTime();
> >> System.out.println("Ended
> >> ms="+TimeUnit.NANOSECONDS.toMillis(now - t0)+" "+clsidStr);
> >> }
> >> }
> >> }
> >>
> >> On Mon, 27 Sept 2021 at 16:45, Duncan Gittins <duncan.gittins at gmail.com
> >
> >> wrote:
> >>
> >>> The jextract parameters which generate the broken Windows OLE API
> >>> code I
> >>> outlined below is:
> >>>
> >>> jextract -source -lole32 -t duncan.win.ole -d
> >>> source\duncan.win\java
> >>> headers\Ole32.h
> >>>
> >>> where headers\Ole32.h contains:
> >>>
> >>> #include <objbase.h>
> >>>
> >>> A temporary workaround for this jextract issue is to edit
> >>> FunctionalInterfaceBuilder.java / emitFunctionalFactoryForPointer() to
> >>> insert the (Addressable) casts for MemoryAddress parameters:
> >>>
> >>> ---
> >>>
> a/src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/FunctionalInterfaceBuilder.java
>
> >>>
> >>> +++
> >>>
> b/src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/FunctionalInterfaceBuilder.java
>
> >>>
> >>> @@ -123,7 +123,7 @@ public class FunctionalInterfaceBuilder extends
> >>> ClassSourceBuilder {
> >>> append(mhConstant.accessExpression() +
> >>> ".invokeExact((Addressable)addr");
> >>> if (fiType.parameterCount() > 0) {
> >>> String params = IntStream.range(0,
> >>> fiType.parameterCount())
> >>> - .mapToObj(i -> "x" + i)
> >>> + .mapToObj(i ->
> >>> (fiType.parameterType(i).getName().endsWith(".MemoryAddress") ?
> >>> "(Addressable)":"")+"x" + i)
> >>> .collect(Collectors.joining(", "));
> >>> append(", " + params);
> >>> }
> >>>
> >>> this fixes the ofAddress callbacks, for example see
> >>> IPersistFileVtbl.java:
> >>>
> >>> public interface Load {
> >>> ....
> >>> static Load ofAddress(MemoryAddress addr) {
> >>> return (jdk.incubator.foreign.MemoryAddress x0,
> >>> jdk.incubator.foreign.MemoryAddress x1, int x2) -> {
> >>> try {
> >>> // WAS: return
> >>> (int)IPersistFileVtbl.Load$MH.invokeExact((Addressable)addr, x0, x1,
> >>> x2);
> >>> return
> >>> (int)IPersistFileVtbl.Load$MH.invokeExact((Addressable)addr,
> >>> (Addressable)x0, (Addressable)x1, x2);
> >>> } catch (Throwable ex$) {
> >>> throw new AssertionError("should not reach
> >>> here", ex$);
> >>> }
> >>> };
> >>> }
> >>>
> >>> Kind regards
> >>>
> >>> Duncan
> >>>
> >>>
> >>> On Fri, 24 Sept 2021 at 16:36, Duncan Gittins
> >>> <duncan.gittins at gmail.com>
> >>> wrote:
> >>>
> >>>> I've pulled latest panama-foreign which has the changes outlined in
> >>>>
> >>>> https://inside.java/2021/09/16/finalizing-the-foreign-apis/
> >>>>
> >>>> I've a few problems to resolve to match up, one is related to jextract
> >>>> which generates interfaces for Windows OLE APIs. The invokeExact
> >>>> params
> >>>> are missing some (Addressable) casts (I think?) eg this is IUnknown
> >>>> Release:
> >>>>
> >>>> public interface Release {
> >>>>
> >>>> int apply(jdk.incubator.foreign.MemoryAddress x0);
> >>>> static CLinker.UpcallStub allocate(Release fi) {
> >>>> return RuntimeHelper.upcallStub(Release.class, fi,
> >>>> IUnknownVtbl.Release$FUNC,
> >>>> "(Ljdk/incubator/foreign/MemoryAddress;)I");
> >>>> }
> >>>> static CLinker.UpcallStub allocate(Release fi, ResourceScope
> >>>> scope) {
> >>>> return RuntimeHelper.upcallStub(Release.class, fi,
> >>>> IUnknownVtbl.Release$FUNC, "(Ljdk/incubator/foreign/MemoryAddress;)I",
> >>>> scope);
> >>>> }
> >>>> static Release ofAddress(MemoryAddress addr) {
> >>>> return (jdk.incubator.foreign.MemoryAddress x0) -> {
> >>>> try {
> >>>> return
> >>>> (int)IUnknownVtbl.Release$MH.invokeExact((Addressable)addr, x0);
> >>>> } catch (Throwable ex$) {
> >>>> throw new AssertionError("should not reach here",
> >>>> ex$);
> >>>> }
> >>>> };
> >>>> }
> >>>> }
> >>>>
> >>>> Stack traces show
> >>>>
> >>>> Caused by: java.lang.invoke.WrongMethodTypeException: expected
> >>>> (Addressable,Addressable)int but found (Addressable,MemoryAddress)int
> >>>>
> >>>> Kind regards
> >>>>
> >>>> Duncan
> >>>>
> >>>>
> >>>>
>
More information about the panama-dev
mailing list