[External] : Re: Jextract 20 binaries are now available

Tue Apr 11 14:52:46 UTC 2023

Thanks Jorn

Your suggestion to allow filtering out of the unnecessary interfaces will
be a great help to simplify the generated code dependencies. I believe the
changes giving this new behaviour was outlined here:

https://mail.openjdk.org/pipermail/jextract-dev/2023-February/000675.html

One query I had on the new UP/DOWN generated code - is there a circumstance
where the FunctionDescriptor for the upcall QueryInterface_DOWN$FUNC
differs from the downcall QueryInterface_DOWN$FUNC and the default
FunctionDescriptor QueryInterface$FUNC? There is a strange dependency in
generated code for QueryInterface callback:

     QueryInterface_DOWN$MH depends on QueryInterface_DOWN$FUNC
     QueryInterface_UP$MH depends on QueryInterface_UP$FUNC

The callback interface QueryInterface uses all three $FUNC definitions
though each appears to be declared as the same value:

     QueryInterface.allocate depends on QueryInterface_UP$MH and
QueryInterface$FUNC and therefore also QueryInterface_UP$FUNC
     QueryInterface ofAddress depends on QueryInterface_DOWN$MH and
therefore QueryInterface_DOWN$FUNC

Kind regards

Duncan

On Tue, 11 Apr 2023 at 14:04, Jorn Vernee <jorn.vernee at oracle.com> wrote:

> Hi Duncan,
>
> In this case the functional interface QueryInterface is being generated
> because a field in the enclosing struct has a field with a function pointer
> type. QueryInterface_UP$MH is a constant required by the implementation of
> that interface.
>
> While I think your solution could fix the immediate issue you're having,
> we recently moved constants like QueryInterface_UP$MH to all be generated
> in the constant support files [1]. I think that is long term a better
> solution, since it takes the proper care to make sure the source files
> don't get too large.
>
> With that patch, it would however still be dicey to manually strip the
> compiled classes from a jar though, since there would still be a reference
> to the class from the constants file. So, if another constant in the same
> file was accessed, the same issue would occur. Unless
> -Djextract.constants.per.class=0 is used as well, so that each constant
> class has only 1 constant.
>
> I think stripping classes from a jar will always be risky, since it is
> done without the knowledge of jextract. We have the --include-* options
> as a supported way to do filtering in jextract. I think adding an
> --include-* option for filtering out functional interface is reasonable
> as well. If you know you are never going to use the generated interface, it
> should be possible to filter it out (and all the constants it depends on).
> I've filled an issue for this:
> https://bugs.openjdk.org/browse/CODETOOLS-7903456
>
> Jorn
>
> [1] : https://github.com/openjdk/jextract/pull/117
> On 08/04/2023 19:10, Duncan Gittins wrote:
>
> Jorn
>
> Thanks everyone for the effort on this. Its been a while since I've built
> my own jextract v20 so your build has given me more recent changes.
>
> Apologies if this is covered in previous trails: I have run into an issue
> related to the way some performance changes were implemented in upcall stub
> generation. The generated code has introduced new class loader dependencies
> for Windows COM interfaces. For example using IShellLinkWVtbl gives rise to
> many interfaces such as IShellLinkWVtbl$QueryInterface.
>
> All is fine if use new jextract and perform my tests with generated file.
> My arguments for jextract are:
>     --source  -lshell32 -t duncan.win.shell --output
> source\duncan.win\java headers\Shell32.h
> where Shell32.h is just:
>      #include <shlobj_core.h>
>
> However I've noticed the new jextract generated code has new class
> dependencies. I lookup COM objects via IUnknown$QueryInterface and never
> call the duplicate version like IShellLinkWVtbl$QueryInterface so I've been
> striping these similar and un-used classes from my jars.
>
> In the old code generation, IShellLinkWVtbl was independent of
> IShellLinkWVtbl$QueryInterface and all other IXYZ COM interfaces.
>
> The new generated code of IShellLinkWVtbl.java references
> IShellLinkWVtbl$QueryInterface.class. Similarly every vtable class (eg
> IUnknown / IPersistFile etc) depends now on every interface class file of
> every COM interface (AddRef / Release / etc) it supports because of the new
> xyz_UP$MH fields:
>
>     Exception java.lang.NoClassDefFoundError:
> duncan/win/shell/IShellLinkWVtbl$QueryInterface
>      |        at IShellLinkWVtbl.<clinit> (IShellLinkWVtbl.java:75)
>
> Here is the snippet of new code which shows the new upcallHandle which
> means every vtable.class requires all com .class files. I am hoping that
> this can be resolved easily simply by generating the declaration of
> QueryInterface_UP$MH inside interface IShellLinkWVtbl$QueryInterface rather
> than into the parent class IShellLinkWVtbl. This would avoid a large number
> of .class files being read when referencing a vtable class:
>
>     public class IShellLinkWVtbl {
>      ...
>       static final MethodHandle QueryInterface_UP$MH =
> RuntimeHelper.upcallHandle(QueryInterface.class, "apply",
> IShellLinkWVtbl.QueryInterface_UP$FUNC);
>     ...
>         /** HRESULT (*QueryInterface)(IShellLinkW*,const IID*,void**); */
>         public interface QueryInterface {
>             //  UP$MH should be here?
>             int apply(java.lang.foreign.MemorySegment _x0,
> java.lang.foreign.MemorySegment _x1, java.lang.foreign.MemorySegment _x2);
>             static MemorySegment allocate(QueryInterface fi, SegmentScope
> scope) {
>             return
> RuntimeHelper.upcallStub(IShellLinkWVtbl.QueryInterface_UP$MH, fi,
> IShellLinkWVtbl.QueryInterface$FUNC, scope);
>         }
>       ...
>     }
>
> Kind regards
>
> Duncan
>
> On Fri, 7 Apr 2023 at 18:34, Jorn Vernee <jorn.vernee at oracle.com> wrote:
>
>> Hello,
>>
>> Jextract 20 binaries are now available at: https://jdk.java.net/jextract/
>> <https://urldefense.com/v3/__https://jdk.java.net/jextract/__;!!ACWV5N9M2RV99hQ!Lub79JM1QWad74DQ066IIkIBcFwi4DYJXQssC4oVgEC8gOf4yh9sd209Pz_Tt0tFZjEMv4eIkP5e9hb0t0W-h2P5ww$>
>>
>> This version of jextract targets the foreign function and memory access
>> API in Java 20, which was released recently as well [1]
>>
>> The existing binaries of jextract 19, which target Java 19, are still
>> available through a link under the "Other versions" section on the
>> jextract download page linked above.
>>
>> Jorn
>>
>> [1]: https://jdk.java.net/20/
>> <https://urldefense.com/v3/__https://jdk.java.net/20/__;!!ACWV5N9M2RV99hQ!Lub79JM1QWad74DQ066IIkIBcFwi4DYJXQssC4oVgEC8gOf4yh9sd209Pz_Tt0tFZjEMv4eIkP5e9hb0t0UTAflM8A$>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jextract-dev/attachments/20230411/57597d49/attachment-0001.htm>