<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi Duncan, as always thanks for the feedback.</p>
<p>Few observation on this topic:</p>
<p>* The PR in [1] will at least ameliorate the situation when it
comes to static initializers being too bloated (and slowing down
startup)<br>
* Adding filtering by functional interface seems to be at odds
with the way in which jextract filtering options work - you can
filter a C entity (e.g. a typedef or a function), but not a Java
one. So that doesn't strike me as the obviously correct solution<br>
* There is a certain tension here between two goals: generating
higher-level bindings on the one hand (which led to add functional
interface accessors [2]) and stripping down a jarfile in an
arbitrary fashion. The more "info" is contained in the bindings,
the less possible this stripping process becomes. For instance, it
is handy that all pointers are modelled as "MemorySegment" - if we
had `Pointer<Foo>` in any of the fields, you would need to
also carry "Foo" in the resulting jar file<br>
* I view all these attempts at "trying to reduce the size of the
generated classes" as transient solutions - at some point, when
better techonologies [3] will be available, _all_ of the constant
classes generated by jextract will disappear<br>
* Perhaps jextract should do more to deduplicate functional
interfaces with the same C signature. Right now, since the name of
e.g. C variable/parameter is embedded into the functional
interface name, this is not possible. While this will remain the
case for typedefs (which, on the other hand can be filtered out
on-demand), I see no reason as to why "anonymous" function types
shouldn't be translated using a more sensible naming scheme which
allows for deduplication. E.g. assuming in your case most of the
functional interfaces share the same signature, no stripping down
would be required, as jextract will only generate few _distinct_
functional interfaces.</p>
<p>Maurizio</p>
<p>[1] - <a class="moz-txt-link-freetext" href="https://git.openjdk.org/jextract/pull/118">https://git.openjdk.org/jextract/pull/118</a><br>
[2] - <a class="moz-txt-link-freetext" href="https://git.openjdk.java.net/panama-foreign/pull/456">https://git.openjdk.java.net/panama-foreign/pull/456</a><br>
[3] - <a class="moz-txt-link-freetext" href="https://openjdk.org/jeps/8209964">https://openjdk.org/jeps/8209964</a><br>
<br>
</p>
<div class="moz-cite-prefix">On 11/04/2023 19:54, Duncan Gittins
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CABOqcFuuX1rNEPFaEpcRjScp+1vcTO7PFnB8kSPjJ=mrZvUOnw@mail.gmail.com">
<div dir="ltr">
<div>Regarding the new class dependencies relating to callback
interfaces mentioned here, and the JIRA Jorn raised for
functional interfaces for function pointer types: </div>
<div><br>
</div>
<div> <a href="https://mail.openjdk.org/pipermail/jextract-dev/2023-April/000742.html" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://mail.openjdk.org/pipermail/jextract-dev/2023-April/000742.html</a><br>
</div>
<div> <a href="https://mail.openjdk.org/pipermail/jextract-dev/2023-February/000675.html" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://mail.openjdk.org/pipermail/jextract-dev/2023-February/000675.html</a><br>
</div>
<div> <a href="https://bugs.openjdk.org/browse/CODETOOLS-7903456" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://bugs.openjdk.org/browse/CODETOOLS-7903456</a></div>
<div>
<div><br>
</div>
<div>I have an application that uses IShellLinkWVtbl (and
several others). Currently, referencing IShellLinkWVtbl
causes class load for every one of the related callback
class files: QueryInterface AddRef Release GetPath
GetIDList SetIDList GetDescription SetDescription
GetWorkingDirectory SetWorkingDirectory GetArguments
SetArguments GetHotkey SetHotkey GetShowCmd SetShowCmd
GetIconLocation SetIconLocation SetRelativePath Resolve
SetPath</div>
<div><br>
</div>
<div>With the JIRA I will be able to exclude the callbacks I
don't need (QueryInterface AddRef Release already defined in
IUnknown) leaving me with the others. I build a jar from
jextract code one but use it in different applications. But
on occasions they may only use the GetX calls, sometimes
GetX and SetX, sometimes SetX. </div>
<div><br>
</div>
<div>I think the jextract code generation should go one step
further and handle loading these functional interfaces for
function pointer types on demand at the time they are used,
not loading all at first reference of parent class
IShellLinkWVtbl. Perhaps something like this for each
callback interface - this would encapsulate all the fields
of the functional interface together and mean any access to
the parent class IShellLinkWVtbl doesn't load all the other
callbacks unless the caller uses them, and not at first
reference of parent class such as IShellLinkWVtbl:</div>
<div><br>
</div>
<div>
<div style="padding:0px 0px 0px 2px">
<div style="color:rgb(0,0,0);font-family:Consolas;font-size:10pt;white-space:pre-wrap"><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">public</span> <span style="color:rgb(127,0,85);font-weight:bold">interface</span> QueryInterface {</p><p style="margin:0px">
</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">static</span> <span style="color:rgb(127,0,85);font-weight:bold">final</span> FunctionDescriptor <span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">QueryInterface$FUNC</span> = FunctionDescriptor.<span style="font-style:italic">of</span>(Constants$root.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">C_LONG$LAYOUT</span>,</p><p style="margin:0px"> Constants$root.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">C_POINTER$LAYOUT</span>,</p><p style="margin:0px"> Constants$root.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">C_POINTER$LAYOUT</span>,</p><p style="margin:0px"> Constants$root.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">C_POINTER$LAYOUT</span></p><p style="margin:0px"> );</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">int</span> apply(java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">_x0</span>, java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">_x1</span>, java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">_x2</span>);</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">static</span> <span style="color:rgb(127,0,85);font-weight:bold">final</span> MethodHandle <span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">UP$MH</span> = RuntimeHelper.<span style="font-style:italic">upcallHandle</span>(QueryInterface.<span style="color:rgb(127,0,85);font-weight:bold">class</span>, <span style="color:rgb(42,0,255)">"apply"</span>, <span style="color:rgb(0,0,192);background-color:rgb(206,204,247);font-style:italic;font-weight:bold">QueryInterface$FUNC</span>);</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">static</span> MemorySegment allocate(QueryInterface <span style="color:rgb(106,62,62)">fi</span>, SegmentScope <span style="color:rgb(106,62,62)">scope</span>) {</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">return</span> RuntimeHelper.<span style="font-style:italic">upcallStub</span>(QueryInterface.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">UP$MH</span>, <span style="color:rgb(106,62,62)">fi</span>, <span style="color:rgb(0,0,192);background-color:rgb(206,204,247);font-style:italic;font-weight:bold">QueryInterface$FUNC</span>, <span style="color:rgb(106,62,62)">scope</span>);</p><p style="margin:0px"> }</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">static</span> <span style="color:rgb(127,0,85);font-weight:bold">final</span> MethodHandle <span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">DOWN$MH</span> = RuntimeHelper.<span style="font-style:italic">downcallHandle</span>(</p><p style="margin:0px"> <span style="color:rgb(0,0,192);background-color:rgb(206,204,247);font-style:italic;font-weight:bold">QueryInterface$FUNC</span></p><p style="margin:0px"> );</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">static</span> QueryInterface ofAddress(MemorySegment <span style="color:rgb(106,62,62)">addr</span>, SegmentScope <span style="color:rgb(106,62,62)">scope</span>) {</p><p style="margin:0px"> MemorySegment <span style="color:rgb(106,62,62)">symbol</span> = MemorySegment.<span style="font-style:italic">ofAddress</span>(<span style="color:rgb(106,62,62)">addr</span>.address(), 0, <span style="color:rgb(106,62,62)">scope</span>);</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">return</span> (java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">__x0</span>, java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">__x1</span>, java.lang.foreign.MemorySegment <span style="color:rgb(106,62,62)">__x2</span>) -> {</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">try</span> {</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">return</span> (<span style="color:rgb(127,0,85);font-weight:bold">int</span>)QueryInterface.<span style="color:rgb(0,0,192);font-style:italic;font-weight:bold">DOWN$MH</span>.invokeExact(<span style="color:rgb(106,62,62)">symbol</span>, <span style="color:rgb(106,62,62)">__x0</span>, <span style="color:rgb(106,62,62)">__x1</span>, <span style="color:rgb(106,62,62)">__x2</span>);</p><p style="margin:0px"> } <span style="color:rgb(127,0,85);font-weight:bold">catch</span> (Throwable <span style="color:rgb(106,62,62)">ex$</span>) {</p><p style="margin:0px"> <span style="color:rgb(127,0,85);font-weight:bold">throw</span> <span style="color:rgb(127,0,85);font-weight:bold">new</span> AssertionError(<span style="color:rgb(42,0,255)">"should not reach here"</span>, <span style="color:rgb(106,62,62)">ex$</span>);</p><p style="margin:0px"> }</p><p style="margin:0px"> };</p><p style="margin:0px"> }</p><p style="margin:0px"> }</p><p style="margin:0px"><span style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">
</span></p><p style="margin:0px"><span style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">Kind regards</span>
</p></div>
</div>
</div>
<div><br>
</div>
<div>Duncan</div>
<div>
<div><br>
<br>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, 17 Feb 2023 at 23:57,
Maurizio Cimadamore <<a href="mailto:mcimadamore@openjdk.org" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">mcimadamore@openjdk.org</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On
Fri, 17 Feb 2023 15:32:53 GMT, Maurizio Cimadamore <<a href="mailto:mcimadamore@openjdk.org" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">mcimadamore@openjdk.org</a>>
wrote:<br>
<br>
> The current translation stragegy for function pointers is
not very efficient. Every time a lambda expression is turned
into an upcall stub, a new method handle lookup is performed,
which is quite a slow operation.<br>
> <br>
> This patch improves the translation strategy so that the
lookup is only performed once. For instance, considering the
following `typedef`:<br>
> <br>
> <br>
> typedef void (*cb)(int, int);<br>
> <br>
> <br>
> jextract will generate the following functional
interface:<br>
> <br>
> <br>
> import static java.lang.foreign.ValueLayout.*;<br>
> /**<br>
> * {@snippet :<br>
> * void (*cb)(int,int);<br>
> * }<br>
> */<br>
> public interface cb {<br>
> <br>
> void apply(int _x0, int _x1);<br>
> static MemorySegment allocate(cb fi, Arena scope) {<br>
> return
RuntimeHelper.upcallStub(constants$0.cb_UP$MH, fi,
constants$0.cb$FUNC, scope);<br>
> }<br>
> static cb ofAddress(MemorySegment addr, Arena arena)
{<br>
> MemorySegment symbol =
addr.reinterpret(arena.scope(), null);<br>
> return (int __x0, int __x1) -> {<br>
> try {<br>
>
constants$0.cb_DOWN$MH.invokeExact(symbol, __x0, __x1);<br>
> } catch (Throwable ex$) {<br>
> throw new AssertionError("should not
reach here", ex$);<br>
> }<br>
> };<br>
> }<br>
> }<br>
> <br>
> <br>
> Where the constants are defined as follows:<br>
> <br>
> <br>
> static final FunctionDescriptor cb$FUNC =
FunctionDescriptor.ofVoid(<br>
> Constants$root.C_INT$LAYOUT,<br>
> Constants$root.C_INT$LAYOUT<br>
> );<br>
> <br>
> static final FunctionDescriptor cb_UP$FUNC =
FunctionDescriptor.ofVoid(<br>
> Constants$root.C_INT$LAYOUT,<br>
> Constants$root.C_INT$LAYOUT<br>
> );<br>
> <br>
> static final MethodHandle cb_UP$MH =
RuntimeHelper.upcallHandle(cb.class, "apply",
constants$0.cb_UP$FUNC);<br>
> <br>
> static final FunctionDescriptor cb_DOWN$FUNC =
FunctionDescriptor.ofVoid(<br>
> Constants$root.C_INT$LAYOUT,<br>
> Constants$root.C_INT$LAYOUT<br>
> );<br>
> <br>
> static final MethodHandle cb_DOWN$MH =
RuntimeHelper.downcallHandle(<br>
> constants$0.cb_DOWN$FUNC<br>
> );<br>
> <br>
> <br>
> And, where `RuntimeHelper::upcallHandler` is defined as
follows:<br>
> <br>
> <br>
> static MethodHandle upcallHandle(Class<?> fi,
String name, FunctionDescriptor fdesc) {<br>
> try {<br>
> return MH_LOOKUP.findVirtual(fi, name,
fdesc.toMethodType());<br>
> } catch (Throwable ex) {<br>
> throw new AssertionError(ex);<br>
> }<br>
> }<br>
> <br>
> <br>
> This allows to perform the method handle lookup only
once. This means that when creating an upcall stub, we only
have to `bind` the functional interface instance to the method
handle, and then create an upcall stub with the resulting
handle. Both operations should be fast after the first time,
because of caching.<br>
<br>
This pull request has now been integrated.<br>
<br>
Changeset: dfafbbe7<br>
Author: Maurizio Cimadamore <<a href="mailto:mcimadamore@openjdk.org" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">mcimadamore@openjdk.org</a>><br>
URL: <a href="https://git.openjdk.org/jextract/commit/dfafbbe7d6ab3c190dae5519236bef718fd10755" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://git.openjdk.org/jextract/commit/dfafbbe7d6ab3c190dae5519236bef718fd10755</a><br>
Stats: 41 lines in 4 files changed: 28 ins; 2 del; 11 mod<br>
<br>
7903437: Improve translation strategy for function pointers<br>
<br>
Reviewed-by: jvernee<br>
<br>
-------------<br>
<br>
PR: <a href="https://git.openjdk.org/jextract/pull/109" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://git.openjdk.org/jextract/pull/109</a><br>
</blockquote>
</div>
</blockquote>
</body>
</html>