<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 06/09/2022 18:14, Manuel
Bleichenbacher wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
<div dir="ltr">
<div>I generally agree that it should not be opinionated. But
you could easily end up writing a C to Java compiler to make
it happen.</div>
<div><br>
</div>
<div>It starts with simple things. How do you deal with the
below variable (which is exportable according to your
definition)?<br>
</div>
<div><br>
</div>
<div>int __declspec(selectany) myvar = 4;</div>
<div><br>
</div>
<div>If you allocate native memory, initialize it with 4 and
generate a getter: great. But an "int" variable is just the
start. You also have to do it for complex, nested structs and
unions including initialization with designators like:</div>
</div>
</blockquote>
I don't get what you mean here at all. Jextract won't _allocate_
anything. It would just look for "myvar", which will be either
there, or not there, depending on how the lib was compiled. If the
symbol is not there, that will result in a runtime error when
looking it up. I think you read way too much into what I said.
<blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>This is a global function definition. In a header file for
consumers of a library, it's nonsense, like "int x = 4;". How
do you treat it in a non-opinionated way? Will you translate
the function to Java code? Will you also do it for any
arbitrarily complex function? Where does this end? Can we then
trick jextract to translate all C code to Java if we just put
it in a header file?<br>
</div>
</div>
</blockquote>
This is absolutely not what I was implying. Jextract should not
"simulate" or "translate" anything. The question is much simpler,
and is whether, for a given definition seen in an header, jextract
should generate lookup logic and accessors or not.<br>
<blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Whatever you do, make jextract more transparent. And this
is probably the biggest complaint about the tool in its
current state. When I started using it, I initially didn't
succeed at all. In most cases, the tool didn't generate the
function or struct I was looking for and, worst of all, didn't
report anything.</div>
<ul>
<li>If you explicitly specify "--include-var myar" and
jextract skips it because it's static, it should tell so.</li>
<li>If you explicitly specify "--include-struct mystruct" and
jextract skips it because it's a typedef, it should tell so.</li>
<li>If you explicitly specify "--include-var eDefault" and
jextract skips it because it's an enum constant and enum
constants are only included if you use "--include-macro", it
should tell so.</li>
<li>If you explicitly specify "--include-macro XY" and
jextract skips it because it's function like, it should tell
so.<br>
</li>
<li>If you explicitly specify "--include-function myfunc" and
jextract can't find such a function, it should tell so.</li>
</ul>
<div>These are many of the things I had to learn the hard way,
and spent (or wasted) time for experiments to derive
jextract's behavior.<br>
</div>
</div>
</blockquote>
<p>Fair enough. For the records, should you have any patch which add
extra diagnostics we'd be happy to accept the contribution!<br>
</p>
<p>Note that, as documented in the jextract readme, the option
"--dump-includes <String>" can be used to dump whatever
jextract found in the header files. From there you can easily see
if the symbol you are looking for is there, what is its header,
and what the symbol kind is. Of course, it's no substitution for
better diagnostics, but I feel it would minimize the
experimentation.</p>
<blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>So before putting any serious effort into implementing
non-opinionated treatment of things that shouldn't have been
in header files in the first place, I propose to put the main
effort into making jextract more transparent. In particular,
it should report whenever something was requested but skipped
or not found. And it should provide the reason, possibly
indicating near misses like "struct" instead of "typedef".<br>
</div>
</div>
</blockquote>
<p>Again, the suggestion in my email was to stop at what we did
here:</p>
<p><a class="moz-txt-link-freetext" href="https://git.openjdk.org/jextract/pull/70">https://git.openjdk.org/jextract/pull/70</a></p>
<p>No extra crazy logic to be implemented. <br>
</p>
<p>Maurizio<br>
</p>
<blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep 6, 2022 at 6:14 PM
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Thinking about some more (after some offline internal
discussion).</p>
<p>I don't think jextract should be "opinionated" as to how
global variables are declared in headers. Is there a
"correct" way to add global variables to a library header?
Yes, by using "extern". Does that mean that all other
declarations should be skipped? That's where I think this
is taking us too far. <br>
</p>
<p>There might be libraries built in non-idiomatic ways,
which expect developers that use them to do other tweaks.
Does it mean that jextract should ignore those? That seems
harsh.</p>
<p>If Jextract finds a global variable definition in a
header file, then it should just treat it at face value,
w/o asking whether it makes sense or not for the header
file to be the way it is. After all, that's why we also
have filters, so that users can exclude symbols based on
additional domain-level knowledge.</p>
<p>So, I think unless a symbol is explicitly marked as
non-exported (e.g. because it's static, or inline, as per
[1]), then jextract should act on it.</p>
<p>Maurizio<br>
</p>
<div>On 06/09/2022 16:08, Manuel Bleichenbacher wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>I mostly agree except for:<br>
</div>
<div dir="ltr">
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<p>It seems to me that all this boil to two rules:<br>
</p>
<p>* we should only generate global var accessor for
"extern" variables<br>
* we should skip static symbols (this is covered
by [1])</p>
</blockquote>
</div>
<div>This opens space for a third case: neither "extern"
nor "static". I think it boils down to a single rule:</div>
<div><br>
</div>
<div>
<ul>
<li>For variables: generate code if declared as
"extern", skip otherwise<br>
</li>
</ul>
</div>
<div><br>
</div>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep 6, 2022
at 4:37 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Some replies inline:</p>
<p><br>
</p>
<div>On 06/09/2022 15:08, Manuel Bleichenbacher
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>I think there are only a few cases:</div>
<div><br>
</div>
<div>
<div>Case 1: extern int i;</div>
<div>It is a declaration and should generate
the lookup for a symbol "i" (plus a method
to get the value). It is already
implemented like this.<br>
</div>
</div>
</div>
</blockquote>
I agree this is the most idiomatic way to do
things.<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div><br>
</div>
Case 2: int i = 4;</div>
<div>It can be skipped as it never appears in
a header file that can be consumed by
multiple compilation units (as it would
generate an error). It's simply not relevant
for jextract, which generates code to access
an existing library.<br>
</div>
<div><br>
</div>
<div>Case 3: int i;</div>
<div>It is the same as case 2 (implicitly
initializes to 0).</div>
</div>
</blockquote>
<p>I also agree that these two seem suspicious.
While in a standalone application, an header
containg symbols like these might make sense, in
a library, this most likely does not make any
sense, as it will create storage in all clients
importing the headerfile.</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
Case 4: int __declspec(selectany) i = 4;<br>
<div>This is a special case *and most likely
Microsoft specific). In theory, the behavior
could be replicated, i.e. a variable or
constant is allocated in native memory and
properly initialized. Since it's a rare
case, I'd rather skip it.</div>
</div>
</blockquote>
<p>Yes, this seems to be special and unique to
Windows. But, I'd rather gate on presence of
"extern", than to obscure __declspec.</p>
<p><br>
</p>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Case 5: static int i = 4;</div>
<div>It is a mixture between case 2 and 4.
It's unlikely to appear in header files
although it does not generate an error. The
behavior could be replicated but I would
spend time on more important things and skip
it for now.</div>
<div><br>
</div>
<div>Case 6: static int i;</div>
<div>It is the same as case 5 (implicitly
initializes to 0).<br>
</div>
</div>
</blockquote>
<p>These seem easier, since declaration says
"static".</p>
<p><br>
</p>
<p>It seems to me that all this boil to two rules:<br>
</p>
<p>* we should only generate global var accessor
for "extern" variables<br>
* we should skip static symbols (this is covered
by [1])</p>
<p>Maurizio</p>
<p>[1] - <a href="https://urldefense.com/v3/__https://github.com/openjdk/jextract/pull/70__;!!ACWV5N9M2RV99hQ!Lbol2roLw9oo_l0GVBCHT9dbxHa0KQGBPdKqbBD18RKaDeuM_ajfqaRi1j0hXELnSrtW6HNUS1X8hErjVv5juTzsQlwnawetxw$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/jextract/pull/70</a><br>
</p>
<p><br>
</p>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep
6, 2022 at 3:29 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi,<br>
I think there's more to it.</p>
<p>On Linux, if I do the following:</p>
<p>```<br>
$ cat foo.h<br>
int i = 4;<br>
```<br>
</p>
<p>and:</p>
<p>```<br>
$ cat foo.c<br>
#include "foo.h"<br>
```<br>
</p>
<p>And then I compile into a shared lib,
the obtained foo.so does have a symbol
for "i" (rightfully so):<br>
</p>
<p>```<br>
$ objdump -T foo.so<br>
<br>
foo.so: file format elf64-x86-64<br>
<br>
DYNAMIC SYMBOL TABLE:<br>
0000000000000000 w D *UND*
0000000000000000 __cxa_finalize<br>
0000000000000000 w D *UND*
0000000000000000
_ITM_registerTMCloneTable<br>
0000000000000000 w D *UND*
0000000000000000
_ITM_deregisterTMCloneTable<br>
0000000000000000 w D *UND*
0000000000000000 __gmon_start__<br>
0000000000004020 g DO .data
0000000000000004 i<br>
<br>
```</p>
<p>E.g. the above header contains a
*definition*, so the compiler will
create storage for it (and add it to the
shared library), and "i" can be looked
up in the shared lib. Conversely, a
global marked as "extern" might or might
not be present in the shared library
(depending on whether it's defined in
one of the associated C/CPP files).<br>
</p>
<p>Maurizio<br>
</p>
<p><br>
</p>
<div>On 06/09/2022 13:57, Manuel
Bleichenbacher wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>There are two indications that
it's definition and not a
declaration:</div>
<div><br>
</div>
<div>- absence of "extern" keyword</div>
<div>- presence of initialization ( =
{ 0x... )</div>
<div><br>
</div>
<div>If it was compiled as C/C++ code,
it would allocate memory in the
current compilation unit while a
declaration would just refer to
something outside the compilation
unit.</div>
<div><br>
</div>
<div>Definitions are rare in header
files as they usually lead to
duplicate symbol errors at link
time. But through the magic of
__declspec(selectany), this is
avoided. </div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Tue, Sep 6, 2022 at 12:11 PM
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex"><br>
On 05/09/2022 16:32, Manuel
Bleichenbacher wrote:<br>
> extern "C" const GUID
__declspec(selectany)
GUID_DEVINTERFACE_USB_DEVICE<br>
> = { 0xA5DCBF10L, 0x6530,
0x11D2, { 0x90, 0x1F, 0x00, 0xC0,
<br>
> 0x4F, 0xB9, 0x51, 0xED } };<br>
><br>
I guess the problem here is the lack
of "dllexport", right?<br>
<br>
But, while dllexport is common, some
libraries can still export symbols <br>
using a .def file [1].<br>
<br>
So, I'm not sure this belongs in the
same category as "static inline", <br>
as it is not possible, just by
looking at the header, to understand
<br>
whether the symbol will be present
or not?<br>
<br>
[1] - <br>
<a href="https://urldefense.com/v3/__https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-def-files?view=msvc-170__;!!ACWV5N9M2RV99hQ!IZ9OZ1GPWyZCLxdiYrnGJewYmhC_BCpH8sxH4Lo2jBtSsuMJPH_j3-vIXRO1emIKo2SMrT3Sj1AqI3daPLx2VOq6oGrpPgb-Xw$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-def-files?view=msvc-170</a><br>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>