<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 06/09/2022 18:14, Manuel
      Bleichenbacher wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
      
      <div dir="ltr">
        <div>I generally agree that it should not be opinionated. But
          you could easily end up writing a C to Java compiler to make
          it happen.</div>
        <div><br>
        </div>
        <div>It starts with simple things. How do you deal with the
          below variable (which is exportable according to your
          definition)?<br>
        </div>
        <div><br>
        </div>
        <div>int __declspec(selectany) myvar = 4;</div>
        <div><br>
        </div>
        <div>If you allocate native memory, initialize it with 4 and
          generate a getter: great. But an "int" variable is just the
          start. You also have to do it for complex, nested structs and
          unions including initialization with designators like:</div>
      </div>
    </blockquote>
    I don't get what you mean here at all. Jextract won't _allocate_
    anything. It would just look for "myvar", which will be either
    there, or not there, depending on how the lib was compiled. If the
    symbol is not there, that will result in a runtime error when
    looking it up. I think you read way too much into what I said.
    <blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
      <div dir="ltr">
        <div><br>
        </div>
        <div>This is a global function definition. In a header file for
          consumers of a library, it's nonsense, like "int x = 4;". How
          do you treat it in a non-opinionated way? Will you translate
          the function to Java code? Will you also do it for any
          arbitrarily complex function? Where does this end? Can we then
          trick jextract to translate all C code to Java if we just put
          it in a header file?<br>
        </div>
      </div>
    </blockquote>
    This is absolutely not what I was implying. Jextract should not
    "simulate" or "translate" anything. The question is much simpler,
    and is whether, for a given definition seen in an header, jextract
    should generate lookup logic and accessors or not.<br>
    <blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
      <div dir="ltr">
        <div><br>
        </div>
        <div>Whatever you do, make jextract more transparent. And this
          is probably the biggest complaint about the tool in its
          current state. When I started using it, I initially didn't
          succeed at all. In most cases, the tool didn't generate the
          function or struct I was looking for and, worst of all, didn't
          report anything.</div>
        <ul>
          <li>If you explicitly specify "--include-var myar" and
            jextract skips it because it's static, it should tell so.</li>
          <li>If you explicitly specify "--include-struct mystruct" and
            jextract skips it because it's a typedef, it should tell so.</li>
          <li>If you explicitly specify "--include-var eDefault" and
            jextract skips it because it's an enum constant and enum
            constants are only included if you use "--include-macro", it
            should tell so.</li>
          <li>If you explicitly specify "--include-macro XY" and
            jextract skips it because it's function like, it should tell
            so.<br>
          </li>
          <li>If you explicitly specify "--include-function myfunc" and
            jextract can't find such a function, it should tell so.</li>
        </ul>
        <div>These are many of the things I had to learn the hard way,
          and spent (or wasted) time for experiments to derive
          jextract's behavior.<br>
        </div>
      </div>
    </blockquote>
    <p>Fair enough. For the records, should you have any patch which add
      extra diagnostics we'd be happy to accept the contribution!<br>
    </p>
    <p>Note that, as documented in the jextract readme, the option
      "--dump-includes <String>" can be used to dump whatever
      jextract found in the header files. From there you can easily see
      if the symbol you are looking for is there, what is its header,
      and what the symbol kind is. Of course, it's no substitution for
      better diagnostics, but I feel it would minimize the
      experimentation.</p>
    <blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
      <div dir="ltr">
        <div><br>
        </div>
        <div>So before putting any serious effort into implementing
          non-opinionated treatment of things that shouldn't have been
          in header files in the first place, I propose to put the main
          effort into making jextract more transparent. In particular,
          it should report whenever something was requested but skipped
          or not found. And it should provide the reason, possibly
          indicating near misses like "struct" instead of "typedef".<br>
        </div>
      </div>
    </blockquote>
    <p>Again, the suggestion in my email was to stop at what we did
      here:</p>
    <p><a class="moz-txt-link-freetext" href="https://git.openjdk.org/jextract/pull/70">https://git.openjdk.org/jextract/pull/70</a></p>
    <p>No extra crazy logic to be implemented. <br>
    </p>
    <p>Maurizio<br>
    </p>
    <blockquote type="cite" cite="mid:CAA7F5jLUnE3TGc8a4OqHU9an8c7bpthGBe4hd6QXWpiAy_UURg@mail.gmail.com">
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Tue, Sep 6, 2022 at 6:14 PM
          Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px
          0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div>
            <p>Thinking about some more (after some offline internal
              discussion).</p>
            <p>I don't think jextract should be "opinionated" as to how
              global variables are declared in headers. Is there a
              "correct" way to add global variables to a library header?
              Yes, by using "extern". Does that mean that all other
              declarations should be skipped? That's where I think this
              is taking us too far. <br>
            </p>
            <p>There might be libraries built in non-idiomatic ways,
              which expect developers that use them to do other tweaks.
              Does it mean that jextract should ignore those? That seems
              harsh.</p>
            <p>If Jextract finds a global variable definition in a
              header file, then it should just treat it at face value,
              w/o asking whether it makes sense or not for the header
              file to be the way it is. After all, that's why we also
              have filters, so that users can exclude symbols based on
              additional domain-level knowledge.</p>
            <p>So, I think unless a symbol is explicitly marked as
              non-exported (e.g. because it's static, or inline, as per
              [1]), then jextract should act on it.</p>
            <p>Maurizio<br>
            </p>
            <div>On 06/09/2022 16:08, Manuel Bleichenbacher wrote:<br>
            </div>
            <blockquote type="cite">
              <div dir="ltr">
                <div>I mostly agree except for:<br>
                </div>
                <div dir="ltr">
                  <blockquote class="gmail_quote" style="margin:0px 0px
                    0px 0.8ex;border-left:1px solid
                    rgb(204,204,204);padding-left:1ex">
                    <p>It seems to me that all this boil to two rules:<br>
                    </p>
                    <p>* we should only generate global var accessor for
                      "extern" variables<br>
                      * we should skip static symbols (this is covered
                      by [1])</p>
                  </blockquote>
                </div>
                <div>This opens space for a third case: neither "extern"
                  nor "static". I think it boils down to a single rule:</div>
                <div><br>
                </div>
                <div>
                  <ul>
                    <li>For variables: generate code if declared as
                      "extern", skip otherwise<br>
                    </li>
                  </ul>
                </div>
                <div><br>
                </div>
                <div class="gmail_quote">
                  <div dir="ltr" class="gmail_attr">On Tue, Sep 6, 2022
                    at 4:37 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
                    wrote:<br>
                  </div>
                  <blockquote class="gmail_quote" style="margin:0px 0px
                    0px 0.8ex;border-left:1px solid
                    rgb(204,204,204);padding-left:1ex">
                    <div>
                      <p>Some replies inline:</p>
                      <p><br>
                      </p>
                      <div>On 06/09/2022 15:08, Manuel Bleichenbacher
                        wrote:<br>
                      </div>
                      <blockquote type="cite">
                        <div dir="ltr">
                          <div>I think there are only a few cases:</div>
                          <div><br>
                          </div>
                          <div>
                            <div>Case 1: extern int i;</div>
                            <div>It is a declaration and should generate
                              the lookup for a symbol "i" (plus a method
                              to get the value). It is already
                              implemented like this.<br>
                            </div>
                          </div>
                        </div>
                      </blockquote>
                      I agree this is the most idiomatic way to do
                      things.<br>
                      <blockquote type="cite">
                        <div dir="ltr">
                          <div>
                            <div><br>
                            </div>
                            Case 2: int i = 4;</div>
                          <div>It can be skipped as it never appears in
                            a header file that can be consumed by
                            multiple compilation units (as it would
                            generate an error). It's simply not relevant
                            for jextract, which generates code to access
                            an existing library.<br>
                          </div>
                          <div><br>
                          </div>
                          <div>Case 3: int i;</div>
                          <div>It is the same as case 2 (implicitly
                            initializes to 0).</div>
                        </div>
                      </blockquote>
                      <p>I also agree that these two seem suspicious.
                        While in a standalone application, an header
                        containg symbols like these might make sense, in
                        a library, this most likely does not make any
                        sense, as it will create storage in all clients
                        importing the headerfile.</p>
                      <p><br>
                      </p>
                      <blockquote type="cite">
                        <div dir="ltr">
                          <div><br>
                          </div>
                          Case 4: int __declspec(selectany) i = 4;<br>
                          <div>This is a special case *and most likely
                            Microsoft specific). In theory, the behavior
                            could be replicated, i.e. a variable or
                            constant is allocated in native memory and
                            properly initialized. Since it's a rare
                            case, I'd rather skip it.</div>
                        </div>
                      </blockquote>
                      <p>Yes, this seems to be special and unique to
                        Windows. But, I'd rather gate on presence of
                        "extern", than to obscure __declspec.</p>
                      <p><br>
                      </p>
                      <br>
                      <blockquote type="cite">
                        <div dir="ltr">
                          <div><br>
                          </div>
                          <div>Case 5: static int i = 4;</div>
                          <div>It is a mixture between case 2 and 4.
                            It's unlikely to appear in header files
                            although it does not generate an error. The
                            behavior could be replicated but I would
                            spend time on more important things and skip
                            it for now.</div>
                          <div><br>
                          </div>
                          <div>Case 6: static int i;</div>
                          <div>It is the same as case 5 (implicitly
                            initializes to 0).<br>
                          </div>
                        </div>
                      </blockquote>
                      <p>These seem easier, since declaration says
                        "static".</p>
                      <p><br>
                      </p>
                      <p>It seems to me that all this boil to two rules:<br>
                      </p>
                      <p>* we should only generate global var accessor
                        for "extern" variables<br>
                        * we should skip static symbols (this is covered
                        by [1])</p>
                      <p>Maurizio</p>
                      <p>[1] - <a href="https://urldefense.com/v3/__https://github.com/openjdk/jextract/pull/70__;!!ACWV5N9M2RV99hQ!Lbol2roLw9oo_l0GVBCHT9dbxHa0KQGBPdKqbBD18RKaDeuM_ajfqaRi1j0hXELnSrtW6HNUS1X8hErjVv5juTzsQlwnawetxw$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/jextract/pull/70</a><br>
                      </p>
                      <p><br>
                      </p>
                      <blockquote type="cite"><br>
                        <div class="gmail_quote">
                          <div dir="ltr" class="gmail_attr">On Tue, Sep
                            6, 2022 at 3:29 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
                            wrote:<br>
                          </div>
                          <blockquote class="gmail_quote" style="margin:0px 0px 0px
                            0.8ex;border-left:1px solid
                            rgb(204,204,204);padding-left:1ex">
                            <div>
                              <p>Hi,<br>
                                I think there's more to it.</p>
                              <p>On Linux, if I do the following:</p>
                              <p>```<br>
                                $ cat foo.h<br>
                                int i = 4;<br>
                                ```<br>
                              </p>
                              <p>and:</p>
                              <p>```<br>
                                $ cat foo.c<br>
                                #include "foo.h"<br>
                                ```<br>
                              </p>
                              <p>And then I compile into a shared lib,
                                the obtained foo.so does have a symbol
                                for "i" (rightfully so):<br>
                              </p>
                              <p>```<br>
                                $ objdump -T foo.so<br>
                                <br>
                                foo.so:     file format elf64-x86-64<br>
                                <br>
                                DYNAMIC SYMBOL TABLE:<br>
                                0000000000000000  w   D  *UND*   
                                0000000000000000 __cxa_finalize<br>
                                0000000000000000  w   D  *UND*   
                                0000000000000000
                                _ITM_registerTMCloneTable<br>
                                0000000000000000  w   D  *UND*   
                                0000000000000000
                                _ITM_deregisterTMCloneTable<br>
                                0000000000000000  w   D  *UND*   
                                0000000000000000 __gmon_start__<br>
                                0000000000004020 g    DO .data   
                                0000000000000004 i<br>
                                <br>
                                ```</p>
                              <p>E.g. the above header contains a
                                *definition*, so the compiler will
                                create storage for it (and add it to the
                                shared library), and "i" can be looked
                                up in the shared lib. Conversely, a
                                global marked as "extern" might or might
                                not be present in the shared library
                                (depending on whether it's defined in
                                one of the associated C/CPP files).<br>
                              </p>
                              <p>Maurizio<br>
                              </p>
                              <p><br>
                              </p>
                              <div>On 06/09/2022 13:57, Manuel
                                Bleichenbacher wrote:<br>
                              </div>
                              <blockquote type="cite">
                                <div dir="ltr">
                                  <div>There are two indications that
                                    it's definition and not a
                                    declaration:</div>
                                  <div><br>
                                  </div>
                                  <div>- absence of "extern" keyword</div>
                                  <div>- presence of initialization ( =
                                    { 0x... )</div>
                                  <div><br>
                                  </div>
                                  <div>If it was compiled as C/C++ code,
                                    it would allocate memory in the
                                    current compilation unit while a
                                    declaration would just refer to
                                    something outside the compilation
                                    unit.</div>
                                  <div><br>
                                  </div>
                                  <div>Definitions are rare in header
                                    files as they usually lead to
                                    duplicate symbol errors at link
                                    time. But through the magic of
                                    __declspec(selectany), this is
                                    avoided. </div>
                                </div>
                                <br>
                                <div class="gmail_quote">
                                  <div dir="ltr" class="gmail_attr">On
                                    Tue, Sep 6, 2022 at 12:11 PM
                                    Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
                                    wrote:<br>
                                  </div>
                                  <blockquote class="gmail_quote" style="margin:0px 0px 0px
                                    0.8ex;border-left:1px solid
                                    rgb(204,204,204);padding-left:1ex"><br>
                                    On 05/09/2022 16:32, Manuel
                                    Bleichenbacher wrote:<br>
                                    > extern "C" const GUID
                                    __declspec(selectany)
                                    GUID_DEVINTERFACE_USB_DEVICE<br>
                                    >     = { 0xA5DCBF10L, 0x6530,
                                    0x11D2, { 0x90, 0x1F,  0x00,  0xC0,
                                    <br>
                                    >  0x4F,  0xB9,  0x51,  0xED } };<br>
                                    ><br>
                                    I guess the problem here is the lack
                                    of "dllexport", right?<br>
                                    <br>
                                    But, while dllexport is common, some
                                    libraries can still export symbols <br>
                                    using a .def file [1].<br>
                                    <br>
                                    So, I'm not sure this belongs in the
                                    same category as "static inline", <br>
                                    as it is not possible, just by
                                    looking at the header, to understand
                                    <br>
                                    whether the symbol will be present
                                    or not?<br>
                                    <br>
                                    [1] - <br>
                                    <a href="https://urldefense.com/v3/__https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-def-files?view=msvc-170__;!!ACWV5N9M2RV99hQ!IZ9OZ1GPWyZCLxdiYrnGJewYmhC_BCpH8sxH4Lo2jBtSsuMJPH_j3-vIXRO1emIKo2SMrT3Sj1AqI3daPLx2VOq6oGrpPgb-Xw$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-def-files?view=msvc-170</a><br>
                                    <br>
                                  </blockquote>
                                </div>
                              </blockquote>
                            </div>
                          </blockquote>
                        </div>
                      </blockquote>
                    </div>
                  </blockquote>
                </div>
              </div>
            </blockquote>
          </div>
        </blockquote>
      </div>
    </blockquote>
  </body>
</html>