<div dir="ltr">Here's feedback from the Windows front:<div><br></div><div>I've managed to get it running using the pre-built binaries.</div><div><br></div><div>Using the pre-built binaries, each run outputs more than 1000 warnings related to bitfields even though I'm not using any of them. And many warnings are output several times:</div><div><br></div><div>WARNING: skipping _PROCESS_MITIGATION_REDIRECTION_TRUST_POLICY.EnforceRedirectionTrust: type is bitfield<br></div><div><br></div><div>This makes it nearly impossible to spot any actual problems, and thus it's not just an annoyance.</div><div><br></div><div>And just for completeness:</div><div><br></div><div>About every 30th run on Windows crashes with the below output. This error has existed for as long as I have been using jextract.</div><div><br></div>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>libclang: crash detected during reparsing<br>Re-parsing failed: Crashed<div class="gmail-yj6qo"></div><br class="gmail-Apple-interchange-newline"><div>Regards</div><div>Manuel</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Am Sa., 17. Feb. 2024 um 20:46 Uhr schrieb Jorn Vernee <<a href="mailto:jorn.vernee@oracle.com">jorn.vernee@oracle.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<p>One more thing I realized about the warnings you get for
/usr/include/libudev.h: the structs about which you get warnings
are not being included (looking at your script, there are no
--include-struct for them), so there shouldn't be any need for us
to warn about them either.</p>
<p>We currently check whether things are included first, and then
mark things that aren't as 'skipped'. However, in the later pass
which checks for unsupported types, we also issue warnings for
those skipped elements. I think we should just skip issuing
warnings for things that are marked as skipped by the earlier
pass.<br>
</p>
<p>Jorn<br>
</p>
<div>On 17/02/2024 17:59, Jorn Vernee wrote:<br>
</div>
<blockquote type="cite">
<p>Hey Manuel,</p>
<p>Thanks for giving jextract 22 a try, and sorry for the
frustrating experience.</p>
<p>- The <font face="monospace">NullPointerException </font>is
definitely a bug. Jextract should not just report an exception.
We'll have a closer look at this on Monday, and see if we can
find what code causes the issue. It would help if you could edit
the jextract launch script, and add
`JLINK_VM_OPTIONS=-Djextract.debug=true` (`JLINK_VM_OPTIONS=`
should already be there), and then retry the failed extraction.
This should print out more information, including the entire
stack trace. that's the part I'm interested in.<br>
<br>
<br>
> I don't understand the error message. What does "xxx
depends on yyy which has been excluded" mean? Have I used it
incorrectly? The error message mostly mentions types (and
possibly functions) that I don't use, neither directly nor
indirectly. </p>
<p>It is meant to tell the user that they've included something
that depends on something else which was not included. This
would previously lead to uncompileable code, since
FunctionDescriptors and MemoryLayouts now refer directly to the
layouts defined in struct classes. For example, if I have a
function like `void foo(struct A)`, then the FunctionDescriptor
for `foo` will depend directly on the class we generate for
`struct A`:<br>
<font face="monospace"><br>
FunctionDescriptor.ofVoid(<br>
A.layout()<br>
);</font></p>
<p>If `A` itself is not included, this code would be
uncompileable. So, in the latest version we (try to) issue an
error about that.</p>
<p>- For the 'ERROR's due to missing dependencies you are seeing:
this also looks like a bug. We are currently issuing errors if a
_skipped_ symbols depends on a skipped symbol as well, which
makes no sense. We can just ignore missing dependencies of
skipped things. I've tried to implement a quick fix here [1].
Though, you might be better off using the pre-built binaries for
now, which don't have this dependency scanning behavior in the
first place (i.e the builds at <a href="http://jdk.java.net/jextract" target="_blank">jdk.java.net/jextract</a>)<br>
<br>
<br>
> And why are udev, udev_list_entry, udev_device etc.
skipped? They used to work with jextract 21.</p>
<p>- I took a look at libudev.h, and the warnings about
unsupported types seem correct to me. e.g. `struct udev` does
not have a definition in libudev.h [2], so jextract can not
generate a class for it (since the layout and fields are not
known). That is all the the warning is trying to say. The other
structs about which you get a warning seems to be the same in
that they don't have a definition in that file. (FWIW this is
one of the areas we put a lot of effort into, so it makes sense
that you didn't seen a warning in JDK 21). I think we need to
clarify the error message to say more than "is not supported"
though.<br>
<br>
<br>
> /usr/include/linux/usbdevice_fs.h: The code generated for
the struct usbdevfs_urb does not compile (it refers to a
type usbdevfs_iso_packet_desc, which is missing).</p>
<p>This is actually the situation we were trying to detect with
the ERRORs you encountered. There should have been an error
about <font face="monospace">usbdevfs_iso_packet_desc</font>
being excluded, but it seems like we are not scanning the
element types of arrays at the moment (linked PR tries to fix
this as well).<br>
<br>
<br>
And finally, yeah: we need to update the README on the master
branch to reference JDK 22 instead of 21.<br>
</p>
<p>Thanks for reporting back! While we try to test as much as
possible, some issues slip through the cracks, and are only
found through user feedback like yours.<br>
Jorn<br>
</p>
<p>[1]: <a href="https://github.com/openjdk/jextract/pull/217" target="_blank">https://github.com/openjdk/jextract/pull/217</a><br>
[2]:
<a href="https://github.com/systemd/systemd/blob/main/src/libudev/libudev.h#L20C1-L20C13" target="_blank">https://github.com/systemd/systemd/blob/main/src/libudev/libudev.h#L20C1-L20C13</a><br>
</p>
<div>On 17/02/2024 15:40, Manuel
Bleichenbacher wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Maurizio
<div><br>
</div>
<div>I have tried to upgrade the JavaDoesUSB library (<a href="https://github.com/manuelbl/JavaDoesUSB" target="_blank">https://github.com/manuelbl/JavaDoesUSB</a>)
from JDK 21 to 22, and it was very frustrating. It failed
on macOS, Windows nor Linux. And the problem is jextract.</div>
<div><br>
</div>
<div>(The header files I'm trying the process are operating
system header files.)</div>
<div><br>
</div>
<div><br>
</div>
<div>On macOS, it always crashes:</div>
<div><br>
</div>
<div>FATAL: Unexpected exception
java.lang.NullPointerException: Cannot invoke
"org.openjdk.jextract.Declaration$Constant.name()" because
"<parameter2>" is null occurred<br>
</div>
<div><br>
</div>
<div>I've tried different branches (master, panama, jdk22)
and the pre-built binaries. They all behave the same
except the error message is sometimes shorter and starts
at "Cannot invoke..."</div>
<div><br>
</div>
<div>The full commands can be found here (remove
"--source"): <a href="https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/macos/gen_macos.sh" target="_blank">https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/macos/gen_macos.sh</a></div>
<div><br>
</div>
<div><br>
</div>
<div>On Linux, jextract worked partially. But for several
files, it refused to generate anything or generated code
that didn't compile:</div>
<div><br>
</div>
<div>/usr/include/fcntl.h:</div>
<div>ERROR: stat depends on timespec which has been excluded<br>
ERROR: stat depends on timespec which has been excluded<br>
ERROR: stat depends on timespec which has been excluded<br>
</div>
<div><br>
</div>
<div>/usr/include/libudev.h:</div>
<div>ERROR: __pthread_list_t depends on
__pthread_internal_list which has been excluded<br>
ERROR: __pthread_slist_t depends on
__pthread_internal_slist which has been excluded<br>
ERROR: __pthread_mutex_s depends on
__pthread_internal_list which has been excluded<br>
WARNING: Skipping va_list (type <error: struct
__va_list_tag> is not supported)<br>
WARNING: Skipping __gnuc_va_list (type <error: struct
__va_list_tag> is not supported)<br>
WARNING: Skipping udev (type Declared(udev) is not
supported)<br>
WARNING: Skipping udev_list_entry (type
Declared(udev_list_entry) is not supported)<br>
WARNING: Skipping udev_device (type Declared(udev_device)
is not supported)<br>
WARNING: Skipping udev_monitor (type
Declared(udev_monitor) is not supported)<br>
WARNING: Skipping udev_enumerate (type
Declared(udev_enumerate) is not supported)<br>
WARNING: Skipping udev_queue (type Declared(udev_queue) is
not supported)<br>
WARNING: Skipping udev_hwdb (type Declared(udev_hwdb) is
not supported)<br>
</div>
<div><br>
</div>
<div>sys/epoll.h:</div>
<div>ERROR: __pthread_list_t depends on
__pthread_internal_list which has been excluded<br>
ERROR: __pthread_slist_t depends on
__pthread_internal_slist which has been excluded<br>
ERROR: __pthread_mutex_s depends on
__pthread_internal_list which has been excluded<br>
ERROR: epoll_data_t depends on epoll_data which has been
excluded<br>
ERROR: epoll_event depends on epoll_data which has been
excluded<br>
</div>
<div><br>
</div>
<div>/usr/include/linux/usbdevice_fs.h:<br>
</div>
<div>The code generated for the structusbdevfs_urb does not
compile (it refers to a type usbdevfs_iso_packet_desc,
which is missing).</div>
<div><br>
</div>
<div>The full commands for jextract can be found here. I've
only removed "--source" as it is no longer needed.</div>
<div><a href="https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/linux/gen_linux.sh" target="_blank">https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/linux/gen_linux.sh</a><br>
</div>
<div><br>
</div>
<div>All these files could successfully be processed with
jextract 21.</div>
<div><br>
</div>
<div>I don't understand the error message. What does "xxx
depends on yyy which has been excluded" mean? Have I used
it incorrectly? The error message mostly mentions types
(and possibly functions) that I don't use, neither
directly nor indirectly.</div>
<div><br>
</div>
<div>And why are udev, udev_list_entry, udev_device etc.
skipped? They used to work with jextract 21.</div>
<div><br>
</div>
<div><br>
</div>
<div>On Windows, it completely failed as well. jextract
outputs more than 5000 lines of errors and warnings. Here
is just a random selection:</div>
<div><br>
</div>
<div>ERROR: mbstate_t depends on _Mbstatet which has been
excluded<br>
</div>
<div>ERROR: LIST_ENTRY depends on _LIST_ENTRY which has been
excluded<br>
</div>
<div>ERROR: depends on _M128A which has been excluded<br>
</div>
<div>ERROR: WIN32_FIND_STREAM_DATA depends on
_WIN32_FIND_STREAM_DATA which has been excluded<br>
</div>
<div>ERROR: tagEXTLOGFONTW depends on tagLOGFONTW which has
been excluded<br>
</div>
<div>ERROR: MULTIKEYHELPW depends on tagMULTIKEYHELPW which
has been excluded<br>
</div>
<div>WARNING: Skipping MEM_EXTENDED_PARAMETER.Reserved
(bitfields are not supported)<br>
</div>
<div>WARNING: Skipping Flags.Reserved (bitfields are not
supported)<br>
</div>
<div><br>
</div>
<div>The full command can be found here (remove
"--source"): <a href="https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/windows/gen_win.cmd" target="_blank">https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/windows/gen_win.cmd</a></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>The REAMDE on <a href="https://github.com/openjdk/jextract" target="_blank">https://github.com/openjdk/jextract</a>
could also profit from an update:</div>
<div><br>
</div>
<div>The command line "$ sh ./gradlew
-Pjdk21_home=<jdk21_home_dir> ..." sent me on search
for the branch with the JDK 22 code. But it's just the
README that's outdated.</div>
<div><br>
</div>
<div>And the instruction "Run the Gradle build with a Java
version appropriate for the Gradle version. For example,
Gradle 7.5.1 supports JDK 21." sent me down another rabbit
hole. I have the latest Gradle version installed, which
nicely runs with JDK 21. But it failed anyway. Turns out
that the installed gradle is irrelevant but the gradle
wrapper relevant (see command line above). And the gradle
wrapper is at version 7.3.3. So there is nothing to choose
really. You must use JDK 17 (or even earlier). It has
never worked with JDK 21.</div>
<div><br>
</div>
<div><br>
</div>
<div>I would appreciate some feedback. Are these bugs? Or
has jextract become so much more restrictive?</div>
<div><br>
</div>
<div>Regards</div>
<div>Manuel</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Am Fr., 16. Feb. 2024 um
16:50 Uhr schrieb Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank">maurizio.cimadamore@oracle.com</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
with JDK 22 near us, we have spent some quality time with
jextract, to <br>
make sure the code it generates is as good as it can be
ahead of the <br>
finalizaton of the FFM API. This resulted in several
changes, both in <br>
the implementation (so, invisible to jextract users) and
in the <br>
generated code, as we cleaned up the translation strategy
to better <br>
adhere with the core principles behind the jextract tool.
These changes <br>
are captured in details in this document:<br>
<br>
<a href="https://cr.openjdk.org/~mcimadamore/panama/jextract_changes.html" rel="noreferrer" target="_blank">https://cr.openjdk.org/~mcimadamore/panama/jextract_changes.html</a><br>
<br>
It might be a good time to take the latest jextract for a
spin (using <br>
your favourite C library!) and report back, in case we
missed anything. <br>
You can find the latest sources in this branch:<br>
<br>
<a href="https://github.com/openjdk/jextract/tree/panama" rel="noreferrer" target="_blank">https://github.com/openjdk/jextract/tree/panama</a><br>
<br>
Binary snapshots of this newer version are also available
here (note <br>
that MacOS/Arm64 builds is also supported now):<br>
<br>
<a href="https://jdk.java.net/jextract/" rel="noreferrer" target="_blank">https://jdk.java.net/jextract/</a><br>
<br>
Cheers<br>
Maurizio<br>
<br>
</blockquote>
</div>
</div>
</blockquote>
</blockquote>
</div>
</blockquote></div>