Growing jextract

Sat Feb 17 14:40:05 UTC 2024

Hi Maurizio

I have tried to upgrade the JavaDoesUSB library (
https://github.com/manuelbl/JavaDoesUSB) from JDK 21 to 22, and it was very
frustrating. It failed on macOS, Windows nor Linux. And the problem is
jextract.

(The header files I'm trying the process are operating system header files.)

On macOS, it always crashes:

FATAL: Unexpected exception java.lang.NullPointerException: Cannot invoke
"org.openjdk.jextract.Declaration$Constant.name()" because "<parameter2>"
is null occurred

I've tried different branches (master, panama, jdk22) and the pre-built
binaries. They all behave the same except the error message is sometimes
shorter and starts at "Cannot invoke..."

The full commands can be found here (remove "--source"):
https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/macos/gen_macos.sh

On Linux, jextract worked partially. But for several files, it refused to
generate anything or generated code that didn't compile:

/usr/include/fcntl.h:
ERROR: stat depends on timespec which has been excluded
ERROR: stat depends on timespec which has been excluded
ERROR: stat depends on timespec which has been excluded

/usr/include/libudev.h:
ERROR: __pthread_list_t depends on __pthread_internal_list which has been
excluded
ERROR: __pthread_slist_t depends on __pthread_internal_slist which has been
excluded
ERROR: __pthread_mutex_s depends on __pthread_internal_list which has been
excluded
WARNING: Skipping va_list (type <error: struct __va_list_tag> is not
supported)
WARNING: Skipping __gnuc_va_list (type <error: struct __va_list_tag> is not
supported)
WARNING: Skipping udev (type Declared(udev) is not supported)
WARNING: Skipping udev_list_entry (type Declared(udev_list_entry) is not
supported)
WARNING: Skipping udev_device (type Declared(udev_device) is not supported)
WARNING: Skipping udev_monitor (type Declared(udev_monitor) is not
supported)
WARNING: Skipping udev_enumerate (type Declared(udev_enumerate) is not
supported)
WARNING: Skipping udev_queue (type Declared(udev_queue) is not supported)
WARNING: Skipping udev_hwdb (type Declared(udev_hwdb) is not supported)

sys/epoll.h:
ERROR: __pthread_list_t depends on __pthread_internal_list which has been
excluded
ERROR: __pthread_slist_t depends on __pthread_internal_slist which has been
excluded
ERROR: __pthread_mutex_s depends on __pthread_internal_list which has been
excluded
ERROR: epoll_data_t depends on epoll_data which has been excluded
ERROR: epoll_event depends on epoll_data which has been excluded

/usr/include/linux/usbdevice_fs.h:
The code generated for the structusbdevfs_urb does not compile (it refers
to a type usbdevfs_iso_packet_desc, which is missing).

The full commands for jextract can be found here. I've only removed
"--source" as it is no longer needed.
https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/linux/gen_linux.sh

All these files could successfully be processed with jextract 21.

I don't understand the error message. What does "xxx depends on yyy which
has been excluded" mean? Have I used it incorrectly? The error message
mostly mentions types (and possibly functions) that I don't use, neither
directly nor indirectly.

And why are udev, udev_list_entry, udev_device etc. skipped? They used to
work with jextract 21.

On Windows, it completely failed as well. jextract outputs more than 5000
lines of errors and warnings. Here is just a random selection:

ERROR: mbstate_t depends on _Mbstatet which has been excluded
ERROR: LIST_ENTRY depends on _LIST_ENTRY which has been excluded
ERROR:  depends on _M128A which has been excluded
ERROR: WIN32_FIND_STREAM_DATA depends on _WIN32_FIND_STREAM_DATA which has
been excluded
ERROR: tagEXTLOGFONTW depends on tagLOGFONTW which has been excluded
ERROR: MULTIKEYHELPW depends on tagMULTIKEYHELPW which has been excluded
WARNING: Skipping MEM_EXTENDED_PARAMETER.Reserved (bitfields are not
supported)
WARNING: Skipping Flags.Reserved (bitfields are not supported)

The full command can be found here (remove "--source"):
https://github.com/manuelbl/JavaDoesUSB/blob/main/java-does-usb/jextract/windows/gen_win.cmd

The REAMDE on https://github.com/openjdk/jextract could also profit from an
update:

The command line "$ sh ./gradlew -Pjdk21_home=<jdk21_home_dir> ..." sent me
on search for the branch with the JDK 22 code. But it's just the README
that's outdated.

And the instruction "Run the Gradle build with a Java version appropriate
for the Gradle version. For example, Gradle 7.5.1 supports JDK 21." sent me
down another rabbit hole. I have the latest Gradle version installed, which
nicely runs with JDK 21. But it failed anyway. Turns out that the installed
gradle is irrelevant but the gradle wrapper relevant (see command line
above). And the gradle wrapper is at version 7.3.3. So there is nothing to
choose really. You must use JDK 17 (or even earlier). It has never worked
with JDK 21.

I would appreciate some feedback. Are these bugs? Or has jextract become so
much more restrictive?

Regards
Manuel

Am Fr., 16. Feb. 2024 um 16:50 Uhr schrieb Maurizio Cimadamore <
maurizio.cimadamore at oracle.com>:

> Hi,
> with JDK 22 near us, we have spent some quality time with jextract, to
> make sure the code it generates is as good as it can be ahead of the
> finalizaton of the FFM API. This resulted in several changes, both in
> the implementation (so, invisible to jextract users) and in the
> generated code, as we cleaned up the translation strategy to better
> adhere with the core principles behind the jextract tool. These changes
> are captured in details in this document:
>
> https://cr.openjdk.org/~mcimadamore/panama/jextract_changes.html
>
> It might be a good time to take the latest jextract for a spin (using
> your favourite C library!) and report back, in case we missed anything.
> You can find the latest sources in this branch:
>
> https://github.com/openjdk/jextract/tree/panama
>
> Binary snapshots of this newer version are also available here (note
> that MacOS/Arm64 builds is also supported now):
>
> https://jdk.java.net/jextract/
>
> Cheers
> Maurizio
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jextract-dev/attachments/20240217/5a9d320d/attachment.htm>