September 2024 - jextract-dev

RFR: 7903783: Jextract should tolerate more CPP constructs when running with xc++
by Maurizio Cimadamore 08 Oct '24

08 Oct '24

This PR fixes a bunch of issues with trying to run jextract in CPP mode, using a `compile_flags.txt` file containing the `-xc++` flag. I've found at least three issues: * extern C clauses are not recursively scanned * the first time we see a cursor with language != C, we stop with an exception (instead of skipping) * typing of enum constant is different in C++ and causes infinite recursion These issues were preventing jextract to work with real world libraries (I tested OpenGL) when `-xc++` was specified. Now jextract will try to parse as much as possible. Unrecognized constructs/languages will be skipped and a diagnostic will be reported by the log. ------------- Commit messages: - Add test - Initial push Changes: https://git.openjdk.org/jextract/pull/256/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=256&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903783 Stats: 198 lines in 8 files changed: 175 ins; 13 del; 10 mod Patch: https://git.openjdk.org/jextract/pull/256.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/256/head:pull/256 PR: https://git.openjdk.org/jextract/pull/256

3 6

Using SymbolLookup#libraryLookup with fallback of SymbolLookup#loaderLookup
by some-java-user-99206970363698485155＠vodafonemail.de 08 Oct '24

08 Oct '24

Hello, it seems currently the various `--library` options of jextract can be summarized as: 1. The library is at an OS-dependent library path, and the name is mapped in an OS-specific way (`--library libname`) 2. The library is at a specific relative or absolute file path (`--library :filepath`); no OS-specific file name mapping is performed, so you limit which OS you support, or you have to omit the `.dll` or `.so` extension to avoid confusion (in case that works) 3. The library is loaded with System#loadLibrary (`--use-system-load-library`); this is similar to (1.) except that it also supports `java.library.path` (related: JDK-8311090), but might have limitations for the dynamic linker (CODETOOLS-7903654) 4. It is assumed that the user has already loaded the library with System#load or System#loadLibrary (no `--library`) A use cases which is not covered is when you expect that the user normally provides the library at an OS-dependent path, but for custom use cases allow them to load the library from a custom location. The context of this use case is not where the user generates the code with jextract themselves, but where they consume some third-party bindings which include jextract generated code. Ideally this would be solved with a user-provided SymbolLookup (see also [1] and CODETOOLS-7903186), but that might not be that easily achievable or complicate the API of jextract-based bindings. A potential solution to address part of the problem might be to have a (opt-in) combination of the library loading variants mentioned above: (1.) respectively (3.) with a fallback of (4.) [^2] The generated code might look then like this: ``` static final SymbolLookup SYMBOL_LOOKUP; static { SymbolLookup lookup = null; try { lookup = SymbolLookup.libraryLookup(System.mapLibraryName("my-library"), LIBRARY_ARENA) .or(SymbolLookup.loaderLookup()); } catch (IllegalArgumentException e) { // ... possibly some opt-in debug logging // Assume the user has already loaded the library manually themselves lookup = SymbolLookup.loaderLookup(); } SYMBOL_LOOKUP = lookup.or(Linker.nativeLinker().defaultLookup()); } ``` (Not sure how it should behave if multiple `--library` arguments are specified; should it give up and use only SymbolLookup#loaderLookup after the first loading failure. Or should it try to load as many of the libraries as possible, but if any of them failed ignore the failed ones and add the SymbolLookup#loaderLookup as fallback lookup.) This way normally users can provide the library at an OS-dependent library path, but if they don't provide it there, it is assumed that they have already loaded it with System#load from a custom path before. This is just a rough general idea. However, if System#load / #loadLibrary is considered legacy and should be avoided, then maybe this approach should be avoided? What do you think? Kind regards [1]: https://mail.openjdk.org/pipermail/jextract-dev/2024-August/001919.html [^2]: Or is this fallback behavior already the current intended behavior, but the current implementation is broken? Because the jextract docs say "When `--library` is specified when generating the bindings, these 2 lookup modes will be used as a fallback." In the current implementation this only applies if the library was successfully loaded with SymbolLookup#libraryLookup, but does not contain the symbols, in which case the fallbacks will be used. However, if loading the library with SymbolLookup#libraryLookup already fails, an IllegalArgumentException is thrown and complete usage of the jextract generated code is not possible; the fallback logic does not apply.

3 4

ETA for JDK 23 build of jextract?
by Oliver Weiler 26 Sep '24

26 Sep '24

Hi, just wanted to ask if there is some JDK 23 based build of jextract planned? Best regards, Oliver

2 1

Re: The code generated by jextract requires to use --enable-native-access where it should not
by Maurizio Cimadamore 19 Sep '24

19 Sep '24

(adding jextract-dev) Hi Remi, you bring up a good point, one we were also thinking ourselves. Currently, jextract assumes that you want "loose" safety when working with pointers. That is, say you call a binding for a native function returning a `char*`. You can immediately call `MemorySegment::getString` on the segment returned by that binding, because jextract automatically resize all pointer segments to `Long.MAX_VALUE`. Then, since the "unsafe" address layout is saved into a static final field of the main header class, the restricted operation required to create such layout will be called no matter what. This is why you get the restricted warnings. So, there's really two questions here: * should jextract assume that developers want "unsafe" behavior by default? This seemed a good assumption at first (to avoid too many reinterpret calls from clients), but I can some developers being happy with being "more in control" of where/how resizes occur; * assuming we want to preserve the existing "loose" behavior, should we tweak the generated code so that an unsafe pointer layout is only created when needed (as opposed to always, like now) ? Of these, I think the first question is more interesting than the second (which seems small matter of e.g. "making initialization of the unsafe layout more lazy"). What do people think? Maurizio On 18/09/2024 22:01, Remi Forax wrote: > Hello, > the code generated by jextract should be re-arranged so it only requires to use --enable-native-access when the user uses a function that can violate the integrity. > > Currently, if a .h only contains a simple struct, like > struct point { > int x; > int y; > }; > > jextract generates two java classes, point and point_h, point describe the structures and point_h the library, > but because point has a runtime dependency on point_h, a code like > > import point; > import point_h; > > try(var arena = Arena.ofConfined()) { > var p = point.allocate(arena); > point.x(p, 42); > System.out.println(point.x(p)); > } > > requires to use --enable-native-access even if using a layout does not violate the integrity of the application/platform. > > regards, > Rémi

2 2

Jextract Gradle plugin
by Tamara Cook 17 Sep '24

17 Sep '24

Hello, FFM deserves more attention and easier setup, so I contributed a Gradle plugin that automatically downloads, verifies, and extracts a platform-specific Jextract EA build from the download page and generates bindings for given native headers. Maybe it could be mentioned on the download page for EA builds, so interested devs get started without relying on local installation. Plugin: https://plugins.gradle.org/plugin/de.infolektuell.jextract Greetings Tamara Cook

2 1

Mapping the jextract version to a sane SDKMAN! version number
by Oliver Weiler 16 Sep '24

16 Sep '24

Hi! due to high demand we've added jextract to SDKMAN! but struggle to map the releases version number in a sane way :D. E.g. version 22+5-33. Would it make sense to map it a version like 22.ea.5 ? Is the +33 really relevant? Best regards, Oliver Weiler (SDKMAN! Core Contributor)

2 1

RFR: Copy powershell script in make build as well
by Jorn Vernee 10 Sep '24

10 Sep '24

I noticed that the powershell script we added is only being copied to the output directory as part of the gradle build. This PR makes it so we also copy it in the make build. ------------- Commit messages: - copy ps1 script in make build Changes: https://git.openjdk.org/jextract/pull/260/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=260&range=00 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jextract/pull/260.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/260/head:pull/260 PR: https://git.openjdk.org/jextract/pull/260

2 2

RFR: Fix issue with missing JSON classes in make build
by Jorn Vernee 09 Sep '24

09 Sep '24

Currently, the `make` build is broken, since we added a new package, but the make build doesn't build the source files in that package. This patch adds compilation of the new sources, and adds the missing `--add-exports` flags when running tests. Instead of listing each package in the javac command, I've introduced a separate call to `find` to find all the sources, so that, if we add another package in the future, we won't have the same problem again. ------------- Commit messages: - Use find - Fix issue with missing JSON classes in make build Changes: https://git.openjdk.org/jextract/pull/259/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=259&range=00 Stats: 9 lines in 2 files changed: 4 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jextract/pull/259.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/259/head:pull/259 PR: https://git.openjdk.org/jextract/pull/259

3 5

RFR: Add support for Linux/AArch64
by Jorn Vernee 09 Sep '24

09 Sep '24

Currently we don't support jextract on Linux/AArch64, but have had some request to add support. This PR contains changes to make that work. I've added a build script for the libclang bundle, which is loosely based on the old build script we had in the JDK repo, that can be used to create a libclang bundle. (and which I did in fact use to create the libclang bundle for aarch64). I've only tested this script on Linux/AArch64, but have included configurations for mac/linux x64/aarch64 as well. ------------- Commit messages: - DEVKIT_HOME -> DEVKIT_DIR - Drop executable permission - Tweak build script - Add some more documentation to build script - fix gradle build - fix failing test - Fix build script - Add libclang build script - Add aarch64 as make target platform Changes: https://git.openjdk.org/jextract/pull/258/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=258&range=00 Stats: 176 lines in 5 files changed: 173 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jextract/pull/258.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/258/head:pull/258 PR: https://git.openjdk.org/jextract/pull/258

2 3