October 2024 - jextract-dev

Supposedly unreachable AssertionError is actually reachable
by some-java-user-99206970363698485155＠vodafonemail.de 21 Oct '24

21 Oct '24

Hello, jextract generated code contains multiple times this pattern when handling the `Throwable` declared by `MethodHandle#invokeExact`: ``` try { mh.invokeExact(...); } catch (Throwable ex) { throw new AssertionError("should not reach here", ex); } ``` The problem is that this actually seems to be reachable: * When for example a closed Arena is used, or a confined Arena is used by a different thread, see for example https://github.com/tree-sitter/java-tree-sitter/issues/44#issuecomment-2402… Maybe in some cases this represents incorrect usage of the jextract generated code, but it seems at least in some cases (e.g. usage from wrong thread) an AssertionError is inappropriate. * Probably when a JVM error occurs So maybe it would be better to generate code like this? ``` try { mh.invokeExact(...); } catch (Error | RuntimeException ex) { throw ex; } catch (Throwable ex) { throw new AssertionError("should not reach here", ex); } ``` Side note: https://bugreport.java.com/bugreport/start_form currently does not support reporting jextract issues; when selecting "Component: Tools Issues", the "Subcomponent" dropdown does not offer "jextract". Is that intended? Kind regards

2 1

RFR: 7903783: Jextract should tolerate more CPP constructs when running with xc++
by Maurizio Cimadamore 08 Oct '24

08 Oct '24

This PR fixes a bunch of issues with trying to run jextract in CPP mode, using a `compile_flags.txt` file containing the `-xc++` flag. I've found at least three issues: * extern C clauses are not recursively scanned * the first time we see a cursor with language != C, we stop with an exception (instead of skipping) * typing of enum constant is different in C++ and causes infinite recursion These issues were preventing jextract to work with real world libraries (I tested OpenGL) when `-xc++` was specified. Now jextract will try to parse as much as possible. Unrecognized constructs/languages will be skipped and a diagnostic will be reported by the log. ------------- Commit messages: - Add test - Initial push Changes: https://git.openjdk.org/jextract/pull/256/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=256&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903783 Stats: 198 lines in 8 files changed: 175 ins; 13 del; 10 mod Patch: https://git.openjdk.org/jextract/pull/256.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/256/head:pull/256 PR: https://git.openjdk.org/jextract/pull/256

3 6

Tooling question: Jextract as a library for programmatic usage
by Tamara Cook 08 Oct '24

08 Oct '24

Hi, Let’s assume that running Jextract can be seen as a step in a complete build process, similar to other code generators e.g. ANTLR, ProtoBuf and alike. In this scenario, the bindings are not checked into source control, they’re build intermediates. This differs a bit from the more traditional approach of manually running Jextract once and committing the bindings. For the first approach, build tools have to integrate with Jextract. I was wondering if it would be realistic to have Jextract also packaged as a Java library for programmatic usage, so build tools or their plugins could use it as implementation dependency. Thanks and greetings Tamara

2 2

Using SymbolLookup#libraryLookup with fallback of SymbolLookup#loaderLookup
by some-java-user-99206970363698485155＠vodafonemail.de 08 Oct '24

08 Oct '24

Hello, it seems currently the various `--library` options of jextract can be summarized as: 1. The library is at an OS-dependent library path, and the name is mapped in an OS-specific way (`--library libname`) 2. The library is at a specific relative or absolute file path (`--library :filepath`); no OS-specific file name mapping is performed, so you limit which OS you support, or you have to omit the `.dll` or `.so` extension to avoid confusion (in case that works) 3. The library is loaded with System#loadLibrary (`--use-system-load-library`); this is similar to (1.) except that it also supports `java.library.path` (related: JDK-8311090), but might have limitations for the dynamic linker (CODETOOLS-7903654) 4. It is assumed that the user has already loaded the library with System#load or System#loadLibrary (no `--library`) A use cases which is not covered is when you expect that the user normally provides the library at an OS-dependent path, but for custom use cases allow them to load the library from a custom location. The context of this use case is not where the user generates the code with jextract themselves, but where they consume some third-party bindings which include jextract generated code. Ideally this would be solved with a user-provided SymbolLookup (see also [1] and CODETOOLS-7903186), but that might not be that easily achievable or complicate the API of jextract-based bindings. A potential solution to address part of the problem might be to have a (opt-in) combination of the library loading variants mentioned above: (1.) respectively (3.) with a fallback of (4.) [^2] The generated code might look then like this: ``` static final SymbolLookup SYMBOL_LOOKUP; static { SymbolLookup lookup = null; try { lookup = SymbolLookup.libraryLookup(System.mapLibraryName("my-library"), LIBRARY_ARENA) .or(SymbolLookup.loaderLookup()); } catch (IllegalArgumentException e) { // ... possibly some opt-in debug logging // Assume the user has already loaded the library manually themselves lookup = SymbolLookup.loaderLookup(); } SYMBOL_LOOKUP = lookup.or(Linker.nativeLinker().defaultLookup()); } ``` (Not sure how it should behave if multiple `--library` arguments are specified; should it give up and use only SymbolLookup#loaderLookup after the first loading failure. Or should it try to load as many of the libraries as possible, but if any of them failed ignore the failed ones and add the SymbolLookup#loaderLookup as fallback lookup.) This way normally users can provide the library at an OS-dependent library path, but if they don't provide it there, it is assumed that they have already loaded it with System#load from a custom path before. This is just a rough general idea. However, if System#load / #loadLibrary is considered legacy and should be avoided, then maybe this approach should be avoided? What do you think? Kind regards [1]: https://mail.openjdk.org/pipermail/jextract-dev/2024-August/001919.html [^2]: Or is this fallback behavior already the current intended behavior, but the current implementation is broken? Because the jextract docs say "When `--library` is specified when generating the bindings, these 2 lookup modes will be used as a fallback." In the current implementation this only applies if the library was successfully loaded with SymbolLookup#libraryLookup, but does not contain the symbols, in which case the fallbacks will be used. However, if loading the library with SymbolLookup#libraryLookup already fails, an IllegalArgumentException is thrown and complete usage of the jextract generated code is not possible; the fallback logic does not apply.

3 4