August 2024 - jextract-dev

RFR: 7903783: Jextract should tolerate more CPP constructs when running with xc++
by Maurizio Cimadamore 08 Oct '24

08 Oct '24

This PR fixes a bunch of issues with trying to run jextract in CPP mode, using a `compile_flags.txt` file containing the `-xc++` flag. I've found at least three issues: * extern C clauses are not recursively scanned * the first time we see a cursor with language != C, we stop with an exception (instead of skipping) * typing of enum constant is different in C++ and causes infinite recursion These issues were preventing jextract to work with real world libraries (I tested OpenGL) when `-xc++` was specified. Now jextract will try to parse as much as possible. Unrecognized constructs/languages will be skipped and a diagnostic will be reported by the log. ------------- Commit messages: - Add test - Initial push Changes: https://git.openjdk.org/jextract/pull/256/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=256&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903783 Stats: 198 lines in 8 files changed: 175 ins; 13 del; 10 mod Patch: https://git.openjdk.org/jextract/pull/256.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/256/head:pull/256 PR: https://git.openjdk.org/jextract/pull/256

3 6

Handling non-thread safe libraries due to globals
by Alison Gravley 21 Aug '24

21 Aug '24

I have a C library that isn't thread safe due to globals. Normally this would result in us running multiple applications for each instance we needed, but wondered if there was a another way to achieve this. I might even be wrong, and due to the way we wrote things it will work anyway. To provide a little bit of context we wrote a Java library, call it Device Client Library, that uses this C library to make a single connection. In your application, you create a Device Client object for each connection you need to make. My understanding is this still doesn't give the separated memory space needed for the globals or do the right kind of operating system calls to allow this to work the same way multiple running applications would. Is this correct? Is there maybe something we could modify in the Runtime Helper to get around this limitation so we don't have to have 30-40 applications running?

2 1

Providing your own SymbolLookup / ServiceLoader for SymbolLookup
by some-java-user-99206970363698485155＠vodafonemail.de 21 Aug '24

21 Aug '24

Hello, first of all thank you for your work on the FFM API and jextract! It is really great that interoperability with native libraries becomes easier. One issue I currently have with jextract is that none of its `--library` or `--use-system-load-library` options seem to offer a way for users to provide their own SymbolLookup. Why is this an issue? - The header class generated by jextract and possibly additional bindings classes might be generated by a third-party project, so you cannot easily modify the source code to change the SymbolLookup creation (unless you want to fork that project). - For convenience Java projects often bundle native libraries, extract them to a temporary file and then load them, for example even JavaFX does this (see its `NativeLibLoader` class). This avoids requiring users to configure `java.library.path`. There are probably advantages and disadvantages with this approach, but nonetheless it would be useful if developers had the choice, by providing their own SymbolLookup and specifying the path of the library. - `SymbolLookup#libraryLookup` seems to be currently the only API which allows manually unloading the library (by closing the `Arena`). All other approaches either keep the library loaded until JVM exit, or until the classloader is garbage collected which might be JVM exit as well (?). Especially on Windows this is a problem because it prevents the deletion of the file while the library is loaded, so `File#deleteOnExit` or similar won't have any effect. Ideally a solution would put the choice of how the library is loaded in the hands of the consumers of the generated headers, and not the users of jextract (since they might be a different person, e.g. if a third-party project provides bindings). Otherwise if this is something users of jextract have to manually enable, then most of them will likely not do it. Therefore maybe one approach could be to offer some kind of `SymbolLookupProvider` which is used with a ServiceLoader: ``` interface SymbolLookupProvider { Optional<SymbolLookup> getSymbolLookup(Class<?> caller, String libName); } ``` The `caller` here is whoever calls `SymbolLookup#libraryLookup(String, Arena)`. This might help if someone is unlucky enough to have multiple dependencies which expect a different library of the same name. If possible it would be useful if `System#loadLibrary` could use SymbolLookupProvider as well. And only in case no SymbolLookupProvider provided a SymbolLookup, the existing default logic is used. This would be a bit similar to the existing `ClassLoader#findLibrary`, except that using that method for this purpose seems difficult (you need a custom class loading setup) respectively impossible (when `SymbolLookup#libraryLookup` is used). There are a few open questions though: - `SymbolLookup#libraryLookup(String, Arena)` already provides an Arena. Should that be ignored if a SymbolLookupProvider can provide a lookup, or should the SymbolLookupProvider have the choice to use the given Arena or use its own. But then the guarantee of `SymbolLookup#libraryLookup` that closing the Arena unloads the library might no longer hold. - `System#loadLibrary` seems to not use SymbolLookup internally at the moment, so supporting that would not be easily possible probably. What do you think? Do you have any other ideas for solving the current issues with jextract mentioned above? Was there already an discussion about this or a similar issue in the past? (I tried to search the jextract archives but could not find anything) Kind regards (Maybe the proposal here is not specific to jextract and applies to the FFM API in general. Please let me know if I should forward this mail to panama-dev.)

2 1

RFR: 7903788: preparation towards json configuration for jextract tool
by Athijegannathan Sundararajan 16 Aug '24

16 Aug '24

* adding json skara code into org.openjdk.jextract.json.parser package. Minor changes done - pretty printing, JSONValue is made sealed and access changes (private final fields). * org.openjdk.jextract.json package for JSON interface to the rest of the jextract tool. Main interface is org.openjdk.jextract.json.JSON. parse and toString methods to convert record into JSON String and vice versa. * Other classes are public mainly for testing purposes. build.gradle adjusted with the required --add-exports clauses so that jextract module does not have to export the new json packages. ------------- Commit messages: - 7903788: preparation towards json configuration for jextract tool Changes: https://git.openjdk.org/jextract/pull/257/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=257&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903788 Stats: 5132 lines in 20 files changed: 5132 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jextract/pull/257.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/257/head:pull/257 PR: https://git.openjdk.org/jextract/pull/257

2 24