RFR: Add jextract guide [v5]

Nir Lisker nlisker at openjdk.org
Fri Apr 12 19:41:56 UTC 2024 

On Wed, 10 Apr 2024 14:34:25 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

>> Add a comprehensive jextract guide under a new `doc/` folder.
>> 
>> This is meant as a comprehensive guide about the features of jextract and the code that it generates (including both examples from header files, the code that jextract generates, and corresponding Java user code). This is a first cut, and I'm anticipating quite a bit of comments.
>> 
>> Some sections of the readme have been moved to the guide (with minor edits here and there), and replaced by a link in the readme.
>
> Jorn Vernee has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Phrasing
>   
>   Co-authored-by: Maurizio Cimadamore <54672762+mcimadamore at users.noreply.github.com>

An excellent guide. I read through all of it as someone who already used jextract and left some minor grammar comments that I found to help with the flow, but I would like to point out something more confusing with the structure.

The first part is about running jextract, including a short review of an example, followed by an explanation about loading libraries. It looks like this section explains the arguments/options passed to the tool, but it skips the discussion about the macros option (appears much later in Preprocessor Definitions) and other options.

The second part is about the code that is generated, but then there is an explanation about `--header-class-name` and `--target-package`, which should be part of the previous part. I would expect that by the time the user gets to learning about the generated Java code, they would know what commands generated it.

Also:
* Did you leave out "Bitfields" on purpose?
* Can a ToC be generated?

doc/GUIDE.md line 11:

> 9: archive of native functions and global variables. The user then has to look up the functions
> 10: they want to call using a `SymbolLookup`, and finally _link_ the functions by using the
> 11: `Linker::downcallHandle` method. Additionally, a client may need to create function pointer for

pointer -> pointers

doc/GUIDE.md line 18:

> 16: 
> 17: This guide shows how to run the jextract tool, and how to use the Java code that it generates.
> 18: The samples under [`samples`](samples) direcotry are also a good source of examples.

"under **the** samples directory"

doc/GUIDE.md line 29:

> 27: library called `mylib` stored at `/path/to/mylib` that has a directory `/path/to/mylib/include`
> 28: where the header files of that library are stored. And let's say that we have a shell open
> 29: in the root directory of the Java project we're working on, which has an `src` source

"a `src`" probably.

doc/GUIDE.md line 30:

> 28: where the header files of that library are stored. And let's say that we have a shell open
> 29: in the root directory of the Java project we're working on, which has an `src` source
> 30: directory corresponding to,the root package. A typical way to run jextract would be like

Space after the comma.

doc/GUIDE.md line 48:

> 46: - `--include-dir /path/to/mylib/include` specifies a header file search directory, which
> 47:    is used to find header files included through `#include` in the main header file.
> 48: - `--output src` specifies the root directory for the output. This matches to root package

"matches **the** root"

doc/GUIDE.md line 51:

> 49:   of the project's source directory.
> 50: - `--target-package org.jextract.mylib` specifies the target package to which the generated
> 51:   classes and interfaces will belong. (note that jextract will automatically create the

note -> Note

doc/GUIDE.md line 72:

> 70: to load libraries specified by `<libspec>`. If `<libspec>` denotes a library name, the
> 71: name is then mapped to a platform dependent name using [`System::mapLibraryName`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#mapLibraryName(java.lang.String)).
> 72: This means for instance that on Linux, when specifying `--library mylib` the bindings will

Commas help here: "This means, for instance,"

doc/GUIDE.md line 75:

> 73: On Mac the relevant environment variable is `DYLD_LIBRARY_PATH`, and on Windows the variable is `PATH`.
> 74: Though, for the latter the overall library search mechanism is entirely different (described [here](https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order)).
> 75: When using the HotSpot JVM, the `-Xlog:library` option can als be use to log where the JVM is trying to load a library from,

als -> also
use -> used

doc/GUIDE.md line 80:

> 78: and on Windows the variable is `PATH`. Though, for the latter the overall library search
> 79: mechanism is entirely different (described [here](https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order)).
> 80: When using the HotSpot JVM, the `-Xlog:library` option can als be use to log where the JVM

als -> also
use -> used

doc/GUIDE.md line 84:

> 82: 
> 83: The `<libspec>` argument of the `--library` option can either be a library name, such as,
> 84: `mylib` which will, be mapped to a platform specific name using [`System::mapLibraryName`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#mapLibraryName(java.lang.String)), or a path to a library file (either relative or

Comma after `mylib` instead of after `will`.

doc/GUIDE.md line 84:

> 82: 
> 83: The `<libspec>` argument of the `--library` option can either be a library name, such as,
> 84: `mylib` which will, be mapped to a platform specific name using [`System::mapLibraryName`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#mapLibraryName(java.lang.String)), or a path to a library file (either relative or

This was already mentioned in the previous paragraph.

doc/GUIDE.md line 85:

> 83: The `<libspec>` argument of the `--library` option can either be a library name, such as,
> 84: `mylib` which will, be mapped to a platform specific name using [`System::mapLibraryName`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#mapLibraryName(java.lang.String)), or a path to a library file (either relative or
> 85: absolute) if `<libspec>` is prefixed with the `:` character, such as `mylib.dll`.

Shouldn't the example then be `:mylib.dll`?

doc/GUIDE.md line 106:

> 104: some examples of how to use the generated Java code.
> 105: 
> 106: Most of the code that jextract generates will be available through a single class. By default

"By default,"

doc/GUIDE.md line 126:

> 124: ```
> 125: 
> 126: Where `org.mypackage` is the package into which jextract put the generates source files

"puts the generated"

doc/GUIDE.md line 134:

> 132: builtin C types.
> 133: 
> 134: The latter import statement imports all the other classes the jextract generates, which

"that jextract generates"

doc/GUIDE.md line 135:

> 133: 
> 134: The latter import statement imports all the other classes the jextract generates, which
> 135: includes: classes representing structs or unions, function types, and struct or union

includes -> include

doc/GUIDE.md line 164:

> 162: `short,` `int`, `long long`, `float`, `double` `long`, and `long double`. Additionally,
> 163: there is a `C_POINTER` layout which represents the layout for any C pointer type (such as
> 164: `T*`). Note that these layouts are platform dependent, depending on the platform that

"platform dependent, depending on the platform"

This is somewhat redundant. Just "these layouts depend on" maybe, or some other phrasing.

doc/GUIDE.md line 167:

> 165: jextract runs on. For instance, since these constants were generated on Windows, the
> 166: `long` type has the same layout as the Java `int` type, indicating a 32-bit value, and the
> 167: `long double` type has the same layout as the Java `double` type. (note that the latter is

note -> Note

doc/GUIDE.md line 181:

> 179: ```
> 180: 
> 181: Jextract will generate the following set of methods for this function, in the main header

function, -> function

doc/GUIDE.md line 185:

> 183: 
> 184: ```java
> 185: // mylib_h.java

Assuming the class name options wasn't specified (if you want to mention that, though not important).

doc/GUIDE.md line 197:

> 195: call the C function (1). Besides that, there are also several accessors that return
> 196: additional meta-data for the method: the function's address (2), the function descriptor
> 197: (3), and the method handle returned by the FFM linker (4), which is used to implement the

I would add inline links for convenience, like:  `[function's address](MemorySegment)`, `[function descriptor](FunctionDescriptor)` and `(method handle)[MethodHandle]`

doc/GUIDE.md line 200:

> 198: static wrapper method (1).
> 199: 
> 200: The parameter types and return type of this method depend on the carrier types of the

Is the reader supposed to know what a "carrier type" is? Can a link be given to an explanation if one exists in FFM maybe?

doc/GUIDE.md line 211:

> 209: // mylib.h
> 210: 
> 211: int bar;

Maybe mention what happens if the definition includes an assignment, `int bar = 4;`, if it matters.

doc/GUIDE.md line 269:

> 267: not supported by jextract. For function-like macros, alternatives include re-writing the
> 268: code inside the macro in Java, using the FFM API, or writing a small C library which wraps
> 269: the function-like macro in a proper exported C function, that can then be linked against

"function," -> "function"

doc/GUIDE.md line 348:

> 346: 
> 347: For working with arrays of structs, we can use the `allocateArray` method which accepts an
> 348: additional element count, indicating the length of the array:

Comma before "which" instead of after "count".

doc/GUIDE.md line 358:

> 356: 
> 357:     for (int i = 0; i < arrLen; i++) {
> 358:         MemorySegment element = Point.asSlice(point, i);

Never used this method, but I assume it should be `asSlice(points, i)`.

doc/GUIDE.md line 369:

> 367: In the above example, the `asSlice` method is used to _slice_ out a section of
> 368: the array, which corresponds to a single `Point` struct element. This method
> 369: can be used to access individual elements of the `points` array, when given

"array," -> "array"

doc/GUIDE.md line 438:

> 436: we received from native code.
> 437: 
> 438: For instance, let's say we have a function that accepts an instance of the `callback_t`

For clarity, I would say "have a native function".

doc/GUIDE.md line 456:

> 454: try (Arena arena = Arena.ofConfined()) {
> 455:     MemorySegment cb = callback_t.allocate((a, b) -> a * b, arena);
> 456:     int result = call_me_back(cb);

Where is `call_me_back` defined?

doc/GUIDE.md line 468:

> 466: can no longer be called).
> 467: 
> 468: Additionally, we can using the `callback_t::invoke` method invoke an instance of

using -> use

doc/GUIDE.md line 485:

> 483: 
> 484: The `get_callback` function returns an instance of `callback_t`, which is a function pointer
> 485: pointing to the native `mult` function. We can call `callback_t` instance that `get_callback()`

"call **the** `callback_t` instance"

doc/GUIDE.md line 486:

> 484: The `get_callback` function returns an instance of `callback_t`, which is a function pointer
> 485: pointing to the native `mult` function. We can call `callback_t` instance that `get_callback()`
> 486: return in Java using the `invoke` method in the `callback_t` class that jextract generates

return -> returns

doc/GUIDE.md line 534:

> 532: ```
> 533: 
> 534: Jextract doesn't generates a regular method, but a _class_, which represents the invoker:

generates -> generate

doc/GUIDE.md line 576:

> 574: ### Typedefs
> 575: 
> 576: As mentioned before: typedefs are either translated as a `static final` memory layout fields

before: -> before,

doc/GUIDE.md line 603:

> 601: ```
> 602: 
> 603: The `MyPoint` `typedef` on the other hand is a typedef for a struct, so it is translated

"`typedef` on the other hand" -> "`typedef`, on the other hand,"

doc/GUIDE.md line 618:

> 616: 
> 617: C allows variable declarations to have an inline anonymous type. Jextract handles in
> 618: particular cases where a struct's field has an inline type specially. For instance, if we

"Jextract handles in particular cases where a struct's field has an inline type specially."

I don't understand this sentence. Does Jextract handle nested types only when a struct's field has an inline type?

doc/GUIDE.md line 634:

> 632: 
> 633: Jextract generates a _nested_ struct and function pointer class for the `bar` and `cb`
> 634: fields, _inside of_ the class it generates for the `Foo` struct itself:

fields, -> fields

doc/GUIDE.md line 713:

> 711: 
> 712: The include options in this file can then be edited down to a set of symbols that is
> 713: desired, for instance using other command line tools such as `grep` or `Select-String`,

instance -> instance,

doc/GUIDE.md line 800:

> 798: 
> 799: The value of these macros also affects the behavior of jextract. Therefore, jextract
> 800: supports setting macro values on the command line using the `-D` or

Perhaps mentioning the distinctions between these macros and the ones described in the Constants section more specifically will be helpful?

-------------

PR Review: https://git.openjdk.org/jextract/pull/231#pullrequestreview-1989727538
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562492438
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562494228
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562499511
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562495661
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562498454
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562500829
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562503617
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1558074831
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562505421
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562506125
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562510865
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562511595
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562515695
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562517992
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562519931
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562520377
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562532950
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562534017
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562536552
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562538150
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562544444
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562548168
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562583132
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562620372
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562631106
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562634105
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1562635428
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563001201
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563006011
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563008874
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563038152
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563038500
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563049288
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563054342
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563069603
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563074084
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563075034
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563083141
PR Review Comment: https://git.openjdk.org/jextract/pull/231#discussion_r1563096318

More information about the jextract-dev mailing list