January 2024 - jextract-dev

RFR: 7903649: Field and global variables of array type should have indexed accessors
by Maurizio Cimadamore 01 Feb '24

01 Feb '24

This PR adds support for indexed accessors for struct fields and global variables whose type is an array type. These accessors feature a number of `long` access coordinates which has the same cardinality as that of the underlying array type. For instance, consider the following global variable declaration: int ints[2][3][4]; For this, jextract now emits: private static SequenceLayout ints$LAYOUT() { class Holder { static final SequenceLayout LAYOUT = MemoryLayout.sequenceLayout(2, MemoryLayout.sequenceLayout(3, MemoryLayout.sequenceLayout(4, foo_h.C_INT))); } return Holder.LAYOUT; } private static MemorySegment ints$SEGMENT() { class Holder { static final MemorySegment SEGMENT = foo_h.findOrThrow("ints") .reinterpret(ints$LAYOUT().byteSize()); } return Holder.SEGMENT; } /** * Getter for variable: * {@snippet lang=c : * int ints[2][3][4] * } */ public static MemorySegment ints() { return ints$SEGMENT(); } /** * Setter for variable: * {@snippet lang=c : * int ints[2][3][4] * } */ public static void ints(MemorySegment varValue) { MemorySegment.copy(varValue, 0L, ints$SEGMENT(), 0L, ints$LAYOUT().byteSize()); } private static VarHandle ints$ELEM_HANDLE() { class Holder { static final VarHandle HANDLE = ints$LAYOUT().varHandle(sequenceElement(), sequenceElement(), sequenceElement()); } return Holder.HANDLE; } /** * Indexed getter for variable: * {@snippet lang=c : * int ints[2][3][4] * } */ public static int ints(long index0, long index1, long index2) { return (int)ints$ELEM_HANDLE().get(ints$SEGMENT(), 0L, index0, index1, index2); } /** * Indexed setter for variable: * {@snippet lang=c : * int ints[2][3][4] * } */ public static void ints(long index0, long index1, long index2, int varValue) { ints$ELEM_HANDLE().set(ints$SEGMENT(), 0L, index0, index1, index2, varValue); } If the array element type is a struct, different code needs to be generated. Consider this global variable declaration: struct Point { int x; int y; } points[2][3][4]; This generates the following: private static SequenceLayout points$LAYOUT() { class Holder { static final SequenceLayout LAYOUT = MemoryLayout.sequenceLayout(2, MemoryLayout.sequenceLayout(3, MemoryLayout.sequenceLayout(4, Point.layout()))); } return Holder.LAYOUT; } private static MemorySegment points$SEGMENT() { class Holder { static final MemorySegment SEGMENT = foo_h.findOrThrow("points") .reinterpret(points$LAYOUT().byteSize()); } return Holder.SEGMENT; } /** * Getter for variable: * {@snippet lang=c : * struct Point points[2][3][4] * } */ public static MemorySegment points() { return points$SEGMENT(); } /** * Setter for variable: * {@snippet lang=c : * struct Point points[2][3][4] * } */ public static void points(MemorySegment varValue) { MemorySegment.copy(varValue, 0L, points$SEGMENT(), 0L, points$LAYOUT().byteSize()); } private static MethodHandle points$ELEM_HANDLE() { class Holder { static final MethodHandle HANDLE = points$LAYOUT().sliceHandle(sequenceElement(), sequenceElement(), sequenceElement()); } return Holder.HANDLE; } /** * Indexed getter for variable: * {@snippet lang=c : * struct Point points[2][3][4] * } */ public static MemorySegment points(long index0, long index1, long index2) { try { return (MemorySegment)points$ELEM_HANDLE().invokeExact(points$SEGMENT(), 0L, index0, index1, index2); } catch (Throwable ex$) { throw new AssertionError("should not reach here", ex$); } } /** * Indexed setter for variable: * {@snippet lang=c : * struct Point points[2][3][4] * } */ public static void points(long index0, long index1, long index2, MemorySegment varValue) { MemorySegment.copy(varValue, 0L, points(index0, index1, index2), 0L, Point.layout().byteSize()); } (note that bulk copy is used for the setter). For struct fields, we generate similar code - the only difference is that the var/method handle for accessing the array element is declared in a plain field (instead of using the holder idiom), similarly to what happens with other struct declarations. Note also that, for structs, the indexed accessors must receive an additional `MemorySegment` parameter (the segment struct being accessed). ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/jextract/pull/198/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=198&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903649 Stats: 522 lines in 8 files changed: 502 ins; 9 del; 11 mod Patch: https://git.openjdk.org/jextract/pull/198.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/198/head:pull/198 PR: https://git.openjdk.org/jextract/pull/198

2 23

RFR: Refresh libclang (again)
by Maurizio Cimadamore 26 Jan '24

26 Jan '24

This PR regenerates the libclang extracted classes. The generated classes now contain all the imporovements that have been recently introduced, such as simpler naming scheme, and less generated classes. ------------- Commit messages: - Initial push - Initial push Changes: https://git.openjdk.org/jextract/pull/197/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=197&range=00 Stats: 3610 lines in 14 files changed: 1313 ins; 1134 del; 1163 mod Patch: https://git.openjdk.org/jextract/pull/197.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/197/head:pull/197 PR: https://git.openjdk.org/jextract/pull/197

2 2

RFR: 7903647: typedef of function pointers generates a redundant class
by Maurizio Cimadamore 26 Jan '24

26 Jan '24

This PR fixes few more issues with typedefs. The first issue is that function pointer typedefs generate two classes. The first is generated when we go in OutpoutFactory::visitTypedef (as expected). The second class is generated when we look at the parameters of a function that declares a parameter whose type is the function pointer typedef. The second issue is that when a typedef for an anon struct is typedeffed again, we end up skipping the second typedef, as the underlying type of the typedef is - again- an anoymous struct. The solution to the first issue is to tweak `Utils::getAsFunctionPointer` not to process typedef types. After looking at all usages of this method, it dawned on me that this is actually the behavior that we want: when processing nested types, we only want to emit stuff for declarations that are truly "inline", and not for typedefs of other declarations. The solution to the second issue is to make the skipping logic for anon struct more nuanced, so that we only skip anon declarations that are _nested_ in the very typedef being processed. I've added a test for both conditions, as existing tests were not enough to cover these discrepancies. ------------- Commit messages: - Improve test, by adding typedef uses - Initial push Changes: https://git.openjdk.org/jextract/pull/196/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=196&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903647 Stats: 116 lines in 4 files changed: 112 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jextract/pull/196.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/196/head:pull/196 PR: https://git.openjdk.org/jextract/pull/196

2 2

RFR: 7903644: typedef of anonymous struct generates a redundant class
by Maurizio Cimadamore 26 Jan '24

26 Jan '24

This PR tweaks jextract so that typedefs of anonymous structs only generate one class (that of the struct). This required some changes in both `NameMangler` (to make sure that the special naming rule was applied), and then in `OutputFactory`, to make sure that in such case the typedef class is not emitted. I've tweaked a number of tests which now reflect the "less verbose" jextract output - as such I don't think we need to add further tests. ------------- Commit messages: - Add comment - Initial push Changes: https://git.openjdk.org/jextract/pull/195/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=195&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903644 Stats: 35 lines in 5 files changed: 27 ins; 1 del; 7 mod Patch: https://git.openjdk.org/jextract/pull/195.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/195/head:pull/195 PR: https://git.openjdk.org/jextract/pull/195

2 7

RFR: 7903642: Code generation for variadic function emits redundant casts
by Maurizio Cimadamore 25 Jan '24

25 Jan '24

This PR slightly changes the code generation scheme for variadic calls. Given a variadic function like this: int foo(int count, ...); We now generate the following: /** * Variadic invoker interface for: * {@snippet lang=c : * int foo(int count, ...) * } */ public interface foo { int apply(int count, Object... x1); /** * Invoke the variadic function with the given parameters. Layouts for variadic arguments are inferred. */ static int invoke(int count, Object... x1) { MemoryLayout[] inferredLayouts$ = foo_h.inferVariadicLayouts(x1); return foo(inferredLayouts$).apply(count, x1); } } /** * {@snippet lang=c : * int foo(int count, ...) * } */ public static foo foo(MemoryLayout... layouts) { FunctionDescriptor baseDesc$ = FunctionDescriptor.of( foo_h.C_INT, foo_h.C_INT ); var mh$ = foo_h.downcallHandleVariadic("foo", baseDesc$, layouts); return (int count, Object... x1) -> { try { if (TRACE_DOWNCALLS) { traceDowncall("foo", count, x1); } return (int) mh$.invokeExact(count, x1); } catch(IllegalArgumentException ex$) { throw ex$; // rethrow IAE from passing wrong number/type of args } catch (Throwable ex$) { throw new AssertionError("should not reach here", ex$); } }; } Main changes: * the invoker interface now has the same name as the variadic function (as opposed to `foo$invoker`) * the header class only has one static method, with same name as variadic function (`foo`); this is a factory for the invoker class with same name * the handy Java varargs method that takes objects and infers layout is now moved inside the invoker class (see static `invoke` method) * the main invoker method name has been changed to `apply` * a bug has been fixed where a redundant case was emitted in the static `invoke` method Example of invoker form (see changes to `TestPrintf`): my_sprintf(layouts).apply(s, arena.allocateFrom(fmt), args.length, args); Example of static invoke: my_sprintf.invoke(s, arena.allocateFrom(fmt), args.length, args); ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/jextract/pull/194/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=194&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903642 Stats: 67 lines in 4 files changed: 23 ins; 31 del; 13 mod Patch: https://git.openjdk.org/jextract/pull/194.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/194/head:pull/194 PR: https://git.openjdk.org/jextract/pull/194

2 5

RFR: 7903639: Add javadoc to all jextract generated API points
by Maurizio Cimadamore 24 Jan '24

24 Jan '24

This PR adds simple javadoc text to all the helper functions generated by jextract: * struct allocation methods * struct slicing methods * struct reinterpret methods * function pointer allocation method * function pointer invoke method Following some offline discussions, I've decided to keep the comments very dry and simple (to avoid copying reams and reams of text into multiple classes). Also, the javadoc does not contain any `@param` or `@return` tag. ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/jextract/pull/193/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=193&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903639 Stats: 63 lines in 4 files changed: 48 ins; 2 del; 13 mod Patch: https://git.openjdk.org/jextract/pull/193.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/193/head:pull/193 PR: https://git.openjdk.org/jextract/pull/193

2 4

RFR: 7903637: Use holder classes for string constants
by Jorn Vernee 22 Jan '24

22 Jan '24

Use holder class idiom for string constants (See JBS issue for motivation). The code in `emitConstant` is a bit copy-pasty. We need to strike a balance between giving a clear picture of the generated code by using a single string template as much as possible, and reducing duplication by splitting the templates up and weaving in more control flow. I went with the repeated, but clearer approach here of having 2 separate templates. Let me know if you'd like me to change that. ------------- Commit messages: - Use holder classes for string constants - Share arena for all string constants Changes: https://git.openjdk.org/jextract/pull/191/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=191&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903637 Stats: 24 lines in 1 file changed: 13 ins; 0 del; 11 mod Patch: https://git.openjdk.org/jextract/pull/191.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/191/head:pull/191 PR: https://git.openjdk.org/jextract/pull/191

2 6

RFR: Make method handle accessor private
by Maurizio Cimadamore 22 Jan '24

22 Jan '24

In my previous PR, I forgot to also make the method handle accessor (for native downcalls) `private`. This was discovered when manually eyeballing at jextracted code for windows.h. ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/jextract/pull/188/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=188&range=00 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jextract/pull/188.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/188/head:pull/188 PR: https://git.openjdk.org/jextract/pull/188

2 6

RFR: Use regex match when looking for unnamed struct/union/enum in doc comment
by Jorn Vernee 22 Jan '24

22 Jan '24

Noticed this small issue when running the samples. We try to look for an anonymous struct in the doc comments we generate. Clang will put the file path of the file containing the struct in the declaration string, which causes all kinds of problems, so we filter it out. A regular expression is used for this, however, it is passed to `String::contains`, which doesn't accept a regex. Switch to `String::matches` so it works. - I've added a new test case to TestDocComments - I've also renamed the enclosing folder of this test, since it still had the bug id as a name (while the test itself doesn't) - I also noticed an issue in the TestGlobalStructAccess test. We access a global variable with a struct type, but this variable is not marked as exported, so the test fails on Windows. ------------- Commit messages: - contains -> matches Changes: https://git.openjdk.org/jextract/pull/190/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=190&range=00 Stats: 31 lines in 11 files changed: 27 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jextract/pull/190.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/190/head:pull/190 PR: https://git.openjdk.org/jextract/pull/190

2 3

RFR: 7903636: update jextract sample scripts for --source option removal
by Athijegannathan Sundararajan 22 Jan '24

22 Jan '24

* removing --source option in scripts * removing binary .class generating scripts (as normal scripts do compilation by javac anyway) * removing unnecessary --enable-preview option in run scripts. * updating gradle version to be the latest (8.5) so that we can use JDK 21 for gradle run itself. ------------- Commit messages: - 7903636: update jextract sample scripts for --source option removal Changes: https://git.openjdk.org/jextract/pull/189/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=189&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903636 Stats: 224 lines in 60 files changed: 0 ins; 182 del; 42 mod Patch: https://git.openjdk.org/jextract/pull/189.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/189/head:pull/189 PR: https://git.openjdk.org/jextract/pull/189

2 8