April 2023 - jextract-dev

RFR: 7903239: ofAddress factory of function pointer type is wrong for struct returns
by Marko 02 Jan '24

02 Jan '24

When generating the lambda inside the `ofAddress` factory, it is missing the `SegmentAllocator` needed when the function returns a struct. This is [implemented for regular functions](https://github.com/openjdk/jextract/blob/0582eaf1b4cdba95f0ee8c2…, but not for function pointers. ------------- Commit messages: - 7903239: ofAddress factory of function pointer type is wrong for struct returns Changes: https://git.openjdk.org/jextract/pull/58/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=58&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903239 Stats: 47 lines in 3 files changed: 44 ins; 2 del; 1 mod Patch: https://git.openjdk.org/jextract/pull/58.diff Fetch: git fetch https://git.openjdk.org/jextract pull/58/head:pull/58 PR: https://git.openjdk.org/jextract/pull/58

3 11

RFR: Add path independence
by Per Minborg 02 Jan '24

02 Jan '24

This small PR incorporated feedback from the original PR: https://github.com/openjdk/jextract/pull/66 made after it was integrated. ------------- Commit messages: - Add path independence Changes: https://git.openjdk.org/jextract/pull/68/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=68&range=00 Stats: 10 lines in 4 files changed: 2 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jextract/pull/68.diff Fetch: git fetch https://git.openjdk.org/jextract pull/68/head:pull/68 PR: https://git.openjdk.org/jextract/pull/68

5 7

RFR: Add Dockerfile, zero-dep reproducible builds
by Gavin Ray 02 Jan '24

02 Jan '24

As title says, adds a Dockerfile that will fetch all necessary deps and build `jextract` for you on Ubuntu 22.10 It would be nice if some official JDK organization could put this on their Dockerhub, so that users could just do: $ docker run --rm -it openjdk/jextract:latest I've pushed it to my personal Dockerhub, so if anyone would like to test it out, you can use the README directions but prepend `gavinray/` before the image name: $ echo "struct Foo { int a; long b; }; int use_foo(struct Foo foo);" >> sample.h $ docker run --rm -it -v ${PWD}:/tmp/jextract \ gavinray/jextract:latest \ -I /tmp/jextract \ --source \ -l libfoo \ --output /tmp/jextract/generated \ --target-package com.example \ ------------- Commit messages: - Add Dockerfile, zero-dep reproducible builds Changes: https://git.openjdk.org/jextract/pull/74/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=74&range=00 Stats: 75 lines in 2 files changed: 75 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jextract/pull/74.diff Fetch: git fetch https://git.openjdk.org/jextract pull/74/head:pull/74 PR: https://git.openjdk.org/jextract/pull/74

3 6

RFR: Add third-party sample list to README, populate with java-vulkan.
by Michael Ennen 02 Jan '24

02 Jan '24

I made a more complex example (1000 lines, mostly because setting up a boilerplate Vulkan program is super lengthy) of using jextract on Vulkan and Win32 APIs. I think it may be too lengthy and too much of a maintenance burden to commit to the `/samples` directory, so I thought maybe including a section on third-party samples would be better. If this is not something you want to entertain that's totally understandable, but thought I would put it out there a bigger sample that interfaces with a pretty complex API may be helpful to others. ------------- Commit messages: - Add third-party sample list to README, populate with java-vulkan. Changes: https://git.openjdk.org/jextract/pull/93/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=93&range=00 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jextract/pull/93.diff Fetch: git fetch https://git.openjdk.org/jextract pull/93/head:pull/93 PR: https://git.openjdk.org/jextract/pull/93

3 3

GPL license
by Shane Pearlman 08 May '23

08 May '23

What’s the reasoning for licensing a tool like this under the GPLv2? Code generators often need to be modified or adapted for large bindings projects, and the classes in org.openjdk.jextract.impl and org.openjdk.jextract.clang could be quite useful as a starting point. Under the current license, however, I will probably have to roll my own. Even the code generation template classes are under GPLv2, which is enough to prevent me from using jextract generated bindings in my non-GPL projects. Maybe someone’s reading of the license is that it is permissible, but is the uncertainty really necessary? That said, I am excited for the Panama project to deliver what looks to be a very well designed solution to a major, decade-long problem with Java. —Shane

3 6

RFR: 7903463: jextract generates empty padding layouts
by Maurizio Cimadamore 02 May '23

02 May '23

This simple patch fixes a bug where jextract would generate a zero-sized padding layout when handling bitfields. This behavior causes a test failure when running against JDK 21. ------------- Commit messages: - Fix failing test due to zero-size padding layout. Changes: https://git.openjdk.org/jextract/pull/119/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=119&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903463 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jextract/pull/119.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/119/head:pull/119 PR: https://git.openjdk.org/jextract/pull/119

2 2

Re: A very basic HelloWorld with jextract and panama
by Maurizio Cimadamore 25 Apr '23

25 Apr '23

I guess it depends on what you mean by boilerplate. In the link you include the Java code just has a single code to the "sayHello", so boilerplate-wise seems good :-) That said, the goal of Panama is that of (mechanically) binding to native libraries (esp. those written in C) w/o the need of intermediate C code (unlike JNI). That is, what Panama provides is similar to what's been done in other languages, see Python's ctypes library [1]. This approach has been used successfully in a lot of real world libraries and frameworks (see the links in [2]). Given all this, I think that a better "hello world" for Panama would be to call a function in some _existing_ library. One obvious, simple enough, example would be the standard library function "strlen" (the document I shared in my previous emails contains code that does exactly that), which can be done 100% in Java (albeit using a low-level API). Now, it might be that Panama does not 100% satisfy your needs, and that's ok too. Maybe you are after something that is higher-level than what Panama provides. If that's the case, the fact that Panama is low-level is a good thing, because it means that a new breed of frameworks can be built on top, providing higher-level ways to interact with native code, w/o the need to generate JNI glue code. JPassport [3] is one such framework, but I believe (and hope!) that JNR and JNA might also, one day, be rebuilt on top of Panama and take advantage of the performance model provided by it. I hope this clarifies what Panama is really about and what you can expect from it. Finally, note that, while the jextract tool might change, the Foreign Function and Memory API [4] is relatively stable now, and we don't expect it to change much at this point. Cheers Maurizio [1] - https://docs.python.org/3/library/ctypes.html [2] - https://mail.openjdk.org/pipermail/panama-dev/2022-December/018182.html [3] - https://github.com/boulder-on/JPassport [4] - https://openjdk.org/jeps/442 On 25/04/2023 20:17, Rebecca Ahlvarsson wrote: > There is a lot of boilerplate still needed. And I somehow miss the > native method declaration. But I understand this is a work in progress > and It will improve in terms of simplicity. > > See here: > https://stackoverflow.com/questions/76103402/using-jextract-and-panama-to-c… > <https://urldefense.com/v3/__https://stackoverflow.com/questions/76103402/us…> > > -Rebecca > > > On Tue, Apr 25, 2023 at 3:16 PM Maurizio Cimadamore > <maurizio.cimadamore(a)oracle.com> wrote: > > Hi Rebecca, > I believe neither JNA nor JNR support C++, so I'm not 100% what > you mean here. > > You would at the very least need to add "extern C" to your > function declaration, to make sure that no mangling occurs. > > Before I answer further, please clarify what you would like to > achieve - if you really want to call a C++ library using Panama > and jextract, I'm afraid that's not possible at the moment (as is > the case for the other frameworks you mentioned). > > If you are ok adding "extern C" or, if C++ is not really central > to the question you are asking, I'm happy to help further. > > Please also refer to the tutorial here (assuming Java 20): > > https://github.com/openjdk/panama-foreign/blob/78d460f04d93f3df8e8f703e7878… > <https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/blob/…> > > Maurizio > > On 25/04/2023 19:08, Michel Trudeau wrote: >> >> [Moving to the right list] >> >> *From: *code-tools-dev <code-tools-dev-retn(a)openjdk.org> >> <mailto:code-tools-dev-retn@openjdk.org> on behalf of Rebecca >> Ahlvarsson <rahlvarsson(a)gmail.com> <mailto:rahlvarsson@gmail.com> >> *Date: *Tuesday, April 25, 2023 at 9:18 AM >> *To: *code-tools-dev(a)openjdk.org <code-tools-dev(a)openjdk.org> >> <mailto:code-tools-dev@openjdk.org> >> *Subject: *A very basic HelloWorld with jextract and panama >> >> This question is so basic that I must be missing something. Sorry >> if that's a dumb rookie question. >> >> I have the following C++ code: >> >> #include <iostream> >> >> using namespace std; >> >> void sayHello() { >> cout << "Hello, Panama!" << endl; >> } >> >> Great!!! >> >> Now how do I go about calling the "sayHello()" C++ method from >> Java using jextract and panama? >> >> *I can use Linux or Mac, you choose. And I have jextract built on >> both machines.* >> >> That's easy and straightforward with JNI, JNA and JNR. But not >> with Panama? :( >> >> Your help clarifying this issue will be greatly appreciated! >> >> -Rebecca >> > > > -- > -Becky

1 0

Re: A very basic HelloWorld with jextract and panama
by Maurizio Cimadamore 25 Apr '23

25 Apr '23

Hi Rebecca, I believe neither JNA nor JNR support C++, so I'm not 100% what you mean here. You would at the very least need to add "extern C" to your function declaration, to make sure that no mangling occurs. Before I answer further, please clarify what you would like to achieve - if you really want to call a C++ library using Panama and jextract, I'm afraid that's not possible at the moment (as is the case for the other frameworks you mentioned). If you are ok adding "extern C" or, if C++ is not really central to the question you are asking, I'm happy to help further. Please also refer to the tutorial here (assuming Java 20): https://github.com/openjdk/panama-foreign/blob/78d460f04d93f3df8e8f703e7878… Maurizio On 25/04/2023 19:08, Michel Trudeau wrote: > > [Moving to the right list] > > *From: *code-tools-dev <code-tools-dev-retn(a)openjdk.org> on behalf of > Rebecca Ahlvarsson <rahlvarsson(a)gmail.com> > *Date: *Tuesday, April 25, 2023 at 9:18 AM > *To: *code-tools-dev(a)openjdk.org <code-tools-dev(a)openjdk.org> > *Subject: *A very basic HelloWorld with jextract and panama > > This question is so basic that I must be missing something. Sorry if > that's a dumb rookie question. > > I have the following C++ code: > > #include <iostream> > > using namespace std; > > void sayHello() { > cout << "Hello, Panama!" << endl; > } > > Great!!! > > Now how do I go about calling the "sayHello()" C++ method from Java > using jextract and panama? > > *I can use Linux or Mac, you choose. And I have jextract built on both > machines.* > > That's easy and straightforward with JNI, JNA and JNR. But not with > Panama? :( > > Your help clarifying this issue will be greatly appreciated! > > -Rebecca >

1 0

RFR: 7903437: Improve translation strategy for function pointers
by Maurizio Cimadamore 23 Apr '23

23 Apr '23

The current translation stragegy for function pointers is not very efficient. Every time a lambda expression is turned into an upcall stub, a new method handle lookup is performed, which is quite a slow operation. This patch improves the translation strategy so that the lookup is only performed once. For instance, considering the following `typedef`: typedef void (*cb)(int, int); jextract will generate the following functional interface: import static java.lang.foreign.ValueLayout.*; /** * {@snippet : * void (*cb)(int,int); * } */ public interface cb { void apply(int _x0, int _x1); static MemorySegment allocate(cb fi, Arena scope) { return RuntimeHelper.upcallStub(constants$0.cb_UP$MH, fi, constants$0.cb$FUNC, scope); } static cb ofAddress(MemorySegment addr, Arena arena) { MemorySegment symbol = addr.reinterpret(arena.scope(), null); return (int __x0, int __x1) -> { try { constants$0.cb_DOWN$MH.invokeExact(symbol, __x0, __x1); } catch (Throwable ex$) { throw new AssertionError("should not reach here", ex$); } }; } } Where the constants are defined as follows: static final FunctionDescriptor cb$FUNC = FunctionDescriptor.ofVoid( Constants$root.C_INT$LAYOUT, Constants$root.C_INT$LAYOUT ); static final FunctionDescriptor cb_UP$FUNC = FunctionDescriptor.ofVoid( Constants$root.C_INT$LAYOUT, Constants$root.C_INT$LAYOUT ); static final MethodHandle cb_UP$MH = RuntimeHelper.upcallHandle(cb.class, "apply", constants$0.cb_UP$FUNC); static final FunctionDescriptor cb_DOWN$FUNC = FunctionDescriptor.ofVoid( Constants$root.C_INT$LAYOUT, Constants$root.C_INT$LAYOUT ); static final MethodHandle cb_DOWN$MH = RuntimeHelper.downcallHandle( constants$0.cb_DOWN$FUNC ); And, where `RuntimeHelper::upcallHandler` is defined as follows: static MethodHandle upcallHandle(Class<?> fi, String name, FunctionDescriptor fdesc) { try { return MH_LOOKUP.findVirtual(fi, name, fdesc.toMethodType()); } catch (Throwable ex) { throw new AssertionError(ex); } } This allows to perform the method handle lookup only once. This means that when creating an upcall stub, we only have to `bind` the functional interface instance to the method handle, and then create an upcall stub with the resulting handle. Both operations should be fast after the first time, because of caching. ------------- Commit messages: - Remove redundant import - Initial push Changes: https://git.openjdk.org/jextract/pull/109/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=109&range=00 Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903437 Stats: 41 lines in 4 files changed: 28 ins; 2 del; 11 mod Patch: https://git.openjdk.org/jextract/pull/109.diff Fetch: git fetch https://git.openjdk.org/jextract pull/109/head:pull/109 PR: https://git.openjdk.org/jextract/pull/109

4 7

RFR: Backport: 7903449: Jextract generates structs that cannot be compiled
by Jorn Vernee 14 Apr '23

14 Apr '23

Backport https://github.com/openjdk/jextract/pull/117 to the master branch. Patch applied relatively cleanly. I had to change some reference from `Arena` to `SegmentScope` in the generated code, and there was a reference to `MS::reinterpret` that was changed back to `ofAddress`. Additional changes needed: - AddressLayout -> ValueLayout.ofAddress - AddressLayout::withTargetLayout -> AddressLayout::asUnbounded - reimplement ValueLayout::withoutName as helper function ------------- Commit messages: - Backport: 7903449: Jextract generates structs that cannot be compiled Changes: https://git.openjdk.org/jextract/pull/118/files Webrev: https://webrevs.openjdk.org/?repo=jextract&pr=118&range=00 Stats: 1370 lines in 10 files changed: 600 ins; 644 del; 126 mod Patch: https://git.openjdk.org/jextract/pull/118.diff Fetch: git fetch https://git.openjdk.org/jextract.git pull/118/head:pull/118 PR: https://git.openjdk.org/jextract/pull/118

2 2