Some small feedback regading jextract as a user

[Jerven Tjalling Bolleman]

Hi Maurizio,

Thank you for the nice tool and work with your team. The code I was 
working with is even more bleeding edge than your early access release.
So please don't waste too much time into looking into this.

On 2020-11-12 22:23, Maurizio Cimadamore wrote:
> Hi, thanks for the feedback, some responses inline below
> 
> On 12/11/2020 20:47, Jerven Tjalling Bolleman wrote:
>> Dear panama-devs,
>> 
>> I hope this feedback is in the useful category. My feedback is 
>> regarding the jextract tool and ffi. Most of it is trivial but I hope 
>> it is still useful.
>> 
>> No --version or -v option to quickly get the exact version of jextract 
>> (would make future feedback easier to pinpoint).
> Sure - right now it's all very "bleeding edge" but this is
> definitively something to consider.
>> 
>> Assume the jextract user has minimal or no knowledge (like me) of how 
>> the library they want to wrap was built. For example the first error I 
>> ran into was “fatal error: 'cstdarg' file not found” which required 
>> “-C -x -C c++” to be added to the command. This was not intuitive and 
>> certainly not the first thing I tried (I started with trying different 
>> -I combinations. Specifically, I am trying to wrap a rust library with 
>> a header generated by rust cbindgen. So thinking of needing to add 
>> clang options to jextract was not high on my lists of things to do.
> Would be interesting and useful to know which libraries did you try to 
> extract
I was trying to extract a new bioinformatics library.
If you want to waste your time and see what I was trying.
Assume you have the rust tools.


   git clone git at github.com:chfi/rs-handlegraph.git
   cd rs-handlegraph
   cargo build --release
   cd ..

git clone git at github.com:JervenBolleman/rs-handlegraph-ffi.git


>> 
>> Having fixed that I ran into a null pointer exception. i.e.
>> WARNING: Using incubator modules: jdk.incubator.jextract, 
>> jdk.incubator.foreign
>> java.lang.NullPointerException
> Did you get the NPE at extraction time, or at runtime?
At extraction time
In git at github.com:JervenBolleman/rs-handlegraph-ffi.git

   cd rs-handlegraph-ffi
   git checkout 9cf693a0ddb33357f35b02e79f9b3fa848e9cca9
   cargo build
   jextract  -C -x -C c++ -l $(pwd)/target/debug/librs_handlegraph_ffi.so 
-l $(pwd)/../rs-handlegraph/target/release/libhandlegraph.so -t 
rs.handlegraph libhandlegraph-ffi-c.h

Gives a null pointer exception

>> 
>> While we might dislike stack traces. I think a bug report (to myself 
>> regarding the C code) or to you would be easier to work on when you 
>> have a stack trace.
>> 
>> When generating the helloworld example I needed to compile with -fPIC. 
>> Which the document at
>> https://github.com/openjdk/panama-foreign/blob/foreign-jextract/doc/panama_jextract.md#jextract-a-jar-file-for-helloworldh 
>> claims generates a jar file but actually generates a directory with 
>> class files. The example worked fine with clang 10.0.1 but my fedora 
>> 10.2.1 gcc needed -fPIC.
> Yep - gcc needs fPIC - I believe most of the guide is MacOS oriented,
> we need to improve that and add other platforms/compilers.
>> 
>> Regarding the FFI interface, I would love to see more examples 
>> regarding the group layout functionality.
> I believe the tensorflow example has some structs being used - but
> perhaps this is not what you meant?

Given code like this.

parse_gfa_into_hash_graph = 
linker.downcallHandle(lookup.lookup("load_hashgraph").get(),
                 MethodType.methodType(MemorySegment.class, 
MemoryAddress.class, long.class),
                 
FunctionDescriptor.of(MemoryLayout.ofStruct(MemoryLayouts.ADDRESS), 
CLinker.C_POINTER, CLinker.C_LONG));

What should a MemoryLayout.ofStruct look like? Looks like the jextract 
tool should avoid me having to do this. Running into issues
with jextract I tried the manual approach.

>> 
>> Otherwise with jextract I quite quickly ran into error: unknown type 
>> name without much of a suggestion why a certain type was not known. 
>> The libraries added to the command line should have had the types. And 
>> I am still working on this one.
> To be able to help we need (a) the header you were trying to extract
> and (b) the exact exception you are getting. It seems like you are
> hitting a bug of some sorts, so it's defo worth looking into it.

In the git at github.com:JervenBolleman/rs-handlegraph-ffi.git

   git checkout 9cb2fbcc257b5e7665a57a036ec170a0f8c67333
   jextract  -C -x -C c++ -l $(pwd)/target/debug/librs_handlegraph_ffi.so 
-l $(pwd)/../rs-handlegraph/target/release/libhandlegraph.so -t 
rs.handlegraph libhandlegraph-ffi-c.h

WARNING: Using incubator modules: jdk.incubator.foreign, 
jdk.incubator.jextract
java.lang.RuntimeException: libhandlegraph-ffi-c.h:14:3: error: unknown 
type name 'HashGraph'

Regards,
Jerven
>> 
>> That said the tools have lots of promise and please keep working on 
>> them!
> 
> Thanks for giving jextract a try!
> 
> Maurizio
> 
>> 
>> Regards,
>> Jerven
>> 
>> 
>> 
>> 

-- 
Jerven Tjalling Bolleman
SIB | Swiss Institute of Bioinformatics
CMU - 1, rue Michel Servet - 1211 Geneva 4
t: +41 22 379 58 85 - f: +41 22 379 58 58
Jerven.Bolleman at sib.swiss - http://www.sib.swiss