jextract generates downcallHandles where I need upcallStubs

[Sebastian Stenzel]

Hi Maurizio,

thank you for the detailed response! What you write makes a lot of sense. I got confused by the fact that for each of the fields of said struct there is an equivalent downcall (!) method defined in fuse.h as well [1]. You're right, of course, that I'm free to ignore any generated upcall stub if I don't need it.

Regarding the fields in a struct: If I understand you correctly, there is (currently) no way other than to manually create a stub for such methods.

I have no idea how such structs look like for the compiler and if the "type information" for such fields get lost, but is there any chance, jextract can generate a FunctionDescriptor when encountering _any_ function-like expression? This would not only save a lot of time and prevent mistakes, but would fail early at compile time after re-extracting FunctionDescriptors in case of an api-breaking upstream library update.

Thanks again!
Sebastian

[1]: https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L817 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L817>

> On 15. Dec 2020, at 12:31, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> Hi Sebastian,
> I had a look at the header you mention, and I also tried to extract it on my machine.
> 
> What I see in that header is a struct, which defines several function pointer fields.
> 
> Let's focus on a simpler example:
> 
> ```
> cat foo.h
> struct Foo {
>   int (*foo)(float, double);
> };
> ```
> 
> When extracting such an header, jextract will see a struct with one field - only, that field will have a function pointer type. But it's still a field - so jextract will generate getter/setters for that field. You get no downcall at all for this - this is not a function - there is no shared symbol for it anywhere, so talking about downcall here is inappropriate.
> 
> With fuse.h the same thing is happening - and I see a lot of getters/setters being generated for the various pieces of functionality declared in the fuse_operations struct - e.g:
> 
> ```
> public static  MemoryAddress readdir$get(MemorySegment seg) { ... }
> 
> public static void readdir$set(MemorySegment seg, MemoryAddress x) { ... }
> 
> ```
> 
> This means that the "readdir" field of fuse_operation can be get/set using these accessors; to do both get/set you need to pass the memory segment corresponding to fuse_operation (the first argument in the accessor). Note that the type to get/set is a MemoryAddress, which is the readdir function pointer.
> 
> So, the way this works is (I think):
> 
> 1) you create a readdir implementation using upcallStub - this gives you a segment
> 2) you set the above implementation on fuse_operation.readdir, using the set accessor above (to turn the segment into an address, just call the MemorySegment::address method)
> 
> The same structure seems to be present here:
> 
> https://github.com/libfuse/libfuse/blob/fuse_2_9_5/example/fusexmp.c <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/example/fusexmp.c>
> 
> Where a struct fuse_operation is allocated and then all its fields are set to some functions declared in that example. If you want to do the same using Panama you will have to create all the function pointers to set on fuse_operations using CLinker::upcallStub.
> 
> Now, it's true that the function pointer type behind "readdir" takes its own function pointer (filler), but that shouldn't be a concern when creating the upcall for (1) - a function pointer is just some MemoryAddress parameter that the upcall receives (and that the runtime of the libfuse library will provide). And, it is true that, in this case jextract will also generate functional interfaces for e.g. the filler upcall, but you don't have to use them in this case (since the filler function pointer will never be generated by you - but by the library) - of course jextract sees a function pointer and generates the corresponding functional interface, it has no way to know which party will allocate the function pointer.
> 
> I hope this helps.
> 
> Maurizio
> 
> 
> 
> On 15/12/2020 08:50, Sebastian Stenzel wrote:
>> Hi all,
>> 
>> I'm pretty new to this topic and therefore I may misunderstand what is going on here, but I'm pretty certain that jextract swaps downcalls and upcalls in some scenarios:
>> 
>> I jextracted [0] libfuse, which mainly consists of the fuse_operations struct [1], which contains a set of file system operations that need to be implemented by library users. These methods are then called by fuse. I.e. this is a typical upcall scenario (native code calls java code). Or am I mistaken?
>> 
>> However, jextract thinks otherwise: While I get correct downcallHandles for methods declared on "top level" in the .h file, such as fuse_main() [2], everything inside of aforementioned struct is reversed. Take for example readdir() [3], which gets called by fuse, so the library user can list directory contents. For this purpuse, fuse passes a "filler" callback as a parameter to this function. But with jextract will generate a downcallHandle for readdir and an upcallStub for the filler.
>> 
>> Tested with build 16-panama+3-385.
>> 
>> Might it be possible that guessing the "direction" of a call is not yet supported for methods inside of structs? Is it even _possible_ to always predict this direction correctly? If not, is this too much of a restriction and shouldn't it rather be the developers choice what to do with a generated FunctionDescriptor?
>> 
>> Regards,
>> Sebastian
>> 
>> [0]: jextract --source -l fuse -d someDir -t com.example.pkg -C-D_FILE_OFFSET_BITS=64 -C-DFUSE_USE_VERSION=29 --filter fuse.h /usr/local/include/fuse/fuse.h
>> [1]: https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L88 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L88> <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L88 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L88>>
>> [2]: https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L766 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L766> <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L766 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L766>>
>> [3]: https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L304 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L304> <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L304 <https://github.com/libfuse/libfuse/blob/fuse_2_9_5/include/fuse.h#L304>>