Creating a sbt plugin for using jextract to create scala/dotty bindings to C

[Maurizio Cimadamore]

Hi Mark

On 06/11/2019 21:24, Mark Hammons wrote:
> Hi all,
>
> As per my last discussion on this mailing list, I'm using jextract and 
> creating outputted .java files that I'm now parsing using javaparser 
> into an AST of my own making. I'll then use this AST to generate a 
> dotty interface (and maybe a scala interface). I'm doing this for a 
> number of reasons, one of which being that I can make structs 
> available to dotty's typeclass derivation, and more. However, going 
> through the generated java, I have a few questions.
>
> First off, I notice that jextract tends to generate what appears to be 
> dummy structs in header bindings that use but don't define said 
> struct. For example
>
> @NativeStruct("[u64(impl):${wlr_backend_impl}${anon$backend_h$449}(events)](wlr_backend)") 
>
>     public static interface wlr_backend extends Struct<wlr_backend> {
>
>     }
>
> this dummy interface is defined in wlr_output_h.java, but the real 
> wlr_backend struct that is used by functions is defined in 
> wlr_backend_h.java.
Looks like a bug in the logic for reconciling structs declarations and 
definitions, possibly caused by an underlying bug in the libclang API. 
Can you reproduce this with something smaller so that we can take a look?
>
> Also, jextract creates dummy callback interfaces such as:
>
>     @Target(ElementType.TYPE_USE)
>     @Retention(RetentionPolicy.RUNTIME)
>     public static @interface wlr_renderer_create_func_t {
>
>     }
>
> this callback is actually defined as FI1 later in the file, and used 
> as such in pretty much all cases. Do I need to bother having my 
> generate define such interfaces or can I ignore these?
These are annotations, not callback interfaces - they come from typedefs 
- I think these should be safely ignored.
>
> Second, I was wondering about native callbacks and scopes. As I 
> mentioned above, callback types like wlr_renderer_create_func_t are 
> renamed to stuff like FI1 by jextract, which is not particularly 
> helpful when figuring out what is needed by that function. However, 
> even if the original name wlr_renderer_create_func_t was used instead, 
> that doesn't help the user a lot.

Agree - this is a problem we noted as well. Mangling the signature in 
the name would be better - there's a nice post from John on how to 
encode pretty much everything in a VM name:

https://blogs.oracle.com/jrose/symbolic-freedom-in-the-vm

Your solution is of course even better: your language has function types :-)

> I'm thinking of autogenerating inline methods from dotty that will 
> make things a bit easier on users. IE:
>
>     public static interface FI1 {
>
>         public 
> java.foreign.memory.Pointer<wlroots.wlr_renderer_h.wlr_renderer> 
> fn(java.foreign.memory.Pointer<wlroots.egl_h.wlr_egl> $arg0, int 
> $arg1, java.foreign.memory.Pointer<?> $arg2, 
> java.foreign.memory.Pointer<java.lang.Integer> $arg3, int $arg4);
>
>     }
>
>     public void iTakeFI1(Callback<FI1> arg0);
>
> turns into:
>
>     sealed trait FI1 {
>
>         def fn(arg0: Pointer[wlr_egl], arg1: Int, arg2: Pointer[_], 
> arg3: Pointer[Int], arg4: Int): Pointer[wlr_renderer]
>
>     }
>
>     def iTakeFI1(arg0: Callback[FI1]): Unit
>
>     inline def iTakeFI1(arg0: (Pointer[wlr_egl], Int, Pointer[_], 
> Pointer[Int], Int) => wlr_renderer) = {
>
>         iTakeFI1(scope.allocateCallback(arg0: FI1))
>
>     }
>
> or something like that. Do I need to worry about leaking callbacks? 
Unfortunately, yes. As anything allocated with a scope, it remains there 
until the scope is not closed.
> I'm looking at other options too, like renaming FI1 to a 
> representation of the arguments and returns so that it's easier for 
> users to tell at a glance what a function wants.
>
> Finally, I guess my final question is if the code that jextract is 
> outputting is wildly different than the one of the last EA release.
No - should be the same. We didn't change much there as we've been 
focusing more on the other branches for the last few months.
>
> Thanks a lot, once I have my sbt plugin in working order I'll post it 
> here.

Cool work!

Thanks
Maurizio

>
> ~Mark
>
>