JExtract - support abstracting of types

[Maurizio Cimadamore]

This is a good question, and I think what you are really asking is, 
really, what is the nature of jextract.

We went back and forth on this a few times, so I think some confusion is 
normal. We started off with the goal of having jextract produce a true 
"good" civilized API, but we quickly had to back off from that goal, 
given that, as you notice, there might be subtle differences between 
extraction runs in different platforms.

After we isolated the foreign memory and the linker API, it became 
clearer that jextract was more of a tool like javah - something which 
would help generating method handles and var handles from an header file.

In other words, what comes out of jextract is NOT an API. And I think 
turning jextract into something which generates a "nice", high level API 
which users will be happy with is an impossible task. Every developer 
might have different notion of what constitutes a "good" API - and we 
cannot bake all these assumptions onto a single tool. That would lead to 
an unavoidable explosion of command line options, which would make the 
tool very hard to approach and use for newcomers. An alternative would 
be that to use some sort of plugin API - this is what we tried to do, 
but to be honest, to date the jextract API hasn't been used much (if at 
all). The main client of the jextract API is jextract itself.

I think where this leaves us is: jextract is a tool which helps you 
wrestling with complex header files; it generates all the "plumbing" for 
you, so that you don't have to worry about matching C signatures etc. 
But, also, jextract doesn't give you a full Java API.

So, if a user just needs to call few function from a random library to 
perform a task, jextract can be the tool which enables that user to do 
just that. For developers looking to provide high level Java bindings of 
a native library (CUDA, Tensorflow, ...) - jextract can _help_ those 
users writing the API they want - but jextract does not, by itself, 
throw the API design process out of the window. In case of supporting 
multiple platforms, an high level API would have to abstract over the 
differences between the various platforms, using Java types that are 
good enough to work in all cases.

But in most cases, this is truly a design exercise (e.g. how do I expose 
the `tm struct` back to a Java user? Is it a class? It is a bunch of 
methods? How are the methods called?) - and an exercise that is best 
left to humans.

I don't exclude that it could be possible, picking a particular shape of 
generated code, to generate something like what you describe - e.g. 
create an abstract class with lots of methods, and have many 
platform-dependent implementation. But that process of "picking a 
particular shape of generated code" means that what comes out of the 
tool will be opinionated and biased. For instance, a setup with abstract 
classes and implementation is less friendly to users who just want to 
call "getpid". Also, is it the goal of jextract to generate the 
hierarchy you describe? Or can that be done separately with another tool 
(which runs jextract and then e.g. runs an annotation processor on the 
generated sources and creates a new class which is implemented by 
jextract bindings)?

To conclude - jextract is a relatively low level tool. It provides low 
level filtering abilities so that you can control how much stuff gets 
generated, and it makes sure that whatever is generated adds as little 
overhead as possible - but that's pretty much it, it makes no attempts 
to generate a nice, or easily distributable Java API. And I think that 
evolving jextract in that direction is very problematic, as history has 
shown, as different users might have wildly different opinions about 
what an optimal API is.

Maurizio


On 28/04/2021 01:24, Radosław Smogura wrote:
> Hi all,
>
> When I worked on my project I noticed I would have to provide bindings between Linux and OSX, both have same API, however final values behinds macros and struct can slightly differ.
>
> This is quite problematic, as due to static nature of generated bindings it’s hard to provide abstraction layer.
>
> I wonder if it would be possible JExtract to support abstraction.
>
> I was thinking about having ability to generate singleton classes where methods will be final instance methods referencing static handles.
>
> With this approach I would be able to use IDE to extract interface or super class and than switch on runtime to implementation I want to use.
>
> What do you think about something like this?
>
> Best regards,
> Rado