[foreign] jextract, distilled

[Maurizio Cimadamore]

On 22/11/2019 20:05, Ty Young wrote:
>
> I understand that this API is more for lower level memaccess usage, 
> but personally I feel like a jextract API could be useful in general 
> for just generating and validating bindings from within Java.
On the contrary - the API is to generate bindings, in whatever shape and 
form you like! Our own jextract tool might use this API to generate low 
level bindings, based on method handles and var handles, but the story 
doesn't have to stop there!
>
>
> Such issues IMO with jextract are:
>
>
> A. Command line can be very long and error-prone as each argument can 
> just blend in with each other in more complicated bindings commands.
Couldn't agree more
>
>
> B. There is no way to generate bindings that are guaranteed to work 
> for all platforms(and sub-platforms, like Linux distros). Generating 
> cross-platform bindings really sucks and sometimes requires a 
> different OS than what the bindings are intended for.
This is more of a distribution problem - e.g. how do we reason about a 
platform-dependent Java artifact; but I agree that jextract doesn't do 
much in that respect. The fact that bindings are platform-dependent is 
unavoidable; in some case the same set of bindings could work across 
multiple platforms/OS, in other case you might have very wild 
differences between the various platforms/OS which can be hard, if not 
impossible to reconcile. This looks like one of those areas where the 
library packager might be in a better position to know what to do in 
terms of providing an uniform API regardless of platforms/OS.
>
>
> C. There is no way(AFAIK) to reverse validate(in other words, after 
> the bindings are generated) the validness of bindings produced by 
> jextract. If the bindings are broken for some reason you can't, as a 
> higher level API dev, provide any information as to why it's broken or 
> even know that the bindings are broken until it's too late.
Writing a tool which compares the native library entry points with the 
set of bindings mechanically generated by jextract, and spot issues 
seems like a very good idea, and pretty much in the real of what would 
be possible to do with this API (combined with some classfile parsing 
handling, e.g. via ASM).
>
>
> Both A and B can be fixed with an interface API for defining the 
> correct way to generate the required bindings and a way to execute 
> that interface implementation from within Java. C could be fixed by 
> providing a way to get the way the bindings where originally generated 
> via meta data.

Not sure I get what you are proposing in terms of A/B (they seem very 
different problems to me). As for C, we have discussed internally to 
maybe sprinkle some metadata on the bindings as a way to keep a 'link' 
to the place where they came from (which is also why the API has this 
Position abstraction - reifying that into, say, a java annotation is 
doable).

Maurizio

>
>
> Could this maybe be added?
>
>
>
> On 11/22/19 10:14 AM, Maurizio Cimadamore wrote:
>> Hi,
>> as we got foreign memory access and foreign function access on a much 
>> more solid footing --- using VarHandle and MethodHandle, respectively 
>> --- I thought it was time to look again at jextract and see how we 
>> could make these new primitives play a more central part in the 
>> extraction process. The writeup below captures my thinking on this 
>> topic:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/jextract_distilled.html
>>
>> Cheers
>> Maurizio
>>
>>