analysis of popular FFI frameworks and their layout descriptions

[Stephen Kell]

> Your suggestion of using debugging symbols is clever - and reminds me of
> some tools in the Java toolchain that do a code coverage analysis by looking
> at the bytecode; it is certainly possible in principle, but the main worry
> is that debugging symbols will be almost always not be available in a given
> .so/dll/dynlib library?

This depends a bit on the distribution model of the native code in 
question, and is also platform-dependent. Nowadays, most Unix systems 
are quite good at packaging debug info, meaning in practice it is no 
harder to install than the header files (usually a libfoo-dbg package, 
cf. a libfoo-dev one). Windows and MacOS probably aren't as good, and 
for closed-source libraries there isn't an analogous story at present.

(... but there easily could be. It would be possible to "strip" the 
DWARF so that mention of internals is removed, and only interface 
details remain. The result is still well-formed. So even a closed source 
binary lib could distribute such a description of itself, just as it 
distributes headers. I don't know a tool that does this stripping... 
maybe I should write one! Not sure if PDB is also amenable, but it 
probably is.)

One option to avoid maintaining two entirely separate front-ends would 
be as follows: if you don't have debug info, use the headers as at 
present to have Clang generate a "stub" version of the library, 
including the debug info describing its interface. Then you can proceed 
from the debug info uniformly, whether you're using the "real" or stub 
library.

I guess you might not want this to replace the header file approach 
entirely, but it might be a useful alternative and/or a long-term target 
to migrate to.

> That said, I don't see jextract in opposition with what you describe - the
> important thing is to nail down the specification of the 'native' Java
> interfaces generated by jextract that are required to achieve native
> interconnect (e.g. which annotations should go on them, which types you are
> allowed to use, etc.). Once that's available, I can imagine other tools
> chime in - and having a tool that produces such interfaces from debugging
> symbols can certainly make sense in some context (and, if you are willing to
> spend some time on it, it would provide us very useful feedback on how the
> Panama API is friendly to such 'external' tools).

Yep, a sibling tool to jextract seems right. I might even rustle one up, 
though no promises right now.

I guess one trick in Panama is making sure there is a common core that 
doesn't bake in too much about headers, Clang, C/C++, etc.. Can you 
point me at the best reference to the Panama API? Am I right that it's 
mostly the annotation classes in java.nicl?

Stephen