Generating a common interface for multiple platform specific binding

[Maurizio Cimadamore]

On 20/12/2022 15:04, Martin Pernollet wrote:
> Here's my use case in more detail : I've been advised to generate 
> *different binding for different OS* (and maybe version). For OpenGL, 
> this lead me to a glut_h binding for macOS, one for Windows and one 
> for Linux.

Expanding a bit more on this point (apologies if my previous reply was 
not clear).

While we suggest that, in the general case, one might have to run 
jextract multiple times (once per target platform), your mileage my vary.

Consider the jextract code. Jextract relies on libclang, a native 
library. For jextract itself, we just generate a single copy of the 
bindings for both MacOS, Linux and Windows:

https://github.com/openjdk/jextract/tree/master/updateclang

This is possible because libclang has been written in a portable way: 
for instance types that are platform dependent such as `long` have been 
expanded to `long long`, so that they are 64 bits no matter what.

Also, the API makes extensive use of the opaque handle pattern: that is, 
a lot of libclang data structures are built inside the library itself, 
and then returned to clients as opaque by-value structs, whose contents 
point to the memory allocated by the library.

As such, the layouts of the structs defined in libclang is never truly 
important, as a client will typically pass them opaquely to other functions.

The combination of these two factors allow the bindings to work 
seamlessly across platforms.

Now, while I'm not an expert on OpenGL, I seem to recall that OpenGL is 
rather well-behaved when it comes to defining new types - e.g. it 
defines its own primitives (GLint, GLfloat and such). This gives me some 
hope that the OpenGL bindings might, in fact, work well across multiple 
platforms.

One complication with OpenGL is GLUT, which doesn't seem (IIRC) to be 
well supported on some platforms (e.g. MacOS). So, some tricks might be 
necessary there.

I guess the main takeaway there is that IF (big if :-)) the library has 
been written in a portable way, then a single jextract run might be all 
you need (maybe with some filtering added on top to get rid of 
platform-dependent functionalities, which you can do with the jextract 
options already available).

The USB code [1] I pointed you at, on the other hand, is much more 
complex, as that code needs to interface with _different_ libraries to 
do its job on different platforms - so of course that kind of 
unification is not something you can achieve using a tool.

So, as often is the case with native code, we have a spectrum of "badness":

1) well-behaved libraries (like libclang and OpenGL), which are written 
with portability in mind
2) libraries that are mostly portable, but might present minor 
mismatches (e.g. long being 64 vs. 32 bits, some API points not 
available in all platforms, etc.)
3) libraries that are hopelessly platform-specific

I believe jextract proves that (1) already works (**). And for (3), as 
stated, not much can be done.

So the question is whether something should be done for (2) - and how 
much can be done "automagically". This will likely require us to come up 
with a precise definition of the boundaries of what gaps we can actually 
bridge, as (2) is a rather fuzzy bucket.

(**) I can see ways in which even libraries in bucket (1) could fail - 
e.g. this can happen because the symbol lookup would fail to initialize 
because of different library names/paths. This is something that we need 
to address, but also something which requires a much much smaller hammer 
(such as the ability to define your own symbol lookup).

Cheers
Maurizio

[1] - https://github.com/manuelbl/JavaDoesUSB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230103/0f1c7173/attachment-0001.htm>