Generating a common interface for multiple platform specific binding

Fri Jan 6 11:15:08 UTC 2023

Really good progress!

On 06/01/2023 10:31, Martin Pernollet wrote:
> Hi everyone,
>
> Thank you a lot for your messages. I got good news to share :
>
>   * First, I have been able to render the OpenGL teapot /inside/ a
>     Swing and AWT window \o/ Please see the video attached or even try
>     it on your macOS here
>     <https://urldefense.com/v3/__https://gitlab.com/jzy3d/panama-gl/-/blob/main/panama-gl-wrappers-macos/src/main/java/demos/panamagl/macos/swing/DemoTeapot_Onscreen_macOS.java__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYeAzE3yAg$>.
>   * Second, I confirm that I running bindings generated for macOS
>     10.15 runs on 10.12.
>
Good to know
>
> The pattern I am going toward for a common interface
>
>   * Have generated bindings, generated once and deployed asstandalone
>     maven artifacts
>     <https://urldefense.com/v3/__https://gitlab.com/jzy3d/panama-gl-bindings__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYfNbASRsQ$>.
>   * Have wrappers of these bindings
>     <https://urldefense.com/v3/__https://gitlab.com/jzy3d/panama-gl/-/blob/main/panama-gl-wrappers-macos/src/main/java/opengl/macos/GL_macOS_10_15_7.java__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYcr23qqcA$>
>     implementing a common interface
>     <https://urldefense.com/v3/__https://gitlab.com/jzy3d/panama-gl/-/blob/main/panama-gl-core/src/main/java/opengl/GL.java__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYeHQ6CP_A$>.
>
>
Do you think that that the same set of bindings will not work across OS? 
Why?
>
> I will generate the common interface and the wrappers. I am thinking 
> of using Javaassist 
> <https://urldefense.com/v3/__http://www.javassist.org/__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYdt6zOTJQ$>. 
> I am interested if you have other suggestions.

Other options, if you are into classfile manipulation are:

* ASM (perhaps the more popular, what the JDK itself uses): 
https://asm.ow2.io/
* the brand new JDK classfile API: 
https://mail.openjdk.org/pipermail/classfile-api-dev/2022-June/000000.html

>
> To answer the "well-behave" nature of OpenGL
>
>   * Yes, there are types to hide differences per platform. OpenGL is
>     an API so one can expect to find the same headers everywhere.
>   * BUT OpenGL has dynamic loading capabilities to allow getting
>     features on computers that have hardware (GPU) supporting some
>     kind of rendering, and others not able to do the same, still
>     running on the same OS but not the same GPU. That is a very
>     interesting case for Panama. In that case I'll have to rely on
>     OpenGL API methods that load functions only if they are supported
>     on the host computer.
>
I think you refer to GL extensions - aren't there like opaque function 
pointers whose existence you can query dynamically? I think stuff like 
this can be made to work well with Panama. Other projects such as the 
Fuse Panama port are using similar function-pointer based approaches:

https://github.com/cryptomator/jfuse

That said, the fact that there are extensions, doesn't mean that the 
jextract bindings would not be portable across OSs ? But more that you 
need to write some kind of support to discover and invoke the extension 
functions? (wild guess here)

>
> A bit more about OpenGL
>
>   * Sometime I have to ask the OS to do things such as executing java
>     runnable on the main macOS thread. For now this requires I keep
>     using manually generated binding from JOGL, allowing to use
>     Objective C code. I don't think JExtract can do this
>     <https://urldefense.com/v3/__https://github.com/manuelbl/JavaDoesUSB/issues/5__;!!ACWV5N9M2RV99hQ!Kml55Ybj-rpsn3N8SEjStiscqW8Ma0NJklvvm_C6lYscfvg2Y8qEDk_fJLbitlLqV4ZvtoW5kfz9q_NBqkWJJWQNXyD0UYfPT9ePYA$>.
>
Does passing this option " -XstartOnFirstThread" to the JVM works? IIRC, 
it was added for things like these. I think I saw this here:

https://foojay.io/today/project-panama-for-newbies-part-3/

Which does have an opengl tutorial for MacOs

>   * There are several additional libraries to GL and GLUT, handling OS
>     specific things (e.g. CGL for mac, WGL for windows, GLX for
>     linux). I'll most probably bind to these rather than glut.
>
Ok - so this part is not portable, and you will need different bindings 
for each OS, which seems ok.
>
> The JavaDoesUSB project is really great and its developer has been 
> really nice by providing help on JExtract, thank you for the advice!
Maurizio
>
> ------- Original Message -------
> Le mardi 3 janvier 2023 à 18:51, Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> a écrit :
>
>>
>> On 20/12/2022 15:04, Martin Pernollet wrote:
>>> Here's my use case in more detail : I've been advised to generate 
>>> *different binding for different OS* (and maybe version). For 
>>> OpenGL, this lead me to a glut_h binding for macOS, one for Windows 
>>> and one for Linux.
>>
>> Expanding a bit more on this point (apologies if my previous reply 
>> was not clear).
>>
>> While we suggest that, in the general case, one might have to run 
>> jextract multiple times (once per target platform), your mileage my vary.
>>
>> Consider the jextract code. Jextract relies on libclang, a native 
>> library. For jextract itself, we just generate a single copy of the 
>> bindings for both MacOS, Linux and Windows:
>>
>> https://github.com/openjdk/jextract/tree/master/updateclang
>>
>> This is possible because libclang has been written in a portable way: 
>> for instance types that are platform dependent such as `long` have 
>> been expanded to `long long`, so that they are 64 bits no matter what.
>>
>> Also, the API makes extensive use of the opaque handle pattern: that 
>> is, a lot of libclang data structures are built inside the library 
>> itself, and then returned to clients as opaque by-value structs, 
>> whose contents point to the memory allocated by the library.
>>
>> As such, the layouts of the structs defined in libclang is never 
>> truly important, as a client will typically pass them opaquely to 
>> other functions.
>>
>> The combination of these two factors allow the bindings to work 
>> seamlessly across platforms.
>>
>> Now, while I'm not an expert on OpenGL, I seem to recall that OpenGL 
>> is rather well-behaved when it comes to defining new types - e.g. it 
>> defines its own primitives (GLint, GLfloat and such). This gives me 
>> some hope that the OpenGL bindings might, in fact, work well across 
>> multiple platforms.
>>
>> One complication with OpenGL is GLUT, which doesn't seem (IIRC) to be 
>> well supported on some platforms (e.g. MacOS). So, some tricks might 
>> be necessary there.
>>
>> I guess the main takeaway there is that IF (big if :-)) the library 
>> has been written in a portable way, then a single jextract run might 
>> be all you need (maybe with some filtering added on top to get rid of 
>> platform-dependent functionalities, which you can do with the 
>> jextract options already available).
>>
>> The USB code [1] I pointed you at, on the other hand, is much more 
>> complex, as that code needs to interface with _different_ libraries 
>> to do its job on different platforms - so of course that kind of 
>> unification is not something you can achieve using a tool.
>>
>> So, as often is the case with native code, we have a spectrum of 
>> "badness":
>>
>> 1) well-behaved libraries (like libclang and OpenGL), which are 
>> written with portability in mind
>> 2) libraries that are mostly portable, but might present minor 
>> mismatches (e.g. long being 64 vs. 32 bits, some API points not 
>> available in all platforms, etc.)
>> 3) libraries that are hopelessly platform-specific
>>
>> I believe jextract proves that (1) already works (**). And for (3), 
>> as stated, not much can be done.
>>
>> So the question is whether something should be done for (2) - and how 
>> much can be done "automagically". This will likely require us to come 
>> up with a precise definition of the boundaries of what gaps we can 
>> actually bridge, as (2) is a rather fuzzy bucket.
>>
>> (**) I can see ways in which even libraries in bucket (1) could fail 
>> - e.g. this can happen because the symbol lookup would fail to 
>> initialize because of different library names/paths. This is 
>> something that we need to address, but also something which requires 
>> a much much smaller hammer (such as the ability to define your own 
>> symbol lookup).
>>
>> Cheers
>> Maurizio
>>
>> [1] - https://github.com/manuelbl/JavaDoesUSB
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230106/2185c056/attachment-0001.htm>