jextract C++ support

Mon May 29 09:00:34 UTC 2023

On 29/05/2023 00:20, Rel wrote:
> dynamic dispatch
>
> I tried following example 
> [https://github.com/enatai/panamaexperiments/blob/main/libcppexperiments/src/main/public/happy.hpp 
> <https://urldefense.com/v3/__https://github.com/enatai/panamaexperiments/blob/main/libcppexperiments/src/main/public/happy.hpp__;!!ACWV5N9M2RV99hQ!JnMeRRKWI0A8qpGkiXzcR0f06AAFEP-dXJV8lT-ZAMcxwE3BhDeGj91EgKkeMudSckPxr_-N9AlNWSGCtyvUZL0$>] 
> and it works fine as long as we generate proper Java bindings.
>
> See test for it 
> [https://github.com/enatai/panamaexperiments/blob/main/cppexperiments/src/test/java/cppexperiments/HappyTests.java#L36 
> <https://urldefense.com/v3/__https://github.com/enatai/panamaexperiments/blob/main/cppexperiments/src/test/java/cppexperiments/HappyTests.java*L36__;Iw!!ACWV5N9M2RV99hQ!JnMeRRKWI0A8qpGkiXzcR0f06AAFEP-dXJV8lT-ZAMcxwE3BhDeGj91EgKkeMudSckPxr_-N9AlNWSGCRH2ufOA$>]
>
> Please let me know which dynamic dispatch use cases you are concerned 
> with. Because this one seems works fine.

Sorry, I can see that working fine because you declare a "static" 
function which accepts a point, so the vtable indirection is generated 
by the CPP compiler in that function.

What I'm worried about is calling virtual methods on classes. E.g. 
calling your "distance" function directly. Jextract gives you two 
possibilities: Point2d::distance and Point3d::distance. If you pass a 
Point3d object to Point2d::distance you will "only" get 
Point2d::distance to be called (as if there was no dynamic dispatch).

>
> std::string
> I totally forgot that such basic type like std::string in C++ is a 
> template.
> But it seems possible to call functions which operate with string 
> objects because symbols for them are present:
>
> 00000000000013ad T _ZN7unhappy10helloWorldB5cxx11Ev
>
> std::string helloWorld();
>
> I guess it is possible to create/extract layout for std::string using 
> FFM but:
> - how to initialize this layout from Java? we cannot just call 
> std::string constructor for it, right?
> - this layout may differ between different C++ runtimes (libstdc++ 
> etc). MS C++ may have not same std::string layout as GCC

On the latter, e.g. layout difference, this is no different than 
anything else with jextract. E.g. each jextract run is 
platform-dependent, as it pulls in header files that are heavily 
influenced by the platform and OS you run on.

If I understand correctly, "string" is the "instantiation" of a template 
in C++. (e.g. some basic_string<char>). That instantiation is fully 
defined (e.g. not partial), and I believe it should be possible, with 
libclang, to obtain more information about it - such as the layout etc. 
(for partial template instantiation, my understanding, reading on what 
Rust bindgen does is that it is not possible to handle them with libclang).

So, ideally, we should be able to construct a layout for 
basic_string<char>, and then pass that to the constructor, yes.

Maurizio

>
> > Their binding generator adopts the same simple approach as the one I showed in the patch.
>
> I will take a look
>
>
> ------- Original Message -------
> On Tuesday, May 23rd, 2023 at 8:58 AM, Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>>
>> On 23/05/2023 05:11, Rel wrote:
>>> > What I meant for "robust analysis" was to try and establish how many _real-world_ C++ 
>>> library can really be tackled in such a direct approach.
>>>
>>> Ohh I see now, I am affraid we know the answer for this :)
>>>
>>> Let's imagine if number of C++ libraries which can be covered 
>>> end-to-end with "simple" approach is 0, does it mean that we should 
>>> discard it and only focus on shim for binding all kinds of APIs? 
>>> What about those cases which can be easily extracted using "simple" 
>>> approach, like Point2d?
>> Perhaps we should reach out to the Rust community? Their binding 
>> generator adopts the same simple approach as the one I showed in the 
>> patch. Given how hard it is to support C++ (because the underlying 
>> libclang C API is not very solid in that respect), I'd be surprised 
>> if they maintained all the necessary code just for stuff like Point2d?
>>>
>>> Because I thought that we would like to do "analysis" of what C++ 
>>> use cases can/cannot be covered with "simple" approach. For example 
>>> from your previous message I see that we are not completely sure 
>>> about exceptions:
>>>
>>> > * (probably way more stuff, like exceptions, etc.)
>>>
>>> Similarly for myself I would like to see what are the problems with 
>>> "dynamic dispatch". I added it to "unhappy" for now, because we 
>>> expect it not to work, but I plan to test it to see what are the 
>>> issues there and share here. Similarly with that anyone would be 
>>> able to reproduce and see same results.
>>>
>>> I guess my question now is: do we think it may be useful to know 
>>> exactly how many C++ use cases can be covered with "simple" FFM 
>>> approach. And if answer is yes, then we can use panamaexperiments as 
>>> a playground where we can have tests for what is covered. This 
>>> (possibly?) can give us more confidence in limitations of "simple" 
>>> approach and how far we can go with it (and this can be easily 
>>> demonstrated to everyone just by running those tests)
>>
>> I think it would be useful to know some answer to that question, yes. 
>> My intuition tells me that there are probably two kinds of C++ 
>> libraries: those who were born that way, and those that moved over 
>> from being simpler C libraries. One such example in the latter 
>> category is OpenCV [1]. While its "core" header [2] declares an 
>> exception, as well as a bunch of classes, eyeballing it, it doesn't 
>> seem "too" problematic? Perhaps that would be a good point where to 
>> start, and, if a library such as that can be used with some degree of 
>> success, perhaps we can expand the search to other similar libraries.
>>
>> [1] - https://opencv.org/
>> [2] - 
>> https://github.com/opencv/opencv/blob/4.x/modules/core/include/opencv2/core.hpp
>>
>>>
>>> Ideas?
>>>
>>> ------- Original Message -------
>>> On Monday, May 22nd, 2023 at 9:14 AM, Maurizio Cimadamore 
>>> <maurizio.cimadamore at oracle.com> wrote:
>>>
>>>>
>>>> On 22/05/2023 04:12, Rel wrote:
>>>>>> But I believe some more robust
>>>>>> analysis should be made to understand exactly how many APIs can be
>>>>>> supported in this "simple" fashion.
>>>>> Yes, I started to gather such analysis herehttps://urldefense.com/v3/__https://github.com/enatai/panamaexperiments__;!!ACWV5N9M2RV99hQ!KOxdK2qmmSzlaaO3kSlSUG2-ifWAVRD6OHlz9NHYvuggmy7NnnNxWvHYcxDm0Vn4gPXlbasjfC-ehIx_KlWNYiU$
>>>>> Currently there is only one happy case [https://urldefense.com/v3/__https://github.com/enatai/panamaexperiments/blob/main/libcppexperiments/src/main/public/happy.hpp__;!!ACWV5N9M2RV99hQ!KOxdK2qmmSzlaaO3kSlSUG2-ifWAVRD6OHlz9NHYvuggmy7NnnNxWvHYcxDm0Vn4gPXlbasjfC-ehIx_K7Nl50c$  ] which is Point2d class from your foo.hpp file.
>>>>
>>>> This is not too surprising - after all the hacky changes I shared 
>>>> were built around that example.
>>>>
>>>> What I meant for "robust analysis" was to try and establish how 
>>>> many _real-world_ C++ library can really be tackled in such a 
>>>> direct approach. My feeling is "not many" - but I don't have any 
>>>> hard data to back up this claim.
>>>>
>>>> Maurizio
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jextract-dev/attachments/20230529/c6276179/attachment-0001.htm>