BUG: SystemABI C_LONG and C_LONGLONG are the same

[Maurizio Cimadamore]

On 21/05/2020 16:50, Ty Young wrote:
>
> On 5/20/20 8:41 AM, Maurizio Cimadamore wrote:
>>
>> On 19/05/2020 15:05, Ty Young wrote:
>>> I don't think hiding low level Panama details is the way to go about 
>>> things, and I hope that if Panama does add a higher level API, it 
>>> doesn't go the route of trying to directly model C code like 
>>> "Pointer" kinda sorta did.
>>>
>>>
>>> I'd personally like to see something like I did, but I'm both biased 
>>> and realize it's probably unrealistic. The amount of classes that 
>>> this otherwise thin abstraction layer is a bit much for the JDK. In 
>>> my defense, the type specific values and arrays only exist because 
>>> Java doesn't support primitive generics.
>>>
>>>
>>> It would be interesting to get a wider perspective on which approach 
>>> is better. Do people think a smaller higher-level API is better 
>>> which has more "god-like" objects or is a more thinly spread, 
>>> targeted, and lots of classes better? I largely stand by the way I 
>>> went about it, but I'm still personally interested in what others 
>>> prefer and think. 
>>
>> This is a very good and constructive point. What kind of high-level 
>> API would people expect?
>>
>> By having spent time looking at what other languages did in this area 
>> (e.g. Kotlin, Swift and the likes), and, by also looking at what more 
>> traditional Java bindings for native libraries do (e.g. LWJGL) - at 
>> some point they all seem to reach out for some kind of pointer-like 
>> abstraction. So I think that _not_ having some kind of higher-level 
>> Pointer abstraction in the long run could be counter-productive.
>
>
> Happy to read this. FMA before the recent changes was already a bit of 
> a "assemble it yourself" type of API and it has already gotten more so 
> with recent changes.
>
>
>>
>> The problem you raise about lack of support for primitives (and, more 
>> generally, looking forward to Valhalla) inline classes inside 
>> generics (although we have plans for those) remains, IMHO, one of the 
>> main blockers to provide one such abstraction. My personal opinion is 
>> that the old pointer API was not too bad, but it failed because it 
>> doesn't go the full way towards supporting the full spectrum of C types.
>
>
> You mean C primitive types, right? If so, I feel like the lack of a 
> abstraction of Struct/Unions was an issue as well.

Yeah - C primitive types.

Structs are a problem, yes, but one can always generate a class to 
describe structs/unions which wraps a segment, and that feels good 
enough from an usability perspective. E.g. Pointer<nvmlProcessInfo_t> is 
absolutely fine - it's Pointer<Integer> that I have problem with - what 
does Integer mean in the context of native interop? What native type 
does it model?

>
>
> A way to put an overarching structure on a function like I have with 
> "NativeFunction" will probably be desirable too. While it wasn't an 
> issue with the old Pointer API because jextract cleanly generated 
> everything for you, now that you need to deal with the function's 
> address, MethodHandle, and FunctionDescriptor... well, shoving 
> everything into one class probably isn't going to be pretty for any 
> decently sized library.
Yes, function pointers will have to be modeled too in a high level API.
>
>
> I got to thinking about other things that would be useful as part of 
> an abstraction as well: library modeling and versioning.
>
>
> It would not be, I think, unexpected for someone making bindings for 
> something like, for example, OpenGL to want to create minimum binding 
> sets for older hardware. The good news is that Java has rich 
> polymorphic features that can allow this... the bad news is that a lot 
> of this is so library specific I don't know how much could be done to 
> help here. You can't just say that version "2.0" of some library is 
> just version "1.0" + some new functions because functions may have 
> been removed or do different things in version "2.0".
Versioning is typically a can of worms - and one that would be better 
handled by dedicated build  tools; let's not put too much meat on the 
burner.
>
>
>> As a side-effect, bindings written this way will also be more 
>> portable too - since abstractions such as `c_unsigned_long` and the 
>> likes can be coded in a way so that they take into account platform 
>> differences. Which means the signatures of your methods do not change 
>> across platforms.
>
>
> I wonder, is this really a good idea, even as part of a higher-level 
> abstraction? A long clearly has different sizes on different platforms 
> and attempting to artificially smooth those differences over at an 
> abstraction layer level seems like its pushing the abstraction part a 
> bit too far to the point of misleading.

I don't know! We are in the real of discussion possibilities here - 
portable C code never uses types whose size can change across platforms. 
So, when you write a good, portable C library, you typically define the 
set of types you want to operate on (with typedefs) and then you take it 
from there. After that point, all the signatures of your functions, and 
the fields in the structs remain fairly stable across platforms.

>
>
> Anyway, I don't know when the final discussion on when "What kind of 
> abstraction will FMA have?" will take place, but I would hope it won't 
> be too late when FMA is already part of the JDK. It is inevitable, I 
> think, that a higher level abstraction may put some spotlight on some 
> minor issues with FMA that may not be fixable if at that point.

I'm aware of that. As I stated in my previous email, w/o some help from 
Valhalla, I'm afraid there's no clear and usable/efficient path that 
leads us to a clean high level API that I would be comfortable with 
adding to the Java SE API.

Put it in other terms, the bindings that are currently generated by 
jextract are very very efficient - many orders of magnitude more 
efficient than what you would get by rolling an API on top. These 
bindings have also come a long way from v1, and the reality of the 
situation is that, in the set of examples we have collected, there's not 
a lot of difference between the code before/after the transition from 
the old jextracted code; there are interfaces generated for callbacks 
and structs - and even for typedefs; and there is an uniform API idom 
which removes the need to deal with var handles manually.

But these interfaces are just a bunch of holders for static helper 
methods - because we didn't want to allocate an object e.g. to read a 
size_t, or to read a struct (as the cost for doing so is prohibitive, if 
you care about performances). This is the part where we need help form 
the VM in order to make progress. Luckily the Valhalla prototype is now 
in a shape that makes it relatively easy to do certain experiments, and 
will keep improving in the upcoming months. This will allow us to start 
making some experiments which will give us a better idea on whether an 
expressive and efficient high-level API is a possibility, and what it 
should look like.

Maurizio