Creating a high level Pointer API

[Maurizio Cimadamore]

Hi think some of the points in this emails have been discussed to death 
(such as the enum translation). The point of having an API is precisely 
that you can decide whatever translation you see fit.
As for your claims about higher-level APIs, again, I think an higher 
level API would need to _hide_ the memory access API, so I see very 
little value in having an higher-level API implements memory access 
interfaces directly. You seem caught in the middle - you want something 
slightly higher level than memory address, but you want to use raw 
bindings which use MemoryAddress and friends.

If you introduce an higher level abstraction - call it HighPointer, then 
you can, through the jextract API, wrap all MemoryAddress coming from 
the library into HighPointer, and back. The user will never have even to 
think about MemoryAddress. That's how I'd go about doing an higher level 
pointer API.

As for the rest, distinguishing between long/double, and selecting 
SystemABI, these are rough edges of the API we are aware of. As you 
point out, this is early days and we're still actively hacking on that 
API (while the memory access API is more stable now) - we know we have 
to do better in order to make some of the choices the ABI does more 
transparent. Opening up the ABI classes, and also opening up the layout 
metadata infrastructure is the likely way for us to get there.

Maurizio


On 10/01/2020 07:23, Ty Young wrote:
>
>
> On 1/9/20 8:01 AM, Maurizio Cimadamore wrote:
>>
>> On 09/01/2020 13:13, Ty Young wrote:
>>>
>>>
>>>> Note that MemoryAddress is not meant to be implemented by the user, 
>>>> and will be made 'sealed' as soon as the sealing support is made 
>>>> available (by Amber project). So I recommend using aggregation, not 
>>>> subtyping here (e.g. a Crosspoint has-a MemoryAddress, instead of a 
>>>> CrossPoint is-a MemoryAddress (and then expose the address with an 
>>>> accessor, if needed)
>>>
>>>
>>> Is there an actual reason for this or is it "just because"? The 
>>> ability to plug a Pointer implementation into whatever MemoryAddress 
>>> argument is kind of a big deal with less boilerplate and all. It 
>>> kinda waters down the benefit of higher level abstractions...
>> It's not "just because" - MemoryAddress is a trusted interface, which 
>> will receive special treatment in some of the C2 optimizations we are 
>> considering. We want to keep that door open to provide better 
>> performances.
>
>
> By denying higher level API makers from seamlessly integrating lower 
> level memory access API into their APIs. You say that things can be 
> provided by some third party API aren't willing to give them the 
> necessary interfaces to do it *properly*.
>
>
> Nor are you seemingly willing to just create a higher level API 
> yourselves, which as I'm finding out, isn't even that hard or much 
> code(arrays and String pointer were easy to implement). In fact it's 
> probably harder for me since I don't have internal API information and 
> the memory access API doesn't publicly expose enough information or 
> functionality, as I'm finding out.
>
>
> Like, how are you supposed to tell a long and a double apart? Both are 
> 64-bits(on Java, anyway) and no information that I'm seeing is 
> available from ValueLayout to tell them apart. There is also the 
> previously mentioned SystemABI classes that aren't exposed, so I need 
> to introduce my own "Platform" enum to represent everything. Why not 
> have ValueLayout expose an Optional SystemABI where in if it is empty 
> it can be interpreted as being based on a Java type?
>
>
> Of course, things are early and are subject to major changes. It's 
> really just (IMO) the stinginess when you expect others to pick up the 
> slack(which is fine in itself) but aren't seemingly willing to provide 
> what's needed for quality higher level APIs, that's all.
>
>
> BTW, what's the deal with all the VarHandle methods? A lot of those 
> are just combined read/write operations. Do they improve performance 
> or something? Also, shouldn't weakCompareAndSet be 
> weakCompareAndSetVolatile and weakCompareAndSetPlain just be 
> weakCompareAndSet?
>
>
>>>
>>>
>>> Frankly, I don't understand what the benefit of "sealing" an 
>>> interface is - all the code needed to actually implement it is 
>>> already hidden inside modules. Its not like someone can build a 
>>> MemoryAddress from scratch.
>> That's a separate discussion
>>>
>>>
>>>> Seems like a good direction to explore - the fact that you have 
>>>> been able to come up with something so quickly seems a good sign - 
>>>> e.g. that the memory access API has the knobs that are needed to 
>>>> build higher level stuff (which is sort of the claim we're making 
>>>> here).
>>>
>>>
>>> Primitives are easy. I haven't looked at the old Pointer API code 
>>> but I imagine implemented Struct, Enum, and function callback types 
>>> are a bit more complicated.
>>>
>>>
>>> How is the new jextract going to handle enums, btw?
>>
>> Probably like the old one - just ints (higher-level grouping can be 
>> added, if  needed, via API).
>
>
> Why not do what I suggested earlier? Just letting people insert random 
> ints into functions is a great way to introduce bugs.
>
>
>>
>> Maurizio
>>