Pointer/Scope API questions

[Ty Young]

On 1/7/20 8:09 AM, Maurizio Cimadamore wrote:
>
> On 07/01/2020 13:47, Ty Young wrote:
>> On 12/22/19 3:10 PM, Maurizio Cimadamore wrote:
>>
>>>
>>> On 21/12/2019 23:59, Ty Young wrote:
>>>> Hi,
>>>>
>>>>
>>>> There are a few questions I'd like to ask after using it for awhile:
>>>>
>>>>
>>>> A. the implementation of equals on Pointers and Scopes... How is it 
>>>> implemented? In the case of Pointers, does the scope of which the 
>>>> pointer was allocated matter? Is comparing Pointers to a struct of 
>>>> a given type as reliable as comparing the structs themselves?
>>> Pointer equality just compares the addresses - not the scopes; that 
>>> is, if two pointers p1 and p2 point to the same memory location, 
>>> they are considered equals. 
>>
>>
>> Is there no way to fix this?
>
> At this point we'd prefer to invest our resources in the foreign 
> memory access and foreign abi efforts (and the minimal jextract). The 
> Pointer API was an experiment on how a pointer class might have looked 
> like in Java - I think it was successful to some extents - but failed 
> in others, as discussed other times.
>
> I don't plan we will be changing Pointer equality in the foreign 
> branch in the near future.
>
>>
>>
>>> As to your question about differences w/r/t struct comparison, I 
>>> don't think the API support struct comparison.
>>
>>
>> Regardless of whether the API supports it or not, equals is apart of 
>> every Object. Maybe an unchecked exception should be thrown?
> you mean calling equals() on two struct objects? Well, the default 
> behavior will compare just the Java references. I suppose this could 
> be enhanced to compare layouts + base pointers - but again, this is 
> not something that I think it's worth investing on at this point.
>>
>>
>>>>
>>>>
>>>> B. Is mixing allocated native types from different scopes and using 
>>>> them together safe? Will it result in any weird side affects?
>>>
>>> You can allocate pointers from different scopes and mix them. The 
>>> general rule of thumb (described here [1]) is that you can only 
>>> write a pointer p1 into a memory pointed by p2 if p1 is guaranteed 
>>> to outlive p2.
>>
>>
>> Thanks for the clarification.
>>
>>
>>>
>>>
>>>>
>>>>
>>>> C. What scope are Pointers from C from exactly? The libraries 
>>>> specific scope? Are there exceptions like If a function returns a 
>>>> Pointer which is derived from function argument Pointer data, is 
>>>> the scope the same as the input arguments?
>>>
>>> Pointers that come from C have the 'unchecked' scope - as such they 
>>> are 'unmanaged' pointers. As such you can do whatever you like with 
>>> them, as the Panama/foreign branch doesn't restrict use of native 
>>> pointers in any way - which also means that it's not easy to make 
>>> them safer also (e.g. if you happen to know more about the native 
>>> pointer bounds). All the work in the Pointer API predates some of 
>>> the discussions on how to make native pointer safer, which we had 
>>> more recently.
>>
>>
>> So once Pointer is rebased on the memory access API, you'll be able 
>> to get the underlying MemoryAddress from it and do with it as you will?
>
> I cannot guarantee you that Pointer will ever be rebased on top of the 
> memory access API. From the experiments we have done working with 
> lower-level bindings I can't exactly say that we're missing Pointer much 


As someone who has been using jextract/Pointer I'm really upset by this. 
I completely understand that more complex APIs require tuning but there 
are also those that don't, and Pointer works perfectly for those use 
cases. I know because I've been using it. My only issues are enums being 
converted to ints, memory not being zero'd, and some method naming 
issues(ptr instead of pointer). Otherwise, as far as I can tell, it 
works perfectly fine.


I say "only" in that those are the only few issues, not how severe those 
are individually. Randomly getting a value of 896241 when reading struct 
data that is supposed to be a GPU's utilization(0-100) is hardly minor.


Likewise I think the enum situation can and should be handled better 
regardless of the way it can be abused in C. An idea that comes to mind 
is to take that int that represents the corresponding C enum and wrap it 
in a Java enum (since you know the C enum name from the header) and have 
it implement an interface generated from the header to lock it to the 
specific methods it is used in(to prevent the current generic int type 
method calls whenever enums are used).



> - and the fact that Pointer would definitively need at least 
> specialized generics to be considered 'good enough' (so that I can say 
> Pointer<int> and not Pointer<Integer>) is a big impediment.


Why does this matter exactly? It's not like Java has the ability to 
specify unsigned or signed ints either nor are all C types represented 
in Java properly(long long, for example). There are plenty of places 
where specialized generics could be useful before now but that hasn't in 
the past prevented those features being released before...


>
> It will be up to custom extractors to decide how to model pointers - 
> the minimal one will just model them as MemoryAddress. Clients will of 
> course be free to use the jextract API to define extensions which will 
> wrap MemoryAddress to something more similar to Pointer - but I don't 
> see Pointer belonging to the JDK (at least not in the way it currently 
> is defined).


So has the goal of Project Panama completely changed then? IIRC, first 
it was about just providing native code access, then it was discovered 
that the use case of better/safer memory access could be supported and 
was in-line with the project's goals, and now the original goal of 
providing native code access seems to be the afterthought as apposed to 
memory access. This has turned into a bit of a roller coaster(although 
that seems to be theme of Java projects in general).


What's the status of the jextract API side? I imagine not much has 
changed because of the holidays but I'd like to get off Pointer if the 
entire API is going to be abandoned ASAP.


I don't feel like Pointer should be abandoned though. Pointer is an 
interface, not an implementation.... If there are platform specific 
details that need to be worked out then that's an implementation issue. 
jextract can(and actively does) have the ability to dump working binding 
code that can be modified as desired. Could jextract not simply have the 
ability to generate very generic bindings for those that just want 
native access quickly(or don't care about cross-platform or whatever 
else) or as a base to work off of to provide higher quality bindings?



>
> Maurizio
>
>>
>>
>>>
>>> Maurizio
>>>
>>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/scopes.html
>>>
>>>>
>>>>
>>>> Thanks for any clarity.
>>>>