Creating a high level Pointer API

[Ty Young]

Apologies for the wording used. In hindsight it was a bit much.


On 1/10/20 6:36 PM, Maurizio Cimadamore wrote:
> Can we please agree to disagree? 


IMO, this is more than just people having different opinions - there are 
negative repercussions for going this route.


> I've shown in the document how the fragmentation you are afraid of is 
> already the status quo, by linking at least 3 different pointer 
> implementation, with radically different properties, both at in terms 
> of the type system choices and in terms of how they are implemented. 
> Some of these libraries even have different pointer classes to denote 
> different state - e.g. pointer with size vs. unchecked pointer. I'm 
> under no illusion that "one pointer will rule them all".


I'm a bit confused here. Java has an array of features to handle 
everything you just mentioned. Fields that might not exist? Optional. 
Entirely different interpretations of the same thing? Interface 
implementations(with abstract classes to get rid of common stuff). The 
JDK already has interfaces with half a dozen or more implementations. It 
isn't new territory.


>
> My definition of higher level API is, to put it more clearly, an API 
> which captures some of the properties of a pointer in the type system, 
> perhaps through generics (e.g. Pointer<Integer>). And, some sprinkle 
> of accessor methods, so that users of the higher-level API need not to 
> worry about constructing VarHandle accessors. So, to me something like 
> roughly like this:
>
> class Pointer<X> {
>    MemoryAddress _addr;
>    VarHandle handle;
>
>    Pointer(MemoryAddress addr, MemoryLayout layout) { ... init var 
> handle ... }
>
>    X get() { return handle.get() }
>
>    void set(X x) { return handle.set(x); }
>
>    ...
>
> }
>
> Would constitute a possible sketch for an higher-level API.


Right, and people are going to write that near exact code over and over 
again.


>
> But we're getting ahead of ourselves - the way we will proceed will be 
> to (1) roll out the minimal extract (2) use it to rewrite the existing 
> foreign examples (3) assess its usability, and add some usability 
> boosts where needed. Here, it seems to me we're jumping to discuss 
> (3), without having even tried the thing in the first place. So, I'd 
> prefer to hold off this discussion, at least until we can make it more 
> concrete as I feel it's going in circles.


As someone who has toyed with both the old jextract generated bindings 
and the new Memory Access API, i'm not entirely sure what "usability" 
tweaks could be done. MemoryAccess objects would, AFAIK, act very much 
the same as Pointer but just without getter/setters for the data in the 
segment. Var handle just seems like boilerplate for the most basic of usage.


There are a number of API changes I would personally make like having 
ValueLayout contain everything that returns a ValueLayout(the same for 
SequenceLayout and GroupLayout interfaces). I'm doing a lot of class 
cycling to find what I need because it's scattered. More to-the-point 
documentation(as was brought up earlier) is always good too. I'm 
currently stuck trying to get multi-dimension arrays working and the 
MemoryAddress to the Array changes once put into memory gets modified 
with limit = 0. Presumably I need to rebase it or something, but it 
isn't working nor am I seeing anything saying such.


>
> As for memory segment not exposing layouts, that's by design - memory 
> layouts are optional - you can use them, or not, depending on your 
> need. A segment doesn't carry around the layout it was used for its 
> creation, also because there might not be one in the first place (e.g. 
> MemorySegment.allocateNative(100)).


Aren't the layouts just predefined byte sizes? Either way, again, 
Optional exists.


> Also, when slicing and dicing a segment, it would be hard, if not 
> impossible, to keep the layout information up to date (what happens if 
> you slice in the middle of, say a 4 byte ValueLayout?).


If the slice is completely invalid then an unchecked exception should be 
thrown(and there are exceptions, I've gotten a few). If it's valid then 
presumably you have the necessary information, otherwise it wouldn't be 
possible to begin with, right?


>
> Maurizio
>
>