Creating a high level Pointer API

[Maurizio Cimadamore]

On 09/01/2020 12:02, Ty Young wrote:
> Hi,
>
>
> After the confirmation that Pointer is being axed, I've started to 
> create my own high level Pointer abstraction called Crosspoint[1]. 
> Corsspoint's goal is to (as seamlessly as possible) smooth over the 
> rough edges of using the raw Memory Access API and mimic the old 
> Pointer API.
>
>
> In short, it:
>
>
> 1. Provides seamless interoperability with lower Memory Access APIs by 
> directly extending Memory Access's interfaces(Pointer interfaces 
> extends MemoryAddress interface, for example).
Note that MemoryAddress is not meant to be implemented by the user, and 
will be made 'sealed' as soon as the sealing support is made available 
(by Amber project). So I recommend using aggregation, not subtyping here 
(e.g. a Crosspoint has-a MemoryAddress, instead of a CrossPoint is-a 
MemoryAddress (and then expose the address with an accessor, if needed)
>
>
> 2. Does not hide any lower level details. In other words, you can go 
> top down(by Pointer.ofJavaByte()) static method or bottom up(by 
> Pointer.fromAddress()) which takes lower level Memory Access objects 
> as paramaters.
>
>
> 3. Allows the explicit creation of platform specific Pointers. Example:
>
>
> Pointer<? super Number> longPointer = 
> Pointer.ofPlatformLong(Platform.WINDOWS);
>
>
> where ofPlatformLong can either return a Pointer of 
> Pointer<Integer>(as above on Windows) or as Pointer<Long>(for 
> everything else).
>
>
> 4. provides a way to read, write, and work with referenced memory, 
> just like the old Pointer API(with different method naming, anyway):
>
>
> long longNumber = 1;
>
> longPointer.setValue(longNumber);
>
> System.out.println(longPointer.getValue());
>
>
> 5. Constant Null Pointer:
>
>
> Pointer<?> nullPointer = Pointer.ofNull();
>
>
> 6. Planned ability to "home" a Pointer that is based on a non native 
> ValueLayout to the equivalent native layout as well as a quick boolean 
> check utility function to check if it isn't.
>
>
> (probably more than I'm forgetting)
>
>
> It isn't much yet - only Byte, Integer, and Long are implemented(no 
> Array or anything like that yet either) but so far it seems to be even 
> more user friendly than the older Pointer API. That's mostly just down 
> to Scope's no longer being in the way.
Seems like a good direction to explore - the fact that you have been 
able to come up with something so quickly seems a good sign - e.g. that 
the memory access API has the knobs that are needed to build higher 
level stuff (which is sort of the claim we're making here).
>
>
> Anyway, what I'm sending this for is to ask for memory access API 
> enhancements so that I can add 6(referring to above). Right now 
> -unless I'm blind- there is no way to determine the current runtime 
> ABI and I need that to determine which ABI to "home" the pointer to. 
> Could this be added if it isn't or can anyone point me to where I can 
> find it? I'd also need the ValueLayout to expose the Platform(perhaps 
> by enums?) in which they belong to. Right now AFAIK they don't. Could 
> all that be added?

Not sure what you mean by 'home' a pointer, and which piece of 
functionality you need - I guess you need something to switch between 
different layouts?

We have plans to rework SystemABI a bit - Jorn and I discussed about 
making all the ABI implementation class public, so that people can 
actually 'see them'.

But in your case, I wonder if what you need is more something like we're 
doing in our LibClang port:

http://hg.openjdk.java.net/panama/dev/file/65925c71ceb3/src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/LayoutUtils.java#l255

And then, when the client creates an 'int' pointer, you just use the 
right layout (since e.g. C_INT is initialized to point to the right place).

>
>
> While I'm at it, could the documentation for var handles be simplified 
> at all? I spent way longer than I'd like to admit trying to figure out 
> how to actually read data from the Var Handle and ended up just 
> referencing the jextract clang code that was linked on this list 
> before. I know working with memory is complicated but when you're met 
> with a method that accepts an ambiguous array of objects and a long, 
> (seemingly) complicated docs it kinda throws you off a bit.
We will keep trying to do our best to make the documentation as 
accessible as possible (but also as complete as possible). I think on 
the MemoryLayout/MemoryAddress/MemorySegment front we did a good work, I 
agree that the VarHandle part is not optimal - in part because, as you 
say, it's just hard.
>
>
> If there is any observations on the code itself or advice that can be 
> given for making a higher level abstraction then I'd appreciate it as 
> well. Keep in mind that the goal isn't to support every use case, but 
> only to cover what the old Pointer API(collectively) could/did do.

What you did seems good - I think the 'homing' stuff seems a tad 
over-engineered, in the sense that I'd just personally prefer a 
straightforward interface where I can allocate a pointer given 'any' 
layout, and, perhaps, on the side, give the user a menu of layouts 
(SysV, Windows, or Portable ones) - perhaps using your Platform abstraction.

In other words, I think your Pointer interface is doing double-duty - 
it's a pointer (so support dereference, etc.) but it also has several 
functionalities which have to do with 'finding the right layout'. I 
think it could be more beneficial to factor these somewhere else. My 0.02$.

Also, you can easily make the Pointer extend AutoCloseable and call 
free() automatically, so that you can use Pointer inside try-with-resources.

Maurizio

>
>
> [1] https://github.com/BlueGoliath/Crosspoint
>