Pointer/Scope API questions

[Maurizio Cimadamore]

On 07/01/2020 23:18, Ty Young wrote:
> What I linked is an example of how an Nvidia attribute is implemented 
> in my API. I linked it because it shows how to-the-point and easy 
> Pointer is. My API implements 50+ Nvidia attributes which is very 
> easy(as I linked) under Pointer... however doing this without some 
> higher level would result in these small implementations becoming much 
> larger - unless of course I make my own Pointer... but for use cases 
> like mine, defining what a Pointer is when Project Panama could 
> provide that definition(from an interface prospective) hurts more than 
> it helps. 

Please see also my other reply on your original email - I mistook the 
example you have linked for a different one you brought up before - I 
don't think the example you shared earlier today is so atypical (the 
other where you have no header is :-) ).

Now, in your example, my feeling is (as in the examples that Michael 
shared last week) that if you replace Pointer with 
MemoryAddress/MemorySegment you don't lose all that much. After all, you 
both are after an higher level wrapper, so you don't want to expose 
Pointer directly - you just want something that _encodes_ a C pointer 
and that can be passed to a C function where a pointer is expected 
(which you can do with native method handles) - or to dereference it to 
read some data (which you can do with the memory access var handles).

For instance in the code in [1], you would need to:

1) change the type of the 'utilStruct' field to MemorySegment
2) in the constructor, just create the segment 
(MemorySegment::allocateNative) - if you have generated the bindings 
using the minimal jextract (which will soon be available), you will have 
some static constant with the layout of the struct you want 
(nvmlUtilization_st), so you will be able to just do:

this.utilStruct = 
MemorySegment.allocateNative(MyBindings.nvmlUtilization_st$LAYOUT);

3) In the update() method, replace:


this.utilStruct.get().gpu$get()

with something like

MyBindings.nvmlUtilization_st$gpu$get(utilStruct);

(again, the static accessor "nvmlUtilization_st$gpu$get" will be 
autogenerated for you by the minimal jextract).

And that's it (for that class, at least).


I think your frustration is understandable - we're moving from an API 
which has a certain level of familiarity (everyone knows what a 
'pointer' is, right?) to a _different_ way of doing things. But one of 
the most important lessons we've learned last year is, I think, that for 
Panama to succeed, and to allow interop between Java and native code, 
the Java code doesn't _have_ to look exactly like C. It is easy to fall 
into a sort of trap where it feels like the goal is is to minimize the 
differences between the Java code and the corresponding C code. While 
attractive, that approach is a siren song, and sends you down a slippery 
slope, where you need many constructs to support features in the native 
language; the first one is Pointer of course, then you realize you need 
Array too - and function pointers? which led us to Callback - then 
there's LongDouble, and along the way you encounter Complex XYZ - after 
a while you end up pulling in the entire C world inside the JDK. And 
then, what about C++ ? Do we need a Reference too (and maybe 
LHSReference and RHSReference) ? And what about other foreign languages? 
I hope you get what I'm trying to say here.

Focusing on the primitive tools (MemoryAddress, MemorySegment, memory 
var handles, native method handles) allows us to deal with a lot of 
stuff (pretty much all we need) introducing basically no coupling with 
the C world directly in the JDK. Which is, I think, a good place to 
land. And, what was surprising, was that after you get adjusted to the 
new perspective, well, it's not like the new world w/o Pointers and 
friends is so much worse; see how few changes were needed in your case - 
I'm sure there are worse cases than that, but still. I've replaced the 
Pointer-based version of the libclang higher-level API we have to use 
the memory access API and native method handles in less than a day - and 
overall there were only a couple of tricky cases which required a bit 
more caution - most of the translation was straightforward.

So I'd say, let's give this approach a try and see how it goes - from 
where I look, it certainly seems like a (much) saner place to be.

Maurizio