Pointer/Scope API questions

[Ty Young]

On 1/7/20 6:02 PM, Maurizio Cimadamore wrote:
>
>
> On 07/01/2020 23:18, Ty Young wrote:
>> What I linked is an example of how an Nvidia attribute is implemented 
>> in my API. I linked it because it shows how to-the-point and easy 
>> Pointer is. My API implements 50+ Nvidia attributes which is very 
>> easy(as I linked) under Pointer... however doing this without some 
>> higher level would result in these small implementations becoming 
>> much larger - unless of course I make my own Pointer... but for use 
>> cases like mine, defining what a Pointer is when Project Panama could 
>> provide that definition(from an interface prospective) hurts more 
>> than it helps. 
>
> Please see also my other reply on your original email - I mistook the 
> example you have linked for a different one you brought up before - I 
> don't think the example you shared earlier today is so atypical (the 
> other where you have no header is :-) ).
>
> Now, in your example, my feeling is (as in the examples that Michael 
> shared last week) that if you replace Pointer with 
> MemoryAddress/MemorySegment you don't lose all that much. After all, 
> you both are after an higher level wrapper, so you don't want to 
> expose Pointer directly - you just want something that _encodes_ a C 
> pointer and that can be passed to a C function where a pointer is 
> expected (which you can do with native method handles) - or to 
> dereference it to read some data (which you can do with the memory 
> access var handles).
>
> For instance in the code in [1], you would need to:
>
> 1) change the type of the 'utilStruct' field to MemorySegment
> 2) in the constructor, just create the segment 
> (MemorySegment::allocateNative) - if you have generated the bindings 
> using the minimal jextract (which will soon be available), you will 
> have some static constant with the layout of the struct you want 
> (nvmlUtilization_st), so you will be able to just do:
>
> this.utilStruct = 
> MemorySegment.allocateNative(MyBindings.nvmlUtilization_st$LAYOUT);
>
> 3) In the update() method, replace:
>
>
> this.utilStruct.get().gpu$get()
>
> with something like
>
> MyBindings.nvmlUtilization_st$gpu$get(utilStruct);
>
> (again, the static accessor "nvmlUtilization_st$gpu$get" will be 
> autogenerated for you by the minimal jextract).
>
> And that's it (for that class, at least).
>

I was expecting there to be 50 lines per implementation TBH. That 
doesn't seem bad at all.


>
> I think your frustration is understandable - we're moving from an API 
> which has a certain level of familiarity (everyone knows what a 
> 'pointer' is, right?) to a _different_ way of doing things. But one of 
> the most important lessons we've learned last year is, I think, that 
> for Panama to succeed, and to allow interop between Java and native 
> code, the Java code doesn't _have_ to look exactly like C. It is easy 
> to fall into a sort of trap where it feels like the goal is is to 
> minimize the differences between the Java code and the corresponding C 
> code. While attractive, that approach is a siren song, and sends you 
> down a slippery slope, where you need many constructs to support 
> features in the native language; the first one is Pointer of course, 
> then you realize you need Array too - and function pointers? which led 
> us to Callback - then there's LongDouble, and along the way you 
> encounter Complex XYZ - after a while you end up pulling in the entire 
> C world inside the JDK. And then, what about C++ ? Do we need a 
> Reference too (and maybe LHSReference and RHSReference) ? And what 
> about other foreign languages? I hope you get what I'm trying to say 
> here.
>
> Focusing on the primitive tools (MemoryAddress, MemorySegment, memory 
> var handles, native method handles) allows us to deal with a lot of 
> stuff (pretty much all we need) introducing basically no coupling with 
> the C world directly in the JDK. Which is, I think, a good place to 
> land. And, what was surprising, was that after you get adjusted to the 
> new perspective, well, it's not like the new world w/o Pointers and 
> friends is so much worse; see how few changes were needed in your case 
> - I'm sure there are worse cases than that, but still. I've replaced 
> the Pointer-based version of the libclang higher-level API we have to 
> use the memory access API and native method handles in less than a day 
> - and overall there were only a couple of tricky cases which required 
> a bit more caution - most of the translation was straightforward.
>
> So I'd say, let's give this approach a try and see how it goes - from 
> where I look, it certainly seems like a (much) saner place to be.
>

Right, C is a very nasty world. The less brought into Java the better.


BTW, is there something wrong with the memaccess-jextract branch? I've 
compiled it, enabled preview features, and added the 
jdk.incubator.foreign and jdk.incubator.jextract modules but get an 
exception when attempting to run the classes at runtime:


Exception in thread "main" java.lang.NoClassDefFoundError: 
jdk/incubator/foreign/SystemABI
     at org.goliath.panamamemtest.PanamaMemTest.main(PanamaMemTest.java:30)
Caused by: java.lang.ClassNotFoundException: jdk.incubator.foreign.SystemABI
     at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
     at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
     at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:523)


Maven 3.6.2 with release set explicitly to Java 15.


The same goes for the jextract module. Given that I can require the 
module and import the classes I'm a bit confused as to why it's doing 
this. The jmods are visible in the JDK builds so... yeah.


> Maurizio
>