Pointer/Scope API questions

[Jorn Vernee]

On 08/01/2020 07:25, Ty Young wrote:
>
> On 1/7/20 6:02 PM, Maurizio Cimadamore wrote:
>>
>>
>> On 07/01/2020 23:18, Ty Young wrote:
>>> What I linked is an example of how an Nvidia attribute is 
>>> implemented in my API. I linked it because it shows how to-the-point 
>>> and easy Pointer is. My API implements 50+ Nvidia attributes which 
>>> is very easy(as I linked) under Pointer... however doing this 
>>> without some higher level would result in these small 
>>> implementations becoming much larger - unless of course I make my 
>>> own Pointer... but for use cases like mine, defining what a Pointer 
>>> is when Project Panama could provide that definition(from an 
>>> interface prospective) hurts more than it helps. 
>>
>> Please see also my other reply on your original email - I mistook the 
>> example you have linked for a different one you brought up before - I 
>> don't think the example you shared earlier today is so atypical (the 
>> other where you have no header is :-) ).
>>
>> Now, in your example, my feeling is (as in the examples that Michael 
>> shared last week) that if you replace Pointer with 
>> MemoryAddress/MemorySegment you don't lose all that much. After all, 
>> you both are after an higher level wrapper, so you don't want to 
>> expose Pointer directly - you just want something that _encodes_ a C 
>> pointer and that can be passed to a C function where a pointer is 
>> expected (which you can do with native method handles) - or to 
>> dereference it to read some data (which you can do with the memory 
>> access var handles).
>>
>> For instance in the code in [1], you would need to:
>>
>> 1) change the type of the 'utilStruct' field to MemorySegment
>> 2) in the constructor, just create the segment 
>> (MemorySegment::allocateNative) - if you have generated the bindings 
>> using the minimal jextract (which will soon be available), you will 
>> have some static constant with the layout of the struct you want 
>> (nvmlUtilization_st), so you will be able to just do:
>>
>> this.utilStruct = 
>> MemorySegment.allocateNative(MyBindings.nvmlUtilization_st$LAYOUT);
>>
>> 3) In the update() method, replace:
>>
>>
>> this.utilStruct.get().gpu$get()
>>
>> with something like
>>
>> MyBindings.nvmlUtilization_st$gpu$get(utilStruct);
>>
>> (again, the static accessor "nvmlUtilization_st$gpu$get" will be 
>> autogenerated for you by the minimal jextract).
>>
>> And that's it (for that class, at least).
>>
>
> I was expecting there to be 50 lines per implementation TBH. That 
> doesn't seem bad at all.
>
>
>>
>> I think your frustration is understandable - we're moving from an API 
>> which has a certain level of familiarity (everyone knows what a 
>> 'pointer' is, right?) to a _different_ way of doing things. But one 
>> of the most important lessons we've learned last year is, I think, 
>> that for Panama to succeed, and to allow interop between Java and 
>> native code, the Java code doesn't _have_ to look exactly like C. It 
>> is easy to fall into a sort of trap where it feels like the goal is 
>> is to minimize the differences between the Java code and the 
>> corresponding C code. While attractive, that approach is a siren 
>> song, and sends you down a slippery slope, where you need many 
>> constructs to support features in the native language; the first one 
>> is Pointer of course, then you realize you need Array too - and 
>> function pointers? which led us to Callback - then there's 
>> LongDouble, and along the way you encounter Complex XYZ - after a 
>> while you end up pulling in the entire C world inside the JDK. And 
>> then, what about C++ ? Do we need a Reference too (and maybe 
>> LHSReference and RHSReference) ? And what about other foreign 
>> languages? I hope you get what I'm trying to say here.
>>
>> Focusing on the primitive tools (MemoryAddress, MemorySegment, memory 
>> var handles, native method handles) allows us to deal with a lot of 
>> stuff (pretty much all we need) introducing basically no coupling 
>> with the C world directly in the JDK. Which is, I think, a good place 
>> to land. And, what was surprising, was that after you get adjusted to 
>> the new perspective, well, it's not like the new world w/o Pointers 
>> and friends is so much worse; see how few changes were needed in your 
>> case - I'm sure there are worse cases than that, but still. I've 
>> replaced the Pointer-based version of the libclang higher-level API 
>> we have to use the memory access API and native method handles in 
>> less than a day - and overall there were only a couple of tricky 
>> cases which required a bit more caution - most of the translation was 
>> straightforward.
>>
>> So I'd say, let's give this approach a try and see how it goes - from 
>> where I look, it certainly seems like a (much) saner place to be.
>>
>
> Right, C is a very nasty world. The less brought into Java the better.
>
>
> BTW, is there something wrong with the memaccess-jextract branch? I've 
> compiled it, enabled preview features, and added the 
> jdk.incubator.foreign and jdk.incubator.jextract modules but get an 
> exception when attempting to run the classes at runtime:
>
>
> Exception in thread "main" java.lang.NoClassDefFoundError: 
> jdk/incubator/foreign/SystemABI
>     at 
> org.goliath.panamamemtest.PanamaMemTest.main(PanamaMemTest.java:30)
> Caused by: java.lang.ClassNotFoundException: 
> jdk.incubator.foreign.SystemABI
>     at 
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
>     at 
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>     at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:523)
>
>
> Maven 3.6.2 with release set explicitly to Java 15.
>
>
> The same goes for the jextract module. Given that I can require the 
> module and import the classes I'm a bit confused as to why it's doing 
> this. The jmods are visible in the JDK builds so... yeah.
>
Are you also adding `--add-modules 
jdk.incubator.foreign,jdk.incubator.jextract` when running? I was seeing 
a similar error when I forgot to do that.

Jorn

>> Maurizio
>>