some thoughts on panama/jextract

[Michael Zucchi]

On 7/1/20 9:20 pm, Maurizio Cimadamore wrote:
>
> On 07/01/2020 04:24, Michael Zucchi wrote:
>> I think most of my initial concerns have been addressed so I will 
>> work on porting some of my code and am interested to see how it works 
>> in practice.  (but no jextract for now!) 
>
> Thanks - I've took a look at your APIs and, since they look generally 
> clean and high-level - I believe they can relatively easily be ported 
> to use the MemoryAddress/MethodHandle machinery in the foreign-abi 
> branch.
>
> Without an extractor you will have to write some mechanical code by 
> hand though - some examples of how to do that:
>
> http://hg.openjdk.java.net/panama/dev/file/foreign-abi/test/jdk/java/foreign/StdLibTest.java 
>
>
> I believe you can use the same approach to implement your library 
> without using JNI. I see you have many 'wrappers' with a 'long' 
> address field - you will have to replace them with a MemoryAddress 
> field (if these structs are to be passed by pointer - or MemorySegment 
> if you want to pass them by value). Then you have to setup the 
> required method handles to be able to call the native functions you want.
>

Cool cheers, I will suss out how it works then just hack up my extractor 
for it.  It looks all quite straightforward, and probably addresses most 
of the questions i've sent to this list.  Sorry I should've read your 
jextract article earlier, I didn't realise it was changing quite so much.

One reason jjmpeg looks clean and high level is that any messy details 
are hidden in the JNI code.

It's moot now, but today I was going to ask about what I thought were 
near-intractible problems with using Pointer and Scope.  So after having 
now read your jextract article and the other thread on it, i'm glad to 
hear that a lower level interface is available.

The problem I hit was with trying to implement something like this (from 
vulkan):

     VkResult res;
     uint32_t devcount;

     res = vkEnumeratePhysicalDevices(g->instance, &devcount, NULL);
     VkPhysicalDevice devs[devcount];
     res = vkEnumeratePhysicalDevices(g->instance, &devcount, devs);

    .. do something to find a suitable devid ..

     g->physicalDevice = devs[devid];


(VkPhysicalDevice is an undefined 'handle', aka a pointer type).

This reads effectively constant data into a temporary array - so 
although the array is managed it's contents shouldn't be.  I couldn't 
work out how to just copy the pointer value to "de-scope" it (and from 
your earlier article about scopes/pointers it seems that is on 
purpose).  The only solution seemed to be just to let the scope retain 
the array for the lifetime of the result.  This type of query api is 
everywhere so that isn't always practical.

Also Scope is a bit heavy and Pointer a bit clumsy to use in practice - 
like having to retain a Pointer<x> to pass around to c, but needing to 
use Pointer::get() for Java (meaning you either need to export Pointer 
to the library user, or proxy every accessor in your glue).

Cheers,
  Michael