[foreign] Panama EA - August 2019 edition

[Henry Jen]

In early days, we have a function in Scope to convert String[] into Pointer<Pointer<Byte>> as this is pretty common use case, it’s took out because it’s considered easy to implement. Anyhow, I found my self add this function all the time.

Cheers,
Henry

public static Pointer<Pointer<Byte>> toCStrArray(Scope sc, String[] ar) {
    if (ar.length == 0) {
        return Pointer.ofNull();
    }

    Pointer<Pointer<Byte>> ptr = sc.allocate(NativeTypes.UINT8.pointer(), ar.length);
    for (int i = 0; i < ar.length; i++) {
        Pointer<Byte> s = sc.allocateCString(ar[i]);
        ptr.offset(i).set(s);
    }

    return ptr;
}

> On Aug 30, 2019, at 1:21 PM, Ty Young <youngty1997 at gmail.com> wrote:
> 
> 
> On 8/30/19 3:12 PM, Maurizio Cimadamore wrote:
>> 
>> On 30/08/2019 20:07, Ty Young wrote:
>>> 
>>> On 8/30/19 12:54 PM, Maurizio Cimadamore wrote:
>>>> 
>>>> On 30/08/2019 18:41, Ty Young wrote:
>>>>> 
>>>>> On 8/30/19 12:25 PM, Maurizio Cimadamore wrote:
>>>>>> 
>>>>>> On 30/08/2019 17:59, Ty Young wrote:
>>>>>>> 
>>>>>>> On 8/28/19 5:11 AM, Maurizio Cimadamore wrote:
>>>>>>>> I just want to clarify on this point below; the foreign memory access work does not, in any way, hinder the higher level functionalities provided by the foreign API/binder. We arrived at the foreign memory access API because we felt that something low level was missing - e.g. that the high level Pointer API was doing too much at once; and that users not interested in a high-level API, but still wanting to access off-heap data would not be served very well by the Pointer API alone.
>>>>>>>> 
>>>>>>>> So, moving forward you can expect the bulk of the foreign API to be relatively stable (well, it's a prototype, so we might tweak things here and there); what will really change is how this API is _implemented_ - that is, moving forward the foreign API will be built _on top_ of the lower memory access and ABI layers. But high-level use cases using jextract need not to worry about this.
>>>>>>>> 
>>>>>>>> I hope this clarifies better where we'd like to land.
>>>>>>> 
>>>>>>> 
>>>>>>> Yes, it does greatly. Thanks for clarifying.
>>>>>>> 
>>>>>>> 
>>>>>>> Speaking of the Pointer API, could a method be added to wrap a pointer in another pointer for **char string pointers? AFAIK, the only way to do that is:
>>>>>>> 
>>>>>>> 
>>>>>>> <LIB>.scope().allocate(<LIB>.scope().allocateCString("").type().pointer()); 
>>>>>>> 
>>>>>>> 
>>>>>>> ...which gets the Pointer<Pointer<Byte>> type that I need but I'm not entirely sure if this is the correct way to go about getting the type. Using the layout of a throwaway pointer layout just seems wrong and wasteful.
>>>>>> 
>>>>>> If you want to allocate a Pointer<Pointer<Byte>> you can do this:
>>>>>> 
>>>>>> 
>>>>>> <LIB>.scope().allocate(NativeTypes.UINT8.pointer());
>>>>>> 
>>>>> 
>>>>> Is the size the same as a C String with that?
>>>> 
>>>> When you allocate a C string using allocateCString, you get back a Pointer<Byte>; this has an 'address' layout - that is, the size of this value is platform dependent, but on x64 platforms you can assume it's 64 bits.
>>>> 
>>>> So, when you have a Pointer<Byte> and call type() on it, as in your previous code, you get the pointee layout type - that for 'Byte' (which is 8 bits). At which point you are calling pointer() on it, which is sending you back to a pointer layout type, with size 64 (again assuming we're on x64). So, the outer allocate will allocate 64 bits.
>>>> 
>>>> The code I suggested does the same - but without the throwaway allocation.
>>>> 
>>>> 
>>>> That said - I think what you really wanted to ask, is another question, one that has been raised before: assuming I have a Pointer<Byte> representing a C string, how do I get a pointer to that?
>>>> 
>>>> Am I correct?
>>> 
>>> 
>>> Yes.
>>> 
>>> 
>>> The function calls for a **char(documented as a string) but there is no obvious way to wrap the type returned by allocateCString(Pointer<Byte>) into another Pointer(Pointer<Pointer<Byte>>), hence the wasteful code above.
>> 
>> So, let's say we have this:
>> 
>> Pointer<Byte> c_str = scope.allocateCString("Hello");
>> 
>> Now we have to create a pointer to that pointer. To do that:
>> 
>> Pointer<Pointer<Byte>> p_c_str = scope.allocate(NativeTypes.UINT8.pointer());
>> 
>> And then initialize the contents of the pointer to pointer:
>> 
>> p_c_str.set(c_str);
>> 
>> You can now pass p_c_str to your function.
>> 
>> Is this what you wanted to do?
> 
> 
> Yes.
> 
> 
>> 
>> Maurizio
>> 
>>> 
>>> 
>>>> 
>>>> Maurizio
>>>> 
>>>>> 
>>>>> 
>>>>>> 
>>>>>> Maurizio
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> Cheers
>>>>>>>> Maurizio
>>>>>>>> 
>>>>>>>> On 19/08/2019 10:33, sundararajan.athijegannathan at oracle.com wrote:
>>>>>>>>>> 
>>>>>>>>>> All that said, how close is Panama? Is this foreign memory API going to stay going forward or will the project take a major shift? I'd *really* like to start putting this to use and am willing to make adjustment where needed if minor changes are made, but if the entire foreign API is scrapped it isn't worth it.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Panama "memory access" ("memaccess" panama-dev branch) API is expected to become stable first and then other parts of java.foreign later ("foreign" branch stuff).