[foreign] Panama EA - August 2019 edition

[Maurizio Cimadamore]

On 30/08/2019 21:12, Maurizio Cimadamore wrote:
>
> On 30/08/2019 20:07, Ty Young wrote:
>>
>> On 8/30/19 12:54 PM, Maurizio Cimadamore wrote:
>>>
>>> On 30/08/2019 18:41, Ty Young wrote:
>>>>
>>>> On 8/30/19 12:25 PM, Maurizio Cimadamore wrote:
>>>>>
>>>>> On 30/08/2019 17:59, Ty Young wrote:
>>>>>>
>>>>>> On 8/28/19 5:11 AM, Maurizio Cimadamore wrote:
>>>>>>> I just want to clarify on this point below; the foreign memory 
>>>>>>> access work does not, in any way, hinder the higher level 
>>>>>>> functionalities provided by the foreign API/binder. We arrived 
>>>>>>> at the foreign memory access API because we felt that something 
>>>>>>> low level was missing - e.g. that the high level Pointer API was 
>>>>>>> doing too much at once; and that users not interested in a 
>>>>>>> high-level API, but still wanting to access off-heap data would 
>>>>>>> not be served very well by the Pointer API alone.
>>>>>>>
>>>>>>> So, moving forward you can expect the bulk of the foreign API to 
>>>>>>> be relatively stable (well, it's a prototype, so we might tweak 
>>>>>>> things here and there); what will really change is how this API 
>>>>>>> is _implemented_ - that is, moving forward the foreign API will 
>>>>>>> be built _on top_ of the lower memory access and ABI layers. But 
>>>>>>> high-level use cases using jextract need not to worry about this.
>>>>>>>
>>>>>>> I hope this clarifies better where we'd like to land.
>>>>>>
>>>>>>
>>>>>> Yes, it does greatly. Thanks for clarifying.
>>>>>>
>>>>>>
>>>>>> Speaking of the Pointer API, could a method be added to wrap a 
>>>>>> pointer in another pointer for **char string pointers? AFAIK, the 
>>>>>> only way to do that is:
>>>>>>
>>>>>>
>>>>>> <LIB>.scope().allocate(<LIB>.scope().allocateCString("").type().pointer()); 
>>>>>>
>>>>>>
>>>>>>
>>>>>> ...which gets the Pointer<Pointer<Byte>> type that I need but I'm 
>>>>>> not entirely sure if this is the correct way to go about getting 
>>>>>> the type. Using the layout of a throwaway pointer layout just 
>>>>>> seems wrong and wasteful.
>>>>>
>>>>> If you want to allocate a Pointer<Pointer<Byte>> you can do this:
>>>>>
>>>>>
>>>>> <LIB>.scope().allocate(NativeTypes.UINT8.pointer());
>>>>>
>>>>
>>>> Is the size the same as a C String with that?
>>>
>>> When you allocate a C string using allocateCString, you get back a 
>>> Pointer<Byte>; this has an 'address' layout - that is, the size of 
>>> this value is platform dependent, but on x64 platforms you can 
>>> assume it's 64 bits.
>>>
>>> So, when you have a Pointer<Byte> and call type() on it, as in your 
>>> previous code, you get the pointee layout type - that for 'Byte' 
>>> (which is 8 bits). At which point you are calling pointer() on it, 
>>> which is sending you back to a pointer layout type, with size 64 
>>> (again assuming we're on x64). So, the outer allocate will allocate 
>>> 64 bits.
>>>
>>> The code I suggested does the same - but without the throwaway 
>>> allocation.
>>>
>>>
>>> That said - I think what you really wanted to ask, is another 
>>> question, one that has been raised before: assuming I have a 
>>> Pointer<Byte> representing a C string, how do I get a pointer to that?
>>>
>>> Am I correct?
>>
>>
>> Yes.
>>
>>
>> The function calls for a **char(documented as a string) but there is 
>> no obvious way to wrap the type returned by 
>> allocateCString(Pointer<Byte>) into another 
>> Pointer(Pointer<Pointer<Byte>>), hence the wasteful code above.
>
> So, let's say we have this:
>
> Pointer<Byte> c_str = scope.allocateCString("Hello");
>
> Now we have to create a pointer to that pointer. To do that:
>
> Pointer<Pointer<Byte>> p_c_str = 
> scope.allocate(NativeTypes.UINT8.pointer());
>
> And then initialize the contents of the pointer to pointer:
>
> p_c_str.set(c_str);
>
> You can now pass p_c_str to your function.
>
> Is this what you wanted to do?

And, I think here what bugs you is the need to allocate the outer 
pointer - you want something akin to C's & operator.

I made a proposal few months ago - which I've been able to find :-)

https://mail.openjdk.java.net/pipermail/panama-dev/2019-February/004629.html

Didn't get much traction back then - but maybe we should revisit?

Maurizio

>
> Maurizio
>
>>
>>
>>>
>>> Maurizio
>>>
>>>>
>>>>
>>>>>
>>>>> Maurizio
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Cheers
>>>>>>> Maurizio
>>>>>>>
>>>>>>> On 19/08/2019 10:33, sundararajan.athijegannathan at oracle.com wrote:
>>>>>>>>>
>>>>>>>>> All that said, how close is Panama? Is this foreign memory API 
>>>>>>>>> going to stay going forward or will the project take a major 
>>>>>>>>> shift? I'd *really* like to start putting this to use and am 
>>>>>>>>> willing to make adjustment where needed if minor changes are 
>>>>>>>>> made, but if the entire foreign API is scrapped it isn't worth 
>>>>>>>>> it.
>>>>>>>>
>>>>>>>>
>>>>>>>> Panama "memory access" ("memaccess" panama-dev branch) API is 
>>>>>>>> expected to become stable first and then other parts of 
>>>>>>>> java.foreign later ("foreign" branch stuff).