Pointer ofNull method naming

[Ty Young]

On 11/14/19 6:13 AM, Maurizio Cimadamore wrote:
> Seems to me that what you have built is something that gives you a 
> bunch of constant pointers of the right type - whose contents are left 
> empty (and likely not meant to be de-referenceable?).


Yes. The idea is to avoid needless Pointer object allocations. While the 
allocation every once in awhile isn't a big deal, it can add up if done 
from multiple function calls which are called once every 100ms(e.g. 
polling hardware usage from some API).


>
> This seems pretty specific to the use case you envision and the API 
> you are interacting with. I can't see this being made available as a 
> part of the foreign API.


Fair enough.


>
> As for 'what are Scopes for' - I think you might be confused a bit 
> here; a scope is a unit of allocation. All stuff crated inside a scope 
> is deallocated at once when the scope is closed. There are some 
> restrictions on putting pointers from scope A into scope B - but the 
> restriction checks that, when you do that, the lifecycle of A is 
> bigger than that of B (e.g. make sure that accessing the pointer from 
> B will never give you an invalid pointer).


I'm a bit confused as to why this is needed. If the pointer is 
dereferenced(pointer variable reference set to null), then isn't the 
object GC'd like any other object? Or does Panama have strong references 
internally to avoid GC?


Presuming that they aren't GC'd until the Scope is closed, what if you 
only need to use a native API once in a very long time span? You'd have 
to keep the scope open since there is no way to reopen it. It seems like 
a waste of memory, IMO.


>
> Since your code is using the global scope to allocate - and the global 
> scope outlives everything, you can safely store these pointers in any 
> other scope. This is not a bug, it's a feature.


It's a good feature.


>
> Maurizio
>
>
> On 14/11/2019 11:19, Ty Young wrote:
>>
>> On 11/13/19 10:01 PM, Henry Jen wrote:
>>> Let me see if I understand correctly, the native API basically have 
>>> an output and expecting a pointer to receive it. if you don’t care, 
>>> you can pass NULL as in C, right?
>>
>>
>> Yes, although the fact that the Pointer happens to be Null is kinda 
>> irrelevant. What's important is that the Pointer is a singleton of 
>> the needed function Pointer type.
>>
>>
>>>
>>> Because the native API accept NULL pointer, that’s why you can use 
>>> Pointer.ofNull(). As Mauricio mentioned, we are reconsidering if we 
>>> can just use java null instead of Pointer.ofNull(), which I prefer 
>>> personally.
>>
>>
>> As in a straight up "null" value being passed? FWIW, I don't like 
>> that idea personally. C source code clearly asks for a null Pointer, 
>> not a null value. Code documentation will probably even state this 
>> and people will go looking for a Pointer representing a null value, 
>> like Pointer.ofNull().
>>
>>
>> (Although Pointer.ofNull conforms to the function argument type using 
>> a null layout, not Null which is semi confusing)
>>
>>
>> The differences between what C and Java sees in regards to function 
>> calls is already annoyingly painful as-is with enums being converted 
>> to ints, forcing you to create Java equivalent enums in order to make 
>> function calls humanly readable. Please don't make it any worse.
>>
>>
>> Anyway, I decided to just make what I want/need[1]. I'm still really 
>> confused as to what the point of Scopes are, seeing as you can use 
>> Pointers from one scope on another scope without issues.
>>
>>
>> [1] https://gist.github.com/BlueGoliath/3fe527ea9bc7df2583a335962a02e1fe
>>
>>
>>>
>>> Cheers,
>>> Henry
>>>
>>>> On Nov 13, 2019, at 4:46 PM, Ty Young <youngty1997 at gmail.com> wrote:
>>>>
>>>>
>>>> On 11/13/19 6:01 PM, Maurizio Cimadamore wrote:
>>>>> Sorry, still not getting what you are trying to say. Let's pop 
>>>>> back; you have this function:
>>>>>
>>>>> int foo(Pointer<Pointer<Byte>> array, Pointer<Integer> size);
>>>>>
>>>>> Can you please explain what this function is expected to do? What 
>>>>> are the inputs and what are the outputs? How is 'size' connected 
>>>>> 'array'?
>>>>
>>>> Pointer<Pointer<Byte>> arrayPointer = 
>>>> Bar.scope().allocate(NativeTypes.CHAR.pointer());
>>>>
>>>> Pointer<Integer> sizePointer = Bar.scope().allocate(NativeTypes.INT);
>>>>
>>>>
>>>> Bar.foo(sizePointer, sizePointer);
>>>>
>>>> Array<Byte> bytes = arrayPointer.get().withSize(size.get());
>>>>
>>>> for(int i = 0; i < bytes.length(); i++)
>>>>      System.out.println(bytes.get(i));
>>>>
>>>>
>>>> Is how you're supposed to use it, but if you aren't interested in 
>>>> in the bitmask values you can just do:
>>>>
>>>>
>>>> arrayPointer.get().get()
>>>>
>>>>
>>>> which returns the first array element, saving both a Pointer and 
>>>> array allocation.
>>>>
>>>>
>>>>
>>>>> What is that 'int' return value?
>>>>
>>>> It's a "Bool" in C. Just returns whether the function call 
>>>> succeeded or not.
>>>>
>>>>
>>>>> I vaguely understand (from your previous emails) that the 'array' 
>>>>> is expected to have a certain encoding where first element is 
>>>>> 'size' (of what?)
>>>>
>>>> It's... complicated. The example provided is extremely simplified. 
>>>> The *actual* function uses 6 parameters, including enums(ints in 
>>>> Java) as well as index values to produce different results for a 
>>>> wide range of things. It's nuts.
>>>>
>>>>
>>>> What the byte value at index 0 represents depends on the arguments 
>>>> passed to this unholy abomination of a function, like the number of 
>>>> fans or displays connected. Some of the parameters are even optional.
>>>>
>>>>
>>>>> then you have some element bitmasks. But it's not clear whether 
>>>>> the 'array' is provided as input by the user, or is expected to be 
>>>>> filled by the function (same for 'size').
>>>>
>>>> Both.
>>>>
>>>>
>>>>> Maurizio
>>>>>
>>>>> On 13/11/2019 23:01, Ty Young wrote:
>>>>>> On 11/13/19 4:28 PM, Maurizio Cimadamore wrote:
>>>>>>> On 13/11/2019 22:15, Ty Young wrote:
>>>>>>>> On 11/13/19 3:19 PM, Maurizio Cimadamore wrote:
>>>>>>>>> On 13/11/2019 12:43, Ty Young wrote:
>>>>>>>>>> On 11/13/19 2:28 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>> I'm ok with making the documentation more precise (I'm not 
>>>>>>>>>>> sure I like ofTyped()). Can you please clarify a bit what 
>>>>>>>>>>> you mean by 'filler' ? Like passing null ptr to a struct 
>>>>>>>>>>> setter?
>>>>>>>>>>
>>>>>>>>>> Not struct types but more primitive native types like 
>>>>>>>>>> Pointer<Byte> or Pointer<Integer>.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> As an example, you have a function like this:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> int foo(Pointer<Pointer<Byte>> array, Pointer<Integer> size);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But what you're actually interested in is at index 0 of the 
>>>>>>>>>> array, so "size" isn't needed and allocating a new Pointer 
>>>>>>>>>> would be a waste.
>>>>>>>>> Not sure I follow - if you are interested in index 0, doesn't 
>>>>>>>>> that mean that the array would need to have at least _one_ 
>>>>>>>>> element (e.g. size >= 1) ?
>>>>>>>>
>>>>>>>> It does. It's just that the data that is actually desired is 
>>>>>>>> located at index 0, so doing:
>>>>>>>>
>>>>>>>>
>>>>>>>> array.get().get()
>>>>>>>>
>>>>>>>>
>>>>>>>> returns index 0 anyway.
>>>>>>>>
>>>>>>>>
>>>>>>>> For context, the function in question returns a Pointer to an 
>>>>>>>> array in the format:
>>>>>>>>
>>>>>>>>
>>>>>>>> Index 0 = number of items
>>>>>>>>
>>>>>>>> index 4 * number of item = bitmask of a given item
>>>>>>> Really confused by this example - if the size is always at 
>>>>>>> position 0 of the array, then what is the second argument 
>>>>>>> Pointer<Integer> used for?
>>>>>>
>>>>>> "size" is the amount of something connected to the system, not 
>>>>>> the Pointer<Integer> size.
>>>>>>
>>>>>>
>>>>>> So the bitmask of item 1 is located at index 4, item 2 at 8, etc. 
>>>>>> in the array.
>>>>>>
>>>>>>
>>>>>>> I'm trying to understand whether this is really a 'new' use for 
>>>>>>> Pointer.ofNull, or whether you are just exploiting the fact that 
>>>>>>> the underlying function you are using happens to work even if 
>>>>>>> the second parameter is null.
>>>>>>
>>>>>> It does exploit the lack of null checks *technically* although 
>>>>>> you could just as easily lazily allocate a non null Pointer of 
>>>>>> the desired type to do the same thing. Would providing something 
>>>>>> like that be out of API scope?
>>>>>>
>>>>>>
>>>>>> Although, it's never explained(nor have I tried it) whether a 
>>>>>> Pointer allocated from a different scope can be used in another 
>>>>>> one so maybe that isn't even possible? If it is, then what's the 
>>>>>> point of Scope class anyway?
>>>>>>
>>>>>>
>>>>>>> in other words, I'm having problems understand what you mean by 
>>>>>>> this 'filler' capability of Pointer.ofNull.
>>>>>>
>>>>>> The function requires a pointer be passed even if you aren't 
>>>>>> interested in what's inside that pointer after calling the 
>>>>>> function, therefor it's "filler".
>>>>>>
>>>>>>
>>>>>>> Maurizio
>>>>>>>
>>>>>>>>
>>>>>>>> However, one might only be interested in index 0 or in this 
>>>>>>>> case, the number of items, so allocating another Pointer just 
>>>>>>>> to make the function call seems like a waste.
>>>>>>>>
>>>>>>>>
>>>>>>>> It seems like *some* functions check whether a given Pointer is 
>>>>>>>> null or not but not all do such as the functions I'm working 
>>>>>>>> with. It just fills the array Pointer regardless.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> (p.s. in these cases we have been also discussing bringing 
>>>>>>>>>>> back special treatment for Java null, so that null is 
>>>>>>>>>>> re-interpreted to Pointer.ofNull)
>>>>>>>>>>
>>>>>>>>>> I haven't gone too deep down the rabbit hole, but I see 
>>>>>>>>>> "LayoutTypeImpl.nullType" in BoundedPointer. Isn't that the 
>>>>>>>>>> null interpretation for a Pointer?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Maurizio
>>>>>>>>>>>