Pointer ofNull method naming

Thu Nov 14 15:57:56 UTC 2019

> On Nov 14, 2019, at 4:50 AM, Ty Young <youngty1997 at gmail.com> wrote:
> 
> 
> On 11/14/19 6:13 AM, Maurizio Cimadamore wrote:
>> Seems to me that what you have built is something that gives you a bunch of constant pointers of the right type - whose contents are left empty (and likely not meant to be de-referenceable?).
> 
> 
> Yes. The idea is to avoid needless Pointer object allocations. While the allocation every once in awhile isn't a big deal, it can add up if done from multiple function calls which are called once every 100ms(e.g. polling hardware usage from some API).
> 

You want a singleton instance as a static buffer to receive data, like the infamous errno in C. As long as you are aware the implications, feel free to do so. That’s just not what Pointer.ofNull is for, the Pointer.ofNull() does NOT have buffer.

> 
>> 
>> As for 'what are Scopes for' - I think you might be confused a bit here; a scope is a unit of allocation. All stuff crated inside a scope is deallocated at once when the scope is closed. There are some restrictions on putting pointers from scope A into scope B - but the restriction checks that, when you do that, the lifecycle of A is bigger than that of B (e.g. make sure that accessing the pointer from B will never give you an invalid pointer).
> 
> 
> I'm a bit confused as to why this is needed. If the pointer is dereferenced(pointer variable reference set to null), then isn't the object GC'd like any other object? Or does Panama have strong references internally to avoid GC?
> 

Pointer object is GC, but the memory it points to does not. Scope is managing the allocation of native memory.

> 
> Presuming that they aren't GC'd until the Scope is closed, what if you only need to use a native API once in a very long time span? You'd have to keep the scope open since there is no way to reopen it. It seems like a waste of memory, IMO.
> 
> 

For Java objects, like Pointet itself, it’s managed by GC.
For native memory, you use a child scope by fork and close promptly when done with it.
Divide your allocation to proper scopes rather than using gloabalScope everywhere.

HTH,
Henry

>> 
>> Since your code is using the global scope to allocate - and the global scope outlives everything, you can safely store these pointers in any other scope. This is not a bug, it's a feature.
> 
> 
> It's a good feature.
> 
> 
>> 
>> Maurizio
>> 
>> 
>> On 14/11/2019 11:19, Ty Young wrote:
>>> 
>>> On 11/13/19 10:01 PM, Henry Jen wrote:
>>>> Let me see if I understand correctly, the native API basically have an output and expecting a pointer to receive it. if you don’t care, you can pass NULL as in C, right?
>>> 
>>> 
>>> Yes, although the fact that the Pointer happens to be Null is kinda irrelevant. What's important is that the Pointer is a singleton of the needed function Pointer type.
>>> 
>>> 
>>>> 
>>>> Because the native API accept NULL pointer, that’s why you can use Pointer.ofNull(). As Mauricio mentioned, we are reconsidering if we can just use java null instead of Pointer.ofNull(), which I prefer personally.
>>> 
>>> 
>>> As in a straight up "null" value being passed? FWIW, I don't like that idea personally. C source code clearly asks for a null Pointer, not a null value. Code documentation will probably even state this and people will go looking for a Pointer representing a null value, like Pointer.ofNull().
>>> 
>>> 
>>> (Although Pointer.ofNull conforms to the function argument type using a null layout, not Null which is semi confusing)
>>> 
>>> 
>>> The differences between what C and Java sees in regards to function calls is already annoyingly painful as-is with enums being converted to ints, forcing you to create Java equivalent enums in order to make function calls humanly readable. Please don't make it any worse.
>>> 
>>> 
>>> Anyway, I decided to just make what I want/need[1]. I'm still really confused as to what the point of Scopes are, seeing as you can use Pointers from one scope on another scope without issues.
>>> 
>>> 
>>> [1] https://gist.github.com/BlueGoliath/3fe527ea9bc7df2583a335962a02e1fe
>>> 
>>> 
>>>> 
>>>> Cheers,
>>>> Henry
>>>> 
>>>>> On Nov 13, 2019, at 4:46 PM, Ty Young <youngty1997 at gmail.com> wrote:
>>>>> 
>>>>> 
>>>>> On 11/13/19 6:01 PM, Maurizio Cimadamore wrote:
>>>>>> Sorry, still not getting what you are trying to say. Let's pop back; you have this function:
>>>>>> 
>>>>>> int foo(Pointer<Pointer<Byte>> array, Pointer<Integer> size);
>>>>>> 
>>>>>> Can you please explain what this function is expected to do? What are the inputs and what are the outputs? How is 'size' connected 'array'?
>>>>> 
>>>>> Pointer<Pointer<Byte>> arrayPointer = Bar.scope().allocate(NativeTypes.CHAR.pointer());
>>>>> 
>>>>> Pointer<Integer> sizePointer = Bar.scope().allocate(NativeTypes.INT);
>>>>> 
>>>>> 
>>>>> Bar.foo(sizePointer, sizePointer);
>>>>> 
>>>>> Array<Byte> bytes = arrayPointer.get().withSize(size.get());
>>>>> 
>>>>> for(int i = 0; i < bytes.length(); i++)
>>>>>      System.out.println(bytes.get(i));
>>>>> 
>>>>> 
>>>>> Is how you're supposed to use it, but if you aren't interested in in the bitmask values you can just do:
>>>>> 
>>>>> 
>>>>> arrayPointer.get().get()
>>>>> 
>>>>> 
>>>>> which returns the first array element, saving both a Pointer and array allocation.
>>>>> 
>>>>> 
>>>>> 
>>>>>> What is that 'int' return value?
>>>>> 
>>>>> It's a "Bool" in C. Just returns whether the function call succeeded or not.
>>>>> 
>>>>> 
>>>>>> I vaguely understand (from your previous emails) that the 'array' is expected to have a certain encoding where first element is 'size' (of what?)
>>>>> 
>>>>> It's... complicated. The example provided is extremely simplified. The *actual* function uses 6 parameters, including enums(ints in Java) as well as index values to produce different results for a wide range of things. It's nuts.
>>>>> 
>>>>> 
>>>>> What the byte value at index 0 represents depends on the arguments passed to this unholy abomination of a function, like the number of fans or displays connected. Some of the parameters are even optional.
>>>>> 
>>>>> 
>>>>>> then you have some element bitmasks. But it's not clear whether the 'array' is provided as input by the user, or is expected to be filled by the function (same for 'size').
>>>>> 
>>>>> Both.
>>>>> 
>>>>> 
>>>>>> Maurizio
>>>>>> 
>>>>>> On 13/11/2019 23:01, Ty Young wrote:
>>>>>>> On 11/13/19 4:28 PM, Maurizio Cimadamore wrote:
>>>>>>>> On 13/11/2019 22:15, Ty Young wrote:
>>>>>>>>> On 11/13/19 3:19 PM, Maurizio Cimadamore wrote:
>>>>>>>>>> On 13/11/2019 12:43, Ty Young wrote:
>>>>>>>>>>> On 11/13/19 2:28 AM, Maurizio Cimadamore wrote:
>>>>>>>>>>>> I'm ok with making the documentation more precise (I'm not sure I like ofTyped()). Can you please clarify a bit what you mean by 'filler' ? Like passing null ptr to a struct setter?
>>>>>>>>>>> 
>>>>>>>>>>> Not struct types but more primitive native types like Pointer<Byte> or Pointer<Integer>.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> As an example, you have a function like this:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> int foo(Pointer<Pointer<Byte>> array, Pointer<Integer> size);
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> But what you're actually interested in is at index 0 of the array, so "size" isn't needed and allocating a new Pointer would be a waste.
>>>>>>>>>> Not sure I follow - if you are interested in index 0, doesn't that mean that the array would need to have at least _one_ element (e.g. size >= 1) ?
>>>>>>>>> 
>>>>>>>>> It does. It's just that the data that is actually desired is located at index 0, so doing:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> array.get().get()
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> returns index 0 anyway.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> For context, the function in question returns a Pointer to an array in the format:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Index 0 = number of items
>>>>>>>>> 
>>>>>>>>> index 4 * number of item = bitmask of a given item
>>>>>>>> Really confused by this example - if the size is always at position 0 of the array, then what is the second argument Pointer<Integer> used for?
>>>>>>> 
>>>>>>> "size" is the amount of something connected to the system, not the Pointer<Integer> size.
>>>>>>> 
>>>>>>> 
>>>>>>> So the bitmask of item 1 is located at index 4, item 2 at 8, etc. in the array.
>>>>>>> 
>>>>>>> 
>>>>>>>> I'm trying to understand whether this is really a 'new' use for Pointer.ofNull, or whether you are just exploiting the fact that the underlying function you are using happens to work even if the second parameter is null.
>>>>>>> 
>>>>>>> It does exploit the lack of null checks *technically* although you could just as easily lazily allocate a non null Pointer of the desired type to do the same thing. Would providing something like that be out of API scope?
>>>>>>> 
>>>>>>> 
>>>>>>> Although, it's never explained(nor have I tried it) whether a Pointer allocated from a different scope can be used in another one so maybe that isn't even possible? If it is, then what's the point of Scope class anyway?
>>>>>>> 
>>>>>>> 
>>>>>>>> in other words, I'm having problems understand what you mean by this 'filler' capability of Pointer.ofNull.
>>>>>>> 
>>>>>>> The function requires a pointer be passed even if you aren't interested in what's inside that pointer after calling the function, therefor it's "filler".
>>>>>>> 
>>>>>>> 
>>>>>>>> Maurizio
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> However, one might only be interested in index 0 or in this case, the number of items, so allocating another Pointer just to make the function call seems like a waste.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> It seems like *some* functions check whether a given Pointer is null or not but not all do such as the functions I'm working with. It just fills the array Pointer regardless.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Maurizio
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> (p.s. in these cases we have been also discussing bringing back special treatment for Java null, so that null is re-interpreted to Pointer.ofNull)
>>>>>>>>>>> 
>>>>>>>>>>> I haven't gone too deep down the rabbit hole, but I see "LayoutTypeImpl.nullType" in BoundedPointer. Isn't that the null interpretation for a Pointer?
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> Maurizio
>>>>>>>>>>>> 