Ability to extend a MemorySegment

[Ty Young]

On 1/20/20 6:32 PM, Maurizio Cimadamore wrote:
> A memory segment has a fixed size - its boundaries are set on 
> construction (pretty much like in a ByteBuffer). Think of a 
> MemorySegment as the result of a 'malloc' - if you want something 
> fancier than that, it's likely you're trying to use a memory segment 
> in a way it's not intended to.


How exactly is expanding the length of a smaller MemorySegment into a 
larger MemorySegment not an intended use? Even beyond arrays, there are 
plenty of cases where one might, for example, need to swap an Integer 
MemorySegment into a Long MemorySegment or something larger for a 
specific platform. Surely using realloc would be faster/better?


>
> So, when you use a layout to do MemorySegment::allocateNative, the 
> layout is used to determine the size (and alignment) of the chunk of 
> memory to be allocated - so the size has to be known at segment creation.
>
> How would an array list be implemented in C? Or, how is it implemented 
> even in Java? You will see that the 'creating new array and copying 
> contents' is really how things are done at the low level. So, if you 
> want a variable-sized data structure based on MemorySegment, yes, you 
> will have to take into account the fact that the MemorySegment will 
> have to be thrown away and re-created with a new size (**)
>
> (*) Actually there is another possibility when working with 
> malloc/free which is "realloc" - it is theoretically possible to 
> provide a re-alloc API point to memory segment which would:
>
> - kill the current segment
> - realloc the underlying native memory to new size
> - create a new segment with new size
>
> But we will have to evaluate this possibility in conjunction with the 
> fact that, one day, memory segment might not exactly map to 
> malloc/free and instead use a different allocator (which might or 
> might not have ability to "reallocate"). Something to keep in mind 
> though.


Isn't it the expectation and/or assumption that Panama will use standard 
C allocators and not some other arbitrary allocator anyway? I understand 
it *technically* falls into the implementation detail bucket but 
malloc/free/realloc are standard C library functions...


>
> Maurizio
>
>
>
>
> On 20/01/2020 21:53, Ty Young wrote:
>> Hi,
>>
>>
>> Would it be possible to add the ability to extend a memory segment(in 
>> other words, increase its limit)? Right now it's not possible to 
>> implement dynamic arrays(read: ArrayList like arrays) as it's not 
>> possible to calculate the length of a SequenceLayout that has no 
>> specified length. You can work around this by creating a new array 
>> and copying the contents of the old array to it but that seems 
>> inefficient...
>>