Ability to extend a MemorySegment

[Michael Zucchi]

On 21/1/20 11:37 am, Ty Young wrote:
>> A memory segment has a fixed size - its boundaries are set on 
>> construction (pretty much like in a ByteBuffer). Think of a 
>> MemorySegment as the result of a 'malloc' - if you want something 
>> fancier than that, it's likely you're trying to use a memory segment 
>> in a way it's not intended to.
>
>
> How exactly is expanding the length of a smaller MemorySegment into a 
> larger MemorySegment not an intended use? Even beyond arrays, there 
> are plenty of cases where one might, for example, need to swap an 
> Integer MemorySegment into a Long MemorySegment or something larger 
> for a specific platform. Surely using realloc would be faster/better? 

There's a much better solution to that case, just allocate the larger of 
the possible items - no need to even check what the size is when you use 
it.  This is exactly the approach one would use in C - it's effectively 
a union.  Seems an odd example though, it's not like you can migrate a 
running application to a different platform and so you have to know all 
these sizes before you get to that point.

Java's ArrayList never grows the array since you can't grow Java arrays 
and that doesn't seem to have hurt it much for most uses.  In cases 
where it is a problem you just use a more appropriate data structure.

realloc isn't magic either.  It isn't guaranteed to do anything more 
than alloc+copy+free.  The one in glibc tries not to do that but it 
relies on there being a following free block that is big enough, and 
once you get to larger allocations (128K by default in the source) it 
switches to mmap blocks and relies on operating system support and 
available holes in the virtual memory map.

I think MemorySegment is way over-engineered already, adding more 
complexity doesn't sound great.  I wish it was just a 1:1 mapping to a 
malloc block (or slice thereof), and had no support for arrays or array 
bytebuffers.  And also that MemoryAddress was literally just an address 
inside of it's segment - and not an 'offset' which may or may not 
actually be an address (with no direct way to find out - from java).  
The need to support this other stuff just makes the api weird and 
confusing since the naming conventions don't make sense if it was just 
backed by malloc and for all that you can never actually use these 
array-based segments with "foreign" functions anyway.  It this stuff 
there just as a (clumsy) mechanism to allow for the general copy() 
method?    I'm sure there are reasons it's the way it is but the whole 
thing is just so, well, 'foreign', to anything from either C or Java.  
And an even more restricted allocator?  Ouch.