Arena/Segment allocator and zero initialized MemorySegment

[forax at univ-mlv.fr]

----- Original Message -----
> From: "Maurizio Cimadamore" <maurizio.cimadamore at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>, "panama-dev" <panama-dev at openjdk.org>
> Sent: Wednesday, June 5, 2024 11:28:18 AM
> Subject: Re: Arena/Segment allocator and zero initialized MemorySegment

> Hi Remi,
> this is a bit of a controversial area, and it is rather easy to end up
> in a slippery slope. Segment allocator is a rather general API, you can
> think of it as a general allocation trait (e.g. all the ways in which a
> segment can be allocated).
> 
> As such, allocators don't provide many guarantees. As you noticed,
> there's no guarantee that the memory you get back if you do e.g.
> allocate(100) will be zero-initialized. There's also no guarantee that,
> if you call allocate(100) twice you will get back _different_ memory
> segments (e.g. aliasing). The only think SegmentAllocator promises is
> that when you get back a segment, the segment will be (a) big enough and
> (b) be aligned consistently with what you requested.
> 
> I'd say that, for most APIs, this behavior is too general to be useful.
> This is why Arena for example promises that the segments it returns will
> never be aliased (e.g. if you call allocate(100) twice on an arena, you
> will get back two different segments). But Arena still doesn't make any
> promises when it comes to zeroing (e.g. if you call allocate(100) the
> API won't say whether the returned segment will be zero-initialized or
> not). And I don't think it should make that promise - how much zeroing
> an application is willing to live with is something that is ultimately
> application-dependent, so making a choice for all applications seems
> questionable.

Thinking a little more, i think what i need is the semantics of calloc (I also need realloc, exactly more Arrays.copyOf()).

> 
> Can you describe a bit more what kind of stuff your method will do with
> the allocator? How many segments is it allocating (one or more?), and
> what should happen if, say, the provided allocator doesn't feature the
> required characteristics (e.g. throw, or workaround, e.g. by calling
> allocateFrom with a zeroed array, which will also zero the returned
> segment) ?

Here is an example
https://github.com/forax/memory-mapper/blob/master/src/main/java/com/github/forax/memorymapper/MemoryCollections.java#L772

I want to have a List of records (or primitive type) implemented as an array of flatten values.
I need a way to allocate a zeroed MemorySegment, and I need a way to grow the storage using the Array.copyOf() semantics.

For the history, I've an existing non-public library that uses Unsafe + ASM for the code generation and i'm trying to replace it with a new publicly available library that uses the foreign memory API + the method handle API + Hidden classes.

> 
> Cheers
> Maurizio

regards,
Rémi

> 
> 
> 
> 
> On 05/06/2024 10:03, Remi Forax wrote:
>> Hello,
>> for one of my API, I want to take a SegmentAllocator as parameter, and this
>> allocator whould provides zero initialized MemorySegment, but it seems there is
>> no way to know if a SegmentAllocator or an Arena will intialize the
>> MemorySegment with zeroes when allocate() is callled.
>>
>> I believe what is missing is a default method isZeroInitialized() in
>> SegmentAllocator that returns false by default and true for the 4 global(),
>> ofAuto(), ofConfined() and ofShared() implementations.
>>
>> regards,
>> Rémi