Arena/Segment allocator and zero initialized MemorySegment

[forax at univ-mlv.fr]

> From: "Maurizio Cimadamore" <maurizio.cimadamore at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "panama-dev" <panama-dev at openjdk.org>
> Sent: Wednesday, June 5, 2024 12:18:47 PM
> Subject: Re: Arena/Segment allocator and zero initialized MemorySegment

> On 05/06/2024 11:11, [ mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] wrote:

>> Here is an example [
>> https://urldefense.com/v3/__https://github.com/forax/memory-mapper/blob/master/src/main/java/com/github/forax/memorymapper/MemoryCollections.java*L772__;Iw!!ACWV5N9M2RV99hQ!PUJIR9R4tNJ_QKk01hd0VFW5mMO0KsBeJk0WxuQsq4MzmNZ0c5WvuGhrAHSJQJGDJog9qIZghlkqNDjgBnxZ8JXAYA$
>> |
>> https://urldefense.com/v3/__https://github.com/forax/memory-mapper/blob/master/src/main/java/com/github/forax/memorymapper/MemoryCollections.java*L772__;Iw!!ACWV5N9M2RV99hQ!PUJIR9R4tNJ_QKk01hd0VFW5mMO0KsBeJk0WxuQsq4MzmNZ0c5WvuGhrAHSJQJGDJog9qIZghlkqNDjgBnxZ8JXAYA$
>> ] I want to have a List of records (or primitive type) implemented as an
>> array of flatten values.
>> I need a way to allocate a zeroed MemorySegment, and I need a way to
>> grow the storage using the Array.copyOf() semantics.

>> For the history, I've an existing non-public library that uses Unsafe +
>> ASM for the code generation and i'm trying to replace it with a new
>> publicly available library that uses the foreign memory API + the method
>>  handle API + Hidden classes.

> As you are allocating only one segment, the use of SegmentAllocator seems fine
> here. But... if you later on need to grow storage, it's not a one-shot
> allocation anymore, and I believe you need an Arena (as you might need to start
> managing different segments with same lifetime).

> You can also go a different route, and use calloc/realloc directly (via Linker),
> and avoid the allocator parameter (not sure how your old API dealt with
> deallocation safely).
The old library uses Unsafe.allocateMemory()/reallocateMemory()/setMemory()/copyMemory()/freeMemory() and uses the finalizer mechanism for the safe but late deallocation. 

For now, the new library do not try to reallocate, so when it needs to grow the storage, it asks for a new MemorySegment and copy the data. 
It seems I should use an Arena instead of a SegmentAllocator because two subsequent calls to allocate() should not return the same address, but i have two reasons to not using it. Arena does not garantee that the storage is zeroed and it keeps the storage until the Arena is closed so if the list grows several times, it will keep all previous segments in memory. 

As far as i understand what an Arena is, an Arena is virtual, it's not contiguous memory*, so supporting deallocation is possible (i'm not saying it should) and having a way to ask for a zero initialized MemorySegement is also something possible (again, not saying it should). 

> Maurizio
Rémi 
* it should be stated more clearly in the javadoc of Arena, IMO, the only other runtime system i know that uses the word arena with the same semantics is Zig, all others are using.arena with the meaning of a region based allocator (see [ https://en.wikipedia.org/wiki/Special:Search?go=Go&search=arena%20memory) | https://en.wikipedia.org/wiki/Special:Search?go=Go&search=arena%20memory) ] . 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240605/f36ba9a4/attachment-0001.htm>