Arena/Segment allocator and zero initialized MemorySegment

[Maurizio Cimadamore]

Let me uplevel the discussion a bit, because I think we're getting hung 
up on zeroing (which is a pesky issue, because of conflicting 
requirements e.g. safety vs. performance) but might be missing the 
broader context.

It is my understanding that your old implementation used a combination 
of Unsafe::allocate/set/reallocate/freeMemory. Then some kind of 
automatic memory deallocation would take place (you mentioned finalizers 
being used).

So, at the very high-level there is a first fork in the road: do you 
want to use restricted methods or not?

If you are ok with using restricted methods, then you can reimplement 
basically what you have on top of the C malloc/realloc/free (and use 
MemorySegment::fill to zero memory). This should be an easy 
apple-to-apple change. This should provide the same performance model as 
your existing solution (except perhaps for some more allocation of 
memory segment wrappers).

Given the question you asked, I suspect you either are not ok with using 
restricted methods or were simply curious to see if there was a more 
supported way to achieve the same functionality (which is a fair point). 
Now I think that an alternate, non-restricted way to do what you are 
after does exist, but to get there you have to embrace the way in which 
the abstractions provided by FFM work, rather than trying to bend them 
to do what your old code was doing. E.g. if you want to go the 
"supported" path you have to be prepared to change your code a little, 
and it is also possible that the resulting code will not have exactly 
the same performance model as the one it replaces.

Now, if you still want to go down this path, since your old system was 
using automatic deallocation and finalizers, I'd claim that the simplest 
way to move to non-restricted FFM would be to use automatic arenas 
everywhere. You did not specify how deallocation of your off-heap arrays 
worked. E.g. was the finalizer in the List class responsible for 
deallocating _all_ the arrays allocated by that List instance? Or was 
each array lifetime managed independently so that, e.g. after a "resize" 
the old off-heap array could have been deallocated (depending on how the 
GC wanted to do things) ? That is, what was the _lifecycle_ of the 
arrays in your library? If every list instance implied its own 
lifecycle, then that would suggest that, in the FFM world, a List should 
encapsulate its own automatic Arena. If, on the other hand, each array 
had its own independent lifetime, then I'd say that a List would simply 
point to the "current" segment, where that segment has been allocated 
(or reallocated) using a one-shot automatic arena (e.g. 
`Arena.ofAuto().allocate(...)`).

This is, I think the easiest way to get what you had on top of the 
non-restricted portion of FFM. Note that, since your List instance is in 
control of allocation, no segment/arena parameters are required - users 
just allocate a new list, and the list creates the automatic arena and 
performs the allocation with it. And, since you control the arena, you 
can pick an arena that has the zeroing semantics you want (again, an 
automatic arena is fine here).

But maybe you wanted the *client* of the List to be in control of when 
the memory associated with a List should be deallocated, and also maybe 
let the client decide how memory should be allocated (in case the client 
has a custom arena with more efficient allocation strategy). If that's 
the case, that can again be accommodated: your List would take an Arena 
from outside as a constructor parameter (an Arena, not just a 
SegmentAllocator, since we need a lifetime here, not just a lone 
segment). It will then use it to allocate all the segments it needs. The 
client is in charge as to when to close the arena, and release the 
memory. For this to work well, I think it's much better to work with a 
growable list of segments, as I explained in my previous email. That is, 
rather than first allocating a segment of 16 bytes, then allocating a 
bigger one of 256 bytes when you run out of space and copy the old data 
over, I think it's much better to allocate fixed size segments - say 256 
bytes (this is an example, you probably have a much better sense of 
which numbers to use). When you run out of space, you just allocate a 
new one, and leave the old one in place. Some logic will be added to 
List::get/add to make sure that the logical index is mapped to the 
correct segment/offset.

If you go down this path, you now have a conflict. On the one hand you 
want the client to be in control (by providing an arena). On the other 
hand, your API is sensitive to zeros (for empty lists, and empty map 
key-value pairs) and might break if the user-provided arena does not 
zero the segments it generates. To resolve this conflict, you have few 
options: you could require in the documentation of the API that the user 
provides a "zeroing" arena. Or, you could proactively zero the segments 
that come out of the provided arena. This can be done either by just 
calling MemorySegment::fill redundantly (for now, maybe will be 
optimized by C2 later). Or you could use the `allocateFrom` trick to 
allocate and copy a "zero segment" in one shot (so that no double 
zeroing occurs).

IMHO documenting the API expectations, and leave it to clients to do the 
right things is not as bad as it sounds. In fact I'd claim it's pretty 
standard with most APIs. E.g. if an API is not thread-safe, we do expect 
clients not to use it in a racy way. Should there be a way to ask the 
provided arena if it's a zeroing one (so that your API can throw if the 
arena has the wrong polarity) ? Maybe (assuming that is the only 
interesting question we can ask an allocator, which I doubt), but now 
we're discussing a corner of one of all the possible implementation 
options provided in this email, so I'd suggest that, instead of jumping 
on trying to address the zeroing problem, we try to reach a shared 
understanding of what your old API was trying to do.

Cheers
Maurizio




On 05/06/2024 10:03, Remi Forax wrote:
> Hello,
> for one of my API, I want to take a SegmentAllocator as parameter, and this allocator whould provides zero initialized MemorySegment, but it seems there is no way to know if a SegmentAllocator or an Arena will intialize the MemorySegment with zeroes when allocate() is callled.
>
> I believe what is missing is a default method isZeroInitialized() in SegmentAllocator that returns false by default and true for the 4 global(), ofAuto(), ofConfined() and ofShared() implementations.
>
> regards,
> Rémi
>