Releasing MemorySegments eagerly and efficiently

[Jorn Vernee]

If MemorySegment had a free() method, anybody would be able to free a 
memory segment. So, if you want to let a client just use a memory 
segment, without the risk of it being freed, you would need some kind of 
way to create a non-closeable view of a segment, and you would have to 
do that every time you need to pass a segment to some other code. That 
is what we had in the past (around the time of JDK 14), and it lead to 
an awkward amount of defensive code creating non-closeable segments, as 
well as GC churn for those new objects. We moved away from that, and 
towards 'inadvertent free' being impossible by design: only someone who 
holds an arena can free the memory. Someone who just holds a memory 
segment can't. That is an important feature (this is discussed in detail 
in the document you linked to as well).


So, the API was designed around arenas governing lifetimes, not memory 
segments. If you want to manage the lifetime of some resource, Arena is 
the abstraction that should be used for that. As far as your case goes: 
you also have the option to embrace lifetimes, and expose them in your 
API. i.e. let your users (indirectly) create and close the arenas that 
should be used to manage these pointers. Doing this would also allow 
users to pick the arena kind that is appropriate for them, such as 
confined arenas, and also allows users to re-use the same arena multiple 
times for different resources.


Regarding safety: the main issue comes from threads racing to access & 
free the same memory. In a multi-threaded context, we can not simply 
check liveness around accesses, as the memory could be freed /during/ 
the access, resulting in memory corruption and/or crashes. Confined 
arenas avoid this issue by restricting the use of the memory to a single 
thread.


Jorn


On 17-2-2025 19:36, Benoit Daloze wrote:
> Thank you for the quick and detailed reply.
>
> Right, given the requirements, calling malloc() and free() ourselves 
> would work, and then we'd create MemorySegments with ofAddress() + 
> reinterpret().
>
> Regarding safety, would there by any safety issue if:
>
>  *
>     MemorySegment would have a free() method, which could only be
>     called if the MemorySegment doesn't belong to an Arena (and throw
>     otherwise). Or maybe the Global Arena could behave like that.
>  *
>     Slices of that MemorySegment would remember their root
>     MemorySegment, so that they would all check if the root segment is
>     freed before/around performing accesses (e.g. like SharedSession
>     does it).
>  *
>     Slices themselves either can't free(), or if they can they would
>     just delegate to the root MemorySegment.
>  *
>     The root MemorySegment could be auto-release for convenience.
>
> Do you see any safety issue with that design?
>
> I guess it's quite similar to having a Shared Arena per (non-slice) 
> MemorySegment but without the overhead of creating that Arena and 
> associated objects.
> ------------------------------------------------------------------------
> *From:* Maurizio Cimadamore <maurizio.cimadamore at oracle.com>
> *Sent:* Monday, February 17, 2025 17:31
> *To:* Benoit Daloze <benoit.daloze at oracle.com>; panama-dev at openjdk.org 
> <panama-dev at openjdk.org>
> *Subject:* Re: Releasing MemorySegments eagerly and efficiently
>
> Hi,
> when it comes to deallocation, I think you can summarize the situation 
> with efficient, shared, safe: pick two.
>
>
> You can have safe and efficient deallocation with confined access (or, 
> in the future, structured access).
>
>
> You can have safe, but less efficient shared deallocation with eithe 
> letting the GC do the work (reachability-base automatic arena) or 
> having some more expensive handshake when the arena is closed (shared 
> arena).
>
>
> You can have _unsafe_ efficient shared allocation: just wrap 
> malloc/free using the Linker API, and then use 
> MemorySegment::reinterpret to resize he unsafely allocated segments 
> accordingly. This will give you something closer to the C feel but it 
> is (as C is), unsafe (reinterpret is a restricted method).
>
>
> (There's also solution in the middle - by having pooling arenas you 
> can greatly reduce the cost of malloc/free -- but your mileage might 
> vary as those solutions will typically only work best if your memory 
> allocation is somewhat structured -- e.g. if allocation occurs in a 
> code block).
>
>
> What you end up using is up to you (and your requirements). The FFM 
> API supports all the above -- but of course the Arena API, that is 
> designed to be a safe API only gives you access to safe way to 
> allocate/deallocate segments.
>
>
> Maurizio
>
>
> On 17/02/2025 16:19, Benoit Daloze wrote:
>> Hello panama-dev,
>>
>> We are looking at migrating from Unsafe off-heap methods to 
>> Panama/MemorySegment.
>> While migrating we hit a blocker for replacing Unsafe#freeMemory() calls.
>>
>> Specifically, we are implementing various languages like Python and 
>> Ruby which have their own Pointer/Buffer-like abstractions (i.e. they 
>> support reading/writing various native types at some offset, 
>> allocation, freeing, etc).
>> Some of these pointers have a known size/length and some don't (and 
>> those that don't are allowed to read/write anywhere, as if they had 
>> infinite length).
>>
>> Some pointers/buffers themselves have auto-release semantics and some 
>> don't (e.g. when they come back from a native call), regardless of 
>> that these languages allow freeing them eagerly, e.g. 
>> https://www.rubydoc.info/gems/ffi/1.16.3/FFI/Pointer#free-instance_method 
>> <https://www.rubydoc.info/gems/ffi/1.16.3/FFI/Pointer#free-instance_method>
>>
>> Freeing eagerly is crucial to avoid excessive resource consumptions 
>> (especially if the allocations are big). It's similar to the problem 
>> of running out of file descriptors when not closing them explicitly, 
>> hence closing explicitly is best, otherwise one might run out of memory.
>>
>> Looking at the docs, there is no MemorySegment#free()/close() or so, 
>> and this seems intentional based on 
>> https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html 
>> <https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html>
>>
>> Instead, some Arenas can be close()'d.
>> Looking at Arena docs, of all 4 builtin arena types, only Confined 
>> and Shared are allowed to be closed explicitly/eagerly.
>> In general those pointer abstractions can be accessed by multiple 
>> threads, so Confined wouldn't work.
>>
>> Using a Shared Arena per pointer/buffer allocation seems very 
>> expensive both in footprint (need to keep various objects alive vs 
>> just a long with Unsafe) and in performance (though I have not 
>> benchmarked it yet against allocateMemory+freeMemory).
>> If also doesn't feel right because those allocations are supposed to 
>> be individual memory allocations, not a whole arena/pool allocation.
>>
>> Therefore, what can be used to be able to free individual 
>> MemorySegment efficiently in this context?
>>
>> Maybe the Automatic arena should have a way to free explicitly a 
>> MemorySegment?
>> I think it would make sense, as while auto-release semantics are 
>> convenient, it's always good practice to release such native 
>> resources eagerly rather than wait for GC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20250217/dc35fd74/attachment-0001.htm>