dissecting memory sessions

[Maurizio Cimadamore]

Hi Sebastian,
that's a good question. I think something like that might be done by 
overloading the `openShared` factory and add an explicit owner thread, 
as in:

`Arena.openShared(Thread.currentThread());`

That said, while this can be easily added, it also seems a bit of a 
corner case (even though I realize you came at it honestly), and there 
might be other ways to get there - for instance, your application could 
be in control of the Arena, and not leak it to those threads that should 
not be able to perform `close`. Instead, those thread could simply have 
access to the Arena session, so that they can easily co-allocate 
segments into the application's arena, but will have no real capability 
to bring things down. In other words: do these threads really need to 
access the arena?

Maurizio

On 03/11/2022 09:24, Sebastian Stenzel wrote:
> Hi Maurizio,
>
> thanks for sharing the article. I often struggle putting ideas into easily digestible written form and really appreciate this effort.
>
> I wonder if there is room for something in between of confined and shared arenas. Arenas that are accessible by any other (non-structured) thread but still have an owner (capable of explicitly closing it).
>
> As you know I have a use case, where a Java (owner) thread creates a long-living arena and allocates some data which needs to be accessible by other threads that get created outside of Java. As soon as these threads get attached to Java, I don't want them to be able to close the arena. This can of course be achieved by just not calling `close()`, but since we're talking about restricting capabilities, I wonder if there is a stricter way.
>
> Thanks!
> Sebastian
>
>> On 2. Nov 2022, at 22:45, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>
>> In case you are interested, below are some more pointers.
>>
>> First, the javadoc which contains the proposed FFM API changes:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/session_handle_hidden_javadoc/javadoc/java.base/java/lang/foreign/package-summary.html
>>
>> For those of you who are more interested at the code, here is the branch that contains the code changes:
>>
>> https://urldefense.com/v3/__https://github.com/mcimadamore/panama-foreign/tree/session_drop_min__;!!ACWV5N9M2RV99hQ!My6SZykftji37fAHPb4wtUyIW-o9m0qrzl2JIZmvlOk2s_k1inawOWpQHGD4_oaTf4EomLNELW3R5S0G1DH37PIzc1jibv2hAQ$
>>
>> And, finally, here is also a jextract branch that supports the proposed FFM API changes:
>>
>> https://urldefense.com/v3/__https://github.com/mcimadamore/jextract/tree/panama_session_refresh__;!!ACWV5N9M2RV99hQ!My6SZykftji37fAHPb4wtUyIW-o9m0qrzl2JIZmvlOk2s_k1inawOWpQHGD4_oaTf4EomLNELW3R5S0G1DH37PIzc1jmQwBFkA$
>>
>> Cheers
>> Maurizio
>>
>> On 02/11/2022 17:48, Maurizio Cimadamore wrote:
>>> Hi,
>>> After the first preview of the FFM API in Java 19, we have identified a couple of areas where the API could use some improvements:
>>>
>>> * Clarify the relationship between MemorySegment and MemoryAddress (this was addressed in [1]); and
>>> * Polish the MemorySession API, and make segments easier to (safely) share with external clients (what this email is about).
>>>
>>> While we have explored solutions to better encapsulate memory sessions in the past (e.g. by dropping session accessors, as described in [2]), nothing seemed to stick. So, for Java 19 we decided to leave session accessors on memory segments in place, but give the option to libraries to protect against "sneaky" close, by creating non-closeable memory session views.
>>>
>>> After staring at this problem long enough, it became increasingly clear that memory sessions, in their current shape and form, are trying to do too much - from allocation, to lifecycle management and more. The issue with "sneaky" close is mostly a manifestation of that more fundamental problem. In that spirit, we have put together a document which teases apart the various "traits" associated with memory sessions, and repackages the same traits into an API that provides better encapsulation and composition. The document can be found here:
>>>
>>> http://cr.openjdk.java.net/~mcimadamore/panama/session_arenas.html
>>>
>>> The main move described in the document is to make MemorySession a "pure" lifetime abstraction, thus dropping SegmentAllocator and AutoCloseable capabilities. Instead, these capabilities are provided by a *second* abstraction, called Arena. Crucially, an Arena _has_ a memory session, which can e.g. be used to allocate segments that have the same lifecycle as that of the arena. This subtle twist, gives us an API that is easier to reason about (and to build upon), and one where memory segments can be shared freely across clients - premature calls to MemorySession::close are no longer possible. At the same time, the API now makes a much clearer distinction between sessions that are closeable (i.e. sessions created through an Arena) and those that aren't (i.e. implicit and global sessions).
>>>
>>> Here's a list of the main API changes, and how they will impact clients of the FFM API:
>>>
>>> * MemorySession no longer has a close() method; try-with-resources against MemorySession will now need to use Arena instead;
>>> * Support for non-closeable session views (MemorySession::asNonCloseable), and related methods (MemorySession::equals/hashCode) has been removed;
>>> * MemorySession::addCloseAction has been removed; instead, clients can specify a cleanup action when creating an unsafe segment (i.e. using MemorySegment::ofAddress);
>>> * Some of the predicates in MemorySession have been made more robust - e.g. instead of MemorySession::ownerThread(), there is now a predicate MemorySession::isOwnedBy(Thread).
>>>
>>> After careful consideration, we believe that the changes described in this document are worth pursuing for the upcoming Java 20 integration [3]: they make the API more principled (no more "sneaky" close), while retaining a similar expressive power.
>>>
>>> Any feedback is greatly appreciated.
>>>
>>> Cheers
>>> Maurizio
>>>
>>> [1] - https://cr.openjdk.java.net/~mcimadamore/panama/segment_address.html
>>> [2] - https://mail.openjdk.org/pipermail/panama-dev/2022-February/016152.html
>>> [3] - https://mail.openjdk.org/pipermail/panama-dev/2022-February/016152.html
>>>
>>>