Doubling down on arenas in the FFM API

[Martin Pernollet]

Hi Maurizio,

Thanks for sharing information about the future of Panama!

I have no other comment on your roadmap notice than confirming that centering FFM API on Arena would make the API simpler and nice (although the API is really clear already).

Do you think the API will still evolve through following releases?

Cheers,

Martin 

ps : Thank you for sharing information on the PanamaGL project in your december email! I just wish to warn that we moved to Gitlab (https://gitlab.com/jzy3d/panama-gl).




Envoyé avec la messagerie sécurisée Proton Mail.

------- Original Message -------
Le mardi 31 janvier 2023 à 19:46, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> a écrit :


> Hi,
> as discussed here [1], it is not clear as to whether Java 20 iteration
> of the Foreign Function & Memory API (FFM API) has yet reached bottom,
> especially when it comes to managing the lifetime of the regions of
> memory backing memory segments. After collecting some rounds of internal
> and external feedback, it was clear that while the Java 20 API has all
> the functionalities we require for writing efficient and robust native
> interop code, some of the concepts in the API were made a bit harder to
> grok, as users had to choose between two toplevel abstractions, namely
> `SegmentScope` and `Arena`. This choice is made even more difficult, as
> some of the functionalities (e.g. allocation) is duplicated in both API
> points. As a result, we have been busy exploring different ways to
> restack the FFM API in search of something more approachable.
> 
> The results of our findings are described in this document:
> 
> http://cr.openjdk.java.net/~mcimadamore/panama/scoped_arenas.html
> 
> Here, we propose a possible simplification of the FFM API, where we make
> `Arena` the true star of the show, which results in the following changes:
> 
> * factories such as `SegmentScope::auto` are now moved to `Arena`;
> * all segment-producing methods (such as `FileChannel::map`) now accept
> an `Arena` parameter;
> * static factories such as `MemorySegment::allocateNative` have been
> dropped;
> * scopes are made less prominent, and moved to a nested class
> (`MemorySegment.Scope`).
> 
> This gives us a remarkably simple API, which brings together the best
> aspects of the Java 19 and Java 20 FFM API iterations. On the one hand,
> `Arena` is now the most important abstraction that users of the FFM API
> have to deal with (in a way, `Arena` is the new `MemorySession`); at the
> same time, we still have a way to model the lifetime of an `Arena` (and
> all the segments allocated by it) using a `MemorySegment.Scope` - which
> is desirable both in terms of debugging (e.g. inspecting whether two
> segments/arenas have the same lifetime) and, more importantly, in terms
> of allowing the definition of custom arenas via simple delegation (as in
> Java 20).
> 
> As always, feedback is welcome. While this proposal does not
> significantly alter the expressiveness of the FFM API, the proposed API
> comes with some limitations. For instance, since all allocation routines
> are now `Arena`-centric (see above), it is no longer possible to
> allocate a new segment if a corresponding arena is not available (we
> call this co-allocation). As explained in the document, while it would
> be possible to add back the missing co-allocation functionality,
> extensive analysis of the code using the FFM API has shown co-allocation
> to be extremely rare () - and of dubious value. For these reasons,
> we would like to aim for a more principled approach which avoids
> co-allocation altogether, and allows for more encapsulation of the
> capabilities associated with an `Arena` object.
> 
> Maurizio
> 
> () We have only found one usage [2] in over 10K Java files and more
> than 11M LoC analyzed. Moreover, this usage is only present in the Java
> 19 branch of the project, and removed in the "main" branch (which tracks
> the Java 20 FFM API). We suspect that this use of co-allocation has been
> made irrelevant after the unification of `MemoryAddress` and
> `MemorySegment`.
> 
> [1] -
> https://mail.openjdk.org/pipermail/panama-dev/2022-December/018182.html
> [2] -
> https://github.com/boulder-on/JPassport/blob/Java_19/jpassport/src/main/java/jpassport/Utils.java#L418
> 
> 
> 
>