Doubling down on arenas in the FFM API

Tue Feb 7 10:59:44 UTC 2023

Hi Chris,
thanks for the comments.

First of all, I agree the javadoc needs more verbiage around what can 
happen when `close` is not supported.

Regarding your question, there are many reasons which has led us towards 
leaving `isCloseable` out - at least for now.

First, the semantics of `isCloseable` is a bit ambiguous: should it mean 
"can I call close on this arena?", or should it mean "is this arena 
associated with a bounded lifetime" ? The two questions aren't the same: 
in the first case you are asking about a property of a specific 
`Arena::close` method (will it throw UnsupportedOperationException?); in 
the other case you are asking about an intrinsic property of the 
lifetime backing the arena. For instance, an automatic arena has a 
bounded lifetime, but its `close` method still throws. Perhaps, the 
solution here would be to have a method which deal with `close` in 
Arena, and then another method in MemorySegment.Scope which tells 
whether a scope is "bounded" or not.

Secondly, as explained in the document, custom arenas muddy the picture 
quite a bit, as it is not clear as to whether `isCloseable` should just 
always delegate to the parent arena `isCloseable`, or if a subclass can 
redefine it in interesting ways - e.g. a client might wrap a closeable 
arena into a new arena whose close method always throws - at which point 
the `isCloseable` method could be overridden to return false. But IMHO, 
it is hard to reason about custom arenas because it is not clear what we 
mean by `isCloseable` (see above).

Finally, all our usages analysis have shown that, so far, of all the 
code using the Java 19 API (which has a MemorySession::isCloseable 
predicate) we didn't find any significant usages (only one usage was 
found in an assert in the Netty code :-)). This seems to suggest that 
these predicates are not only hard to nail down semantics-wise, they 
also don't seem to be used much. I think I can understand why: typically 
code that has a MemorySession/Arena has created it, so it _knows_ what 
are the properties of that session/arena.

For this reasons, I find your example a bit on the contrived side: two 
clients exchange segments/arenas - which means they are under the same 
maintenance domain (otherwise client B would not have access to arenas). 
The way I'd approach the situation you describe would be to put 
resources in distinct arenas. Clearly you have identified cases as to 
why some resources are tied with the lifetime of a single request (e.g. 
local resources) whereas other resources can be reused across multiple 
requests (e.g. global or cached resources). To me, this suggests that 
there should be an Arena for what B wants to do. The code in B is always 
surrounded with a try-with-resources (at least logically - in practice 
the lifetime can start in A and end with B), so that all the resources 
that are "local" to B get released when B is done. But "global" 
resources are simply passed in (e.g. using some context object) - and B 
is not expected to invalidate them (nor it can't because they are backed 
by an arena it can't control).

In other words, my feeling here is that the need for `isCloseable` 
arises because A and B simply exchange segments/arenas in a fairly 
unstructured way, but - as you say - not all segments in this 
communication are created equal. Which suggests that the communication 
protocol between A and B should be made a bit more sophisticated, so 
that B can distinguish between local and global resources.

But, even w/o making things overly complex, something like this should work:

```
B::process(Arena sessionArena, MemorySegment[] segments)
```

Here, B has a bunch of segments to work with. Some will be backed by 
`sessionArena` (which B is expected to close anyway at the end of its 
computation). Some will be backed by some other arena - which means they 
will survive the `process` method. This way, B always knows how to 
operate: it should work on the received segments and then call 
`sessionArena.close()` when done. Note that A could even inject custom 
close logic on `sessionArena`, by providing an arena with an overridden 
close method. This might be useful if the segments that should be closed 
in reality might belong to multiple arenas, all of which should be 
closed (or when custom cleanup logic should be executed by A). But I 
think an API like this makes it easier to understand how B is expected 
to operate.

That said, I don't want to dismiss your concern - but I do believe that 
a "wait and see" approach is a sane way to proceed: future code will 
inform us as to whether such predicates are needed or not (and _where_ 
should such predicates live - e.g. in Arena vs. MemorySegment.Scope).

Cheers
Maurizio

On 05/02/2023 19:56, Chris Vest wrote:
> Hi,
>
> This looks like a good and simple API overall, with a clear path on 
> how each individual use case is solved.
>
> I found that details of the behavior of Arena.close() were lacking in 
> the API description, now that we are back to having close methods on 
> things that can't be closed:
> It is not described what happens when you call close() on global and 
> auto scoped arenas. I also see no mechanism for querying if an arena 
> has one of these scopes.
> The document says it's unclear what isCloseable() should return for an 
> arena where close() always throws. I'm not so sure.
> I think the query could be framed as "if I were to call close(), is it 
> likely to succeed?" (I use "likely" here since shared scopes could race).
>
> To illustrate where this might be relevant, suppose we have web 
> application with a component A in charge of formulating responses to 
> clients, and another component B in charge of doing IO.
> Component A sends MemorySegments and Arenas to B, and these could be a 
> mix of shared or confined scopes, and auto-scoped for cached data like 
> images and CSS files.
> Lifetimes start in A, but only conditionally end in B. This will need 
> to be tracked, since the arenas and scopes can't be queried for this 
> information.
> While this is certainly possible, it seems like a wasteful step, as 
> the arena ought to have this knowledge in order to implement close() 
> itself.
>
> This is the only thing I'm wondering about. The rest of the API looks 
> very good.
>
> Cheers,
> Chris
>
> On Tue, Jan 31, 2023 at 10:46 AM Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>     Hi,
>     as discussed here [1], it is not clear as to whether Java 20
>     iteration
>     of the Foreign Function & Memory API (FFM API) has yet reached
>     bottom,
>     especially when it comes to managing the lifetime of the regions of
>     memory backing memory segments. After collecting some rounds of
>     internal
>     and external feedback, it was clear that while the Java 20 API has
>     all
>     the functionalities we require for writing efficient and robust
>     native
>     interop code, some of the concepts in the API were made a bit
>     harder to
>     grok, as users had to choose between two toplevel abstractions,
>     namely
>     `SegmentScope` and `Arena`. This choice is made even more
>     difficult, as
>     some of the functionalities (e.g. allocation) is duplicated in
>     both API
>     points. As a result, we have been busy exploring different ways to
>     restack the FFM API in search of something more approachable.
>
>     The results of our findings are described in this document:
>
>     http://cr.openjdk.java.net/~mcimadamore/panama/scoped_arenas.html
>
>     Here, we propose a possible simplification of the FFM API, where
>     we make
>     `Arena` the true star of the show, which results in the following
>     changes:
>
>     * factories such as `SegmentScope::auto` are now moved to `Arena`;
>     * all segment-producing methods (such as `FileChannel::map`) now
>     accept
>     an `Arena` parameter;
>     * static factories such as `MemorySegment::allocateNative` have been
>     dropped;
>     * scopes are made less prominent, and moved to a nested class
>     (`MemorySegment.Scope`).
>
>     This gives us a remarkably simple API, which brings together the best
>     aspects of the Java 19 and Java 20 FFM API iterations. On the one
>     hand,
>     `Arena` is now the most important abstraction that users of the
>     FFM API
>     have to deal with (in a way, `Arena` is the new `MemorySession`);
>     at the
>     same time, we still have a way to model the lifetime of an `Arena`
>     (and
>     all the segments allocated by it) using a `MemorySegment.Scope` -
>     which
>     is desirable both in terms of debugging (e.g. inspecting whether two
>     segments/arenas have the same lifetime) and, more importantly, in
>     terms
>     of allowing the definition of custom arenas via simple delegation
>     (as in
>     Java 20).
>
>     As always, feedback is welcome. While this proposal does not
>     significantly alter the expressiveness of the FFM API, the
>     proposed API
>     comes with some limitations. For instance, since all allocation
>     routines
>     are now `Arena`-centric (see above), it is no longer possible to
>     allocate a new segment if a corresponding arena is not available (we
>     call this co-allocation). As explained in the document, while it
>     would
>     be possible to add back the missing co-allocation functionality,
>     extensive analysis of the code using the FFM API has shown
>     co-allocation
>     to be _extremely_ rare (**) - and of dubious value. For these
>     reasons,
>     we would like to aim for a more principled approach which avoids
>     co-allocation altogether, and allows for more encapsulation of the
>     capabilities associated with an `Arena` object.
>
>     Maurizio
>
>     (**) We have only found _one_ usage [2] in over 10K Java files and
>     more
>     than 11M LoC analyzed. Moreover, this usage is only present in the
>     Java
>     19 branch of the project, and removed in the "main" branch (which
>     tracks
>     the Java 20 FFM API). We suspect that this use of co-allocation
>     has been
>     made irrelevant after the unification of `MemoryAddress` and
>     `MemorySegment`.
>
>     [1] -
>     https://mail.openjdk.org/pipermail/panama-dev/2022-December/018182.html
>     [2] -
>     https://github.com/boulder-on/JPassport/blob/Java_19/jpassport/src/main/java/jpassport/Utils.java#L418
>     <https://urldefense.com/v3/__https://github.com/boulder-on/JPassport/blob/Java_19/jpassport/src/main/java/jpassport/Utils.java*L418__;Iw!!ACWV5N9M2RV99hQ!MB7MDEPjnE7sfsm2kjOWvsGp1DN6u1XAGyMAzkjSdZw7-11yWTo39B6YRtcK8gPYPZ-5ZlaeI3KxuCT3F5flAIOvAA9yWQ$>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230207/5d7b1545/attachment-0001.htm>