[foreign-memaccess] wrapping up JVMLS 2019

[Jorn Vernee]

Some quick thoughts on this:

> - what to do about 'forgotten' segments - e.g. segments that go out of
> GC scope, but that have not been closed

I think tracking MemorySegment instances with a Cleaner as part of a 
MemoryPool implementation is a good idea. It seems there are many 
different schemes for memory pools possible, for instance, besides 
waiting for every subsegment to be closed before being able to close the 
MemoryPool, there could also be an implementation that keeps track of 
subsegments and forcibly closes them when the pool is closed.

We could also add a utility that tracks memory segment instances when 
they are created, and reports any that have been forgotten when the 
application exits.

Jorn

On 2019-08-03 01:54, Maurizio Cimadamore wrote:
> Hi,
> I'm just heading back home after a great week at the JVMLS 2019, where
> I gave a talk about the foreign memory access API [1]. The talk was
> very well received and we had a follow-up breakout session in the
> context of the OpenJDK Committer Workshop, which was co-located with
> JVMLS.
> 
> I'd like to summarize the feedback I've received on the API (of course
> this is "my side of the elephant", of course there can be other sides
> there :-)):
> 
> * The MemorySegment API in itself feels 'right'. It has the right
> level of abstraction, and it allows enough flexibility to peek/poke at
> memory using var handles, which will make it easier to implement
> things like off-heap tensors and Python-like ndarray in Java
> 
> * There seem to be an agreement that there are at least two category
> of users of this API: casual users who might just want to allocate a
> segment in a single shot, serialize some objects off-heap. For these
> users the single-threaded limitations of the API, as well as the
> requirement to always 'close' the segment via try-with-resources is
> probably fine.
> 
> * There is also another category of users - high-performance
> frameworks which, of course care about a bunch of other things:
> 
> - allocation performances: don't want to do one malloc per allocation 
> request
> - what to do about 'forgotten' segments - e.g. segments that go out of
> GC scope, but that have not been closed
> - need to handle things like concurrency, e.g. accessing same segment
> from multiple threads
> 
> As we were chatting about the various alternatives, and some of the
> things described in the concurrency writeup I published few weeks ago
> [2], it seems to me that there's a logical split to be made here,
> where we keep the current API for the simple use cases (after all the
> API is fine for those) - and then we could build an higher-level
> allocator/scope/whatever which will handle the remaining concerns.
> That is, if you are an advanced user you probably want to create your
> segments indirectly, through an allocator:
> 
> try (MemoryPool pool : MemoryPool.ofNative()) { // big slab of memory
> allocated here
> 
>      MemorySegment segment = pool.segment(layout);
>      ... //segment can be shared
> 
> } // pool memory de-allocated here
> 
> 
> This moves allows us to put all the complexity in the allocator -
> which will be responsible for handing over chunks of a private segment
> to clients - clients will be able to either close pooled segments
> (thus returning them into the pool) and, the pool can also use a
> Cleaner to make sure that GC-unreacheable segments will have a chance
> to be forcibly closed. At the same time, since the underlying native
> memory will be guaranteed to be 'available' for the entire life-cycle
> of the pool, that means that all segments created within a pool can
> safely be accessed by multiple threads concurrently. Under the
> curtains, we can keep a counter in the pool of how many segments are
> 'pending' and only allow a pool::close if that count is zero (this is
> similar to some of the conclusions reached in [2]).
> 
> I think I like this a lot - not only it builds nicely _on top_ of the
> memory segment API, but it also addresses problems that users of the
> memory segment API are likely to run into and try to implement ad-hoc
> solutions for anyway.
> 
> I'll keep thinking about it, but so far it looks like a promising 
> direction.
> 
> Cheers
> Maurizio
> 
> [1] - https://www.youtube.com/watch?v=r4dNRVWYaZI
> [2] - http://cr.openjdk.java.net/~mcimadamore/panama/confinement.html