[foreign-memaccess+abi] RFR: 8263018: Improve API for lifecycle of native resources [v10]

[Maurizio Cimadamore]

On Mon, 15 Mar 2021 17:34:10 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>>> A possible, improved naming for the two segment allocators:
>>> 
>>> SegmentAllocator.of(MemorySegment) -> SegmentAllocator.slicing(MemorySegment)
>>> SegmentAllocator.of(ResourceScope) -> SegmentAllocator.scoped(ResourceScope)
>>> 
>>> What do you think?
>> 
>> Yes, that's better. I find it hard to come up with a better concise name for the former, allocation returns a sliced prefix of the given segment, thus allocations are aliased, which is fundamentally different to all the other cases.
>
>> > A possible, improved naming for the two segment allocators:
>> > SegmentAllocator.of(MemorySegment) -> SegmentAllocator.slicing(MemorySegment)
>> > SegmentAllocator.of(ResourceScope) -> SegmentAllocator.scoped(ResourceScope)
>> > What do you think?
>> 
>> Yes, that's better. I find it hard to come up with a better concise name for the former, allocation returns a sliced prefix of the given segment, thus allocations are aliased, which is fundamentally different to all the other cases.
> 
> One option which I still find kind of appealing, is to add _instance_ methods to connect the various APIs. E.g. instead:
> 
>  SegmentAllocator.of(scope) -> scope.toAllocator();
>  SegmentAllocator.of(segment) -> segment.toSegment();
>  
>  I think these conversions are great in certain contexts: think of a client slicing a segment and wanting to put the result of a native call in a certain place:
> 
> MemorySegment segment = ...
> MemorySegment out = segment.asSlice(...);
> linkerHandle.invokeExact(out.toAllocator(), ...);
> 
> Similarly, if you have an resource scope and simply want an allocator that returns segments attached to that scope:
> 
> try (ResourceScope scope = ...) {
>    linkerHandle.invokeExact(scope.toAllocator(), ...);
> }
> 
> I find both more concise that their static variants, but also (crucially) easier to explain. What we lose if we go down that path is "independence" between the various abstractions - in the sense that a ResourceScope is no longer Panama-neutral.

I've been doing some validation, e.g. port jextract to the new API:

https://github.com/openjdk/panama-foreign/compare/foreign-jextract...mcimadamore:jextract+resourceScope?expand=1

I think overall the results is neat. I liked how the new API doesn't _force_ me to have a scope for everything (there are many cases where we need to create a segment on the fly, as a step to be able to call a function once) - simple cases can just rely on cleaner and the GC, thus simplifying the code. At the same time, complex cases where lifecycle of multiple resources is linked, can be expressed way more clearly, since the user is now in control of the previously hidden "scope" component.

This patch adds overloads to native function wrappers returning structs, so that users can pass a NativeScope if they wish allocation to occur in that scope.

To minimize disruption, jextract is generating NativeScope in the target package; I think _some_ abstraction linking together ResourceScope and SegmentAllocator would be nice to have for the native use case, but we're unsure as to what the best way to expose this is.

One option would be to have a simple subtype of SegmentAllocator called ScopedSegmentAllocator - which expresses the notion of "I'm allocator that has its own scope". I think this will also be useful for integration with Jim's QBA allocator.

Another option would be to associate an _optional_ scope to every SegmentAllocator. That is, add a default method to SegmentAllocator:

default Optional<ResourceScope> allocatorScope() { returns Optional.empty(); }

Honestly, now that I look back at this, it doesn't seem as bad as I originally thought - most of the current allocators actually already have a natural scope - the only exception being malloc() - which doesn't have a scope, since all the returned segments are independent. Thoughts?

-------------

PR: https://git.openjdk.java.net/panama-foreign/pull/466