changes to the foreign preview API

[Maurizio Cimadamore]

Hi,
the example you mention is a trade off we have considered.

The problem with the status quo, where scope is freely accessible in all 
resources, is that there is no way to control that a scope will not be 
closed by _clients_ of a library. For example you might want to allocate 
a big segment, and give chunks of that segment to clients, in response 
to certain requests. If a client can effectively bring the entire world 
down by doing `segment.scope().close()`, that doesn't seem like a great 
place to be either; that scope manages the lifecycle of the library, not 
that of the client. There are other things a client can do that are 
questionable, such as registering a never-terminating close action on a 
scope that is owned by some other actor, which will effectively render 
the scope uncloseable, and result in a memory leak.

So, the thinking here is that there are clients that _produce_ resources 
(those clients will have a scope) and clients that _consume_ resources 
(those clients will only have e.g. a segment). The former are of course 
unrestricted in what they can do - but in a way, if they mess up their 
own scope, it's their business. The latter, being only consumers, cannot 
really alter the lifecycle of the resource they are dealing with in any 
way.

Your example is tricky because it does both: it is a client of a memory 
segment (it need to read bytes from the segment), but it is also a 
producer of segments (it wraps a raw pointer into a _new_ segment). The 
latter part is worth stressing: I think you are making the `getA()` 
accessor simpler than it actually is: that accessor is reading a pointer 
from a segment; that pointer will point to _another_ memory region 
(which could be completely unrelated from that of the original segment), 
and returns a _new_ segment, backed by this new region.

Now, in the general case, there is no guarantee that the two regions 
will share the same lifecycle (although that can sometimes be the case). 
In which case, the options you listed seem morally correct: either you 
tweak the `getA()` accessor to also accept a scope (that of the returned 
segment), or you upgrade the whole class to be not just a _consumer_, 
but also a _producer_ of segments. This means that the struct will need 
its own scope, either passed from outside, or created within.

For the records, I think passing a scope to the accessor might not be as 
bad; first, the application allocating these structs classes probably 
have a scope handy to pass. Secondly, accepting a scope also opens up 
the possibility of controlling the lifecycle of the returned struct some 
more - e.g.

```
B b = ...
try (var scope = ResourceScope.newConfinedScope()) {
    A a = b.getA(scope);
    ...
}
// a's segment cannot be dereferenced here
```

Summing up, I think the change can be a little inconvenient, but only in 
cases where the API wants to assert that the scope of the returned 
segment is _really_ the same as the current one (we have this problem 
with `VaList::copy`, for instance). But in the general case of a struct 
that has a pointer to some other struct, if an API wants to create a 
segment out of a raw pointer, there is a choice to make when it comes to 
the lifecycle of the returned segment. Ideally, that choice has to be 
reflected by the API in some way. The fact that the proposed changes 
makes that choice bubble up some more is, IMHO, a good thing.

Maurizio


On 02/02/2022 01:30, Michael Zucchi wrote:
> On 1/2/22 23:18, Maurizio Cimadamore wrote:
>> The other change we'd like to make is to remove the scope() accessors 
>> from all the resources (MemorySegment, NativeSymbol, VaList). This 
>> comes from a desire to make the API more principled: only the owner 
>> of a scope/session should have the "right" to close it. In fact, we 
>> have spent significant API estate (e.g. ResourceScope::whileAlive) 
>> just from preventing random clients to shut down scopes. We now 
>> believe a better approach would be to simply make the scope 
>> associated e.g. with a memory segment inaccessible. In other words, a 
>> resource scope/memory session becomes a _capability_: it is up to the 
>> owner of the scope/session to decide who to share that capability 
>> with. Once a client has a scope, it can close that scope, but also 
>> affect the scope by registering cleanup actions, or allocating on 
>> that scope. Conversely, if a client only has a segment, there's no 
>> way for that client to affect the owning scope/session in a meaninful 
>> (and possibly, harmful) way
>
> There's some cases where this will be a bit inconvenient.
>
> A simple example would be something like:
>
> struct a {
>  int c;
> };
>
> struct b {
>   struct a *a;
> };
>
> Where 'a'  is created/managed by 'b'.  The java side might be:
>
>  class b {
>   MemorySegment segment;
>
>   b(MemorySegment seg) {
>     segment=seg;
>   }
>    a getA() {
>       MemoryAddress addr = varhandle$a.get(segment);
>       return new a(MemorySegment.ofAddress(addr, a.layout.byteSize(), 
> segment.scope());
>    }
> }
>
> You don't really want to have to track and pass the scope to the 
> accessor each time.  The alternative is to add a scope field to 'b', 
> but then it also needs to be passed to every constructor of such a class.
>
>  Michael
>