[foreign] Pulling the string on Scope

[Maurizio Cimadamore]

On 19/12/2018 15:08, Jorn Vernee wrote:
> Hi,
>
> I have been thinking for a while that there could be some improvements 
> made to the Scope API. (And I'm currently in the process of debugging 
> what seems to be a life-cycle problem)
>
> * Scopes could optionally have names. If an exception is thrown 
> because a Resource's scope is closed, and don't know where this 
> scope/resource is coming from, you have to debug that. By allowing 
> scopes to have optional names, we can add additional debugging 
> information to a scope which can be displayed in the exception 
> message; "Scope is not alive" vs. "Scope 'MyScope' is not alive".
Not feeling too strongly on this...

>
> * There should imho be a way to transfer resources between scopes. I 
> suggest adding a `void transferTo(Scope scope)` method to Resource for 
> this. Transferring resources between scopes could be useful for 
> instance in the case where a Resource is returned from a native 
> function and it's up to the caller to manage that resource. Currently 
> the resource is leaked, which makes sense when the return value is not 
> supposed to be managed by the caller, but for the former case it's 
> nice to be able to transfer the resource into some other, non-leaked, 
> scope. (From what I can tell the allocation strategy of NativeScope 
> would have to be changed to allow for this).

Transferring resources from one scope to another is something we are 
actively looking into. In a way, we can model 'freeing' a resource as 
moving it into the 'NullScope' (or BitBucketScope :-)).

Now, as I started to exploring that model more seriously, I found a 
pretty serious blocker: the current Pointer/Scope API is designed around 
two principles:

1) Pointers are immutable, and value-ready (hint hint Valhalla :-))
2) Obtaining the scope of a given Pointer is an O(1) operation

(2) is important: since liveness is a property of Scope and since we can 
go in O(1) from Pointer to its owning Scope, we can, in O(1) establish 
if a pointer is alive!

If we start supporting moving resources from one Scope to another, I'm 
having hard time in figuring out how that can be achieved in an 
efficient manner. If Pointer is immutable (as per (1)) that means we 
can't simply change its owning scope - if we did that, it would be 
impossible to migrate Pointer to values once Valhalla is ready.

That leaves us few options, one of which is that Pointer do not store 
owning scope in a field, but instead Scopes keep them in tables. But 
this gives up (2) - now determining the scope owning a resource is no 
longer O(1), as there's at least a table lookup.

I still have not found a satisfactory way to resolve this tension.


>
> * boxing/unboxing code should take a Scope as an argument. When 
> returning values they can be put into this scope, or in the case of 
> Windows a copy of a struct might be needed for the duration of the 
> call. For upcalls the boxed arguments can also be added to this scope. 
> Each call will have 2 scopes, one for the duration of the call and one 
> for return values, which is 'leaked', and it's up to the caller to 
> manage this scope and it's resources. Which of the two is passed to 
> box/unbox depends on if we're doing an upcall or downcall. So in 
> abstract you'd have this;
>
> ```
> try(var callScope = Scope.newNativeScope()) {
>     var returnScope = Scope.newNativeScope(); // leaked to caller
>
>     unbox(callScope, ...);
>
>     // do call
>
>     box(returnScope, ...);
> }
> ```
>
> Where for upcalls box and unbox are switched.

This is not related to the Scope API, but I agree that the code should 
be more symmetric between upcalls and downcalls - while right now we 
create the scope in the box routine only; I agree with your proposal to 
add a scope parameter, and let these methods be fully symmetric.


Maurizio

>
> What do you think about about Scope?
>
> Thanks,
> Jorn