scopes writeup

[Maurizio Cimadamore]

On 23/01/2019 10:56, Jorn Vernee wrote:
> Hi Maurizio,
>
> Per the proposal, a Pointer<Integer> from a parent scope can only be 
> assigned to a Pointer<Pointer<Integer>> from a child scope (but not 
> from an ancestor scope). How does this work for native function calls 
> in general?
Accommodating native function calls is the primary reason for having an 
unchecked scope which is used internally by the binder. When calling a 
function it is typical to copy all incoming arguments into a temporary 
area - and if you do that, with scope ownership you quickly realize that 
there's no safe way to do that (you'd have to find a scope which is the 
'child' of all scopes associated to all the arguments). But this is, I 
believe, a very special case, and also an internal one, so using some 
kind of 'trust-me' scope works well in this case.
>
> - What, if any, are the restrictions on the Scopes of Pointers passed 
> to native library functions?
I think the proposed patch doesn't add any new restriction on this side 
(because of unchecked scopes).
> - You've said before that a Pointer returned from a library call will 
> get the library's scope, is that sufficient for the case where a 
> library returns a Pointer that we passed into it from Java? Does this 
> imply you can only pass pointers to native calls if they have been 
> allocated from the library's scope (or an ancestor)?

I think this is a valid concern, but ultimately, I think it's pointless 
to try and enforce structure on pointers that are coming out of a native 
call. There's no way the Java code can know what's coming out of the 
library and what its life-cycle would be like.

In other words, the safety goals for the Scope API, is that if you 
allocate memory, play with pointers etc., read/write you can never shoot 
yourself in the foot - that is, the API should be safe - and should 
prevent common mistakes, such as leaving dandling pointers and such.

Calling a native method is a completely different problem - and we can't 
expect there to have (much) more guarantees than those we get with JNI. 
That is, calling a native method has an inherent safety risk for the 
code involved in your application - which means you have to know what 
you are doing, and what the library is doing.

Extreme example: the library could return a pointer that is already 
dandling (e.g. the library allocate memory with malloc, saves the 
pointer somewhere, frees the memory, and then returns the pointer). 
There's no possible scope abstraction that can save you from that.

So, when it comes to native calls, I lean towards flexibility - you 
should be able to pass any pointer to the native library, and you should 
make minimal assumption about the lifecycle of the pointers that come 
out of that library (the most basic assumption is that these pointers 
will be alive for the duration of the library).

Maurizio

>
> Jorn
>
> Maurizio Cimadamore schreef op 2019-01-22 17:27:
>> Hi,
>> at the end of last year I tried coming up with a proposal [1] which
>> aimed at finding the basic concepts behind Panama scopes - I've been
>> iterating (with the help of some internal feedback) on that proposal,
>> and put it back together in a different form which, all things
>> considered, it's not too far off from where we are now.
>>
>> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html
>>
>> That is, we still have a Scope abstraction - memory regions are still
>> hidden under Scopes, and not part of the public API (in an attempt to
>> limit the number of concepts exposed by the API - at least for now).
>> Pointer is an immutable cursor into a memory region. Resources are
>> just things with a scope (turns out, a native library is another such
>> thing, since a native library needs a library-wide scope for
>> allocating its own globals, etc.).
>>
>> On top of what we have now, this writeup pushes forward an improved
>> ownership model, as well as some discussions concerning
>> thread-confinement of scopes.
>>
>> I think this tweaked API proposal has the potential of fixing the
>> current lifecycle issues, w/o making the API too complex to use.
>> Additional abstraction (such as MemoryRegion) can be added back to the
>> public API on a by need basis (e.g. provided relevant use cases).
>>
>> An experimental patch implementing the proposed approach can be found 
>> here:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/
>>
>> Comments welcome!
>>
>> Maurizio
>>
>> [1] -
>> https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html 
>>