resource scopes and close actions

[Maurizio Cimadamore]

>>> - (based on the proposed api change) also keep a copy of the 
>>> ResourceScope if it wants to be able to propagate it automatically 
>>> which would often be desirable (... i think).
>> stay tuned on this. We're looking into ways to mitigate the impact of 
>> the proposed changes which will not require you to keep a 
>> ResourceScope around (and use a segment instead as a "proxy" of the 
>> temporal bounds you want to use e.g. to create another segment).
>
> What if you have a NativeSymbol instead?
The mechanism we had in mind would work with all "scoped" resources - 
segments, valist, native symbols.
>
>>>
>>> Doesn't seem too elegant but that should work?  I'll explore that 
>>> with opencl as i've the most experience with that api and it has 
>>> some consistent conventions that i think will mesh well.
>>>
>>> e.g. as a first thought, does this look about right?
>>>
>>> An example case is CLCommandQueue which is an object which is 
>>> created by and belongs to a CLContext and can be closed either by an 
>>> explicit unref or when the context is unreffed.
>>>
>>> public CLCommandQueue {
>>>     NativeSymbol symbol;
>>>     ResourceScope scope;
>>>     MemoryAddress addr;
>>>
>>>    // package private
>>>     close() {
>>> clReleaseCommandQueue$MH.invokeExact((Addressable)addr);
>>>     }
>>> }
>> This looks mostly ok - not sure whether you need to keep the `addr` 
>> around. I would have expected `addr` to only really be used in a 
>> close action, so when you create CLCommandQueue (e.g. 
>> CLCommandQueue::create). Am I missing something?
>
> Only because it needs to be stored somewhere, either in the lambda or 
> the object.  I was thinking of having the ability to implement an 
> idempotent close at the time so you can't just have it in the lambda, 
> but I don't think that will be possible with scopes - unless you have 
> scope-per-object which as you mentioned isn't optimal. Otherwise 
> you're basically just back to a stale object on an open scope and 
> would need more checks every time you use it.  I was also thinking 
> broader such as the ffmpeg AVFormatContext example earlier where some 
> other object state defines which close to use.
Btw, it's not that having a scope per object isn't optimal - you 
basically get what you used to get with previous iterations of the API: 
each segment has its own private lifetime instance. If the objects you 
create are relatively long-lived, that might work. If not, you might see 
extra overhead for creating a scope for each new struct. Again, it 
really depends on the library you are dealing with. But yes, if you are 
using scopes to manage _groups_ of resources, having a close() on a 
single resource doesn't make a lot of sense (as then one resource can 
bring down everything else with it).
>
>>
>> In general, starting off with single scope, GC-managed, sounds like a 
>> good default choice - and then sprinkle scopes as you see needs to 
>> deallocate more promptly and/or add more fine-grained lifecycle 
>> management to your library (assuming that is required). Btw, note 
>> that a GC-backed scope can still be released explicitly, if you so 
>> wish (so you have both options at once).
>>
>
> The case i'm trying to solve is where you don't really know the 
> lifecycle of the object.  With opencl you use reference counting to 
> hold objects as long as you need them so they can change ownership 
> and/or be shared amongst different functions and libraries in ways the 
> caller can't control.  My JNI wrapping hid all that by just using java 
> reference gc to effectively do the same thing as far as java was 
> concerned.

Yep, I recall that from our previous discussions. In reality there's no 
reason why you couldn't do the same thing here, right? E.g. you could 
set up a map between memory address and some resource XYZ (to dedup, as 
you did in ZCL), and then create a separate implicit scope for each 
resource XYZ created in the system (with a corresponding cleanup action 
for when it becomes unreachable). And, if you have a separate scope per 
(deduped) Object, you can also support explicit closure as well 
(although that might not make sense for OpenCL in general).

If that system works well for you, and for the library you are working 
it, maybe that's ok?

We have explored ideas for managing complex network of temporal 
dependencies using scopes:

https://inside.java/2021/10/12/panama-scope-dependencies/

This is based on the idea that if we had a way to set up explicit 
temporal dependencies between scope A and scope B (such that B cannot be 
closed until A is), you can now reason about these dependencies in a 
meaningful way. Consider the case of a function which takes two segments 
and return a new one:

MemorySegment makeFrom(MemorySegment a, MemorySegment b, ResourceScope 
target)

With the API we have today, the only sensible option for this API is to 
force that  scope(a) ==(b) == target (with a corner case where target == 
global scope).

If we had dependencies, we could be more flexible:

* we could check that scope(a) <= target and that scope(b) <= target 
(where <= means "child of", meaning that e.g. target cannot be closed 
while scope(a)  is alive)
* we could not even accept a scope, look at infer a scope = 
min(scope(a), scope(b)) - that is, the smaller lifecycle between a and b 
(reasoning being that if either a/b are destroyed, the same should 
happen for the returned segment)

I believe these ideas are promising and powerful, but I also believe 
that they need more bake time, and more evaluation. But I would 
certainly would like to leave the door open in the API to revisit this 
at some later point, as I think that, especially when managing nested 
structs in native library, being able to define dependencies between 
different lifetime is a crucial aspect (at least if you want to lean on 
the explicit closure mechanism).



>
>
> I've also made an embarrassing mistake, closing the context doesn't 
> release it's objects, in opencl they all have a reference count on the 
> context so the other discussion is a little moot.  I believe it does 
> in vulkan although it's an error to close the instance before all of 
> it's objects are closed.
>
> Oh well i'll have to about it more.  Implicit scopes wont be enough in 
> the case of such an api unless each object also retains a java 
> reference to it's parent - leaving it up to the application isn't 
> really safe enough is it?
Implicit scopes alone rely on reachability, so you would need to keep 
things reachable somehow (but something like that was probably also done 
in ZCL, otherwise how did you keep things alive there?).
>
>
> Just FYI command queues are not short lived, a simple app might be:
>
> create a context
> create a small number of queues
> create your kernels
>
> then set up a processing pipeline - could be multiple, could come and go:
> create some memory buffers and images
> maybe create some specific kernels
>
> then execute a processing pipeline - this will happen multiple times, 
> and an individual execution could span multiple threads:
> execute kernels, track execution using events - some will have the 
> lifetime of the pipeline, others a fragment of it, perhaps 
> allocate/free temp image/memory along the way
>
> close everything in reverse
>
> Given the realisation of my mistake on how the refcounting works in 
> opencl i think this should be covered by a few relatively simple scope 
> mechanisms under application control.

I believe that to be the case. Having a separate scope per every single 
OpenCL entity that can be created independently, while theoretically 
possible, seems overkill to me. In reality, as you outline above, some 
of these entities will always be created in batches, to perform a 
certain task. I think that's the time unit you should be focussing on.

E.g. from the above, I'd be tempted to put all "global stuff" (context, 
queues, kernels) in a long-lived, implicit scope. And then have 
shorter-lived, client managed, scopes for each processing pipeline (and 
maybe even shorter-lived ones on top to allocate temp data when 
executing kernels).

Maurizio

>
>  !Z
>