MemorySegment.close(), threads

[Maurizio Cimadamore]

On 15/01/2020 12:32, Ty Young wrote:
>
> On 1/15/20 5:38 AM, Maurizio Cimadamore wrote:
>>
>> On 15/01/2020 11:00, Michael Zucchi wrote:
>>>
>>> Evening,
>>>
>>> Has it been considered to have MemorySegment.close() work on any 
>>> thread?
>>>
>>> As far as I can tell doing so wouldn't impact the other checks on 
>>> usage and it would still 'safely' throw appropriate exceptions for 
>>> incorrect use for anything that erroneously retained a stale reference.
>>>
>>> This would enable a couple of common processing flows which are 
>>> currently impossible such as worker threads or parallel streams 
>>> returning results, and using reference queues for any object that 
>>> uses a java-allocated memory segment. This doesn't really occur very 
>>> often as such resources are typically allocated by C (being 
>>> unbeholden to such restrictions), and in the worst case values can 
>>> be copied by hand-marshalling to pojos.
>>
>> You can share a segment across multiple threads using the 
>> MemorySegment::acquire method. The workflow is as follows:
>>
>> 1) master thread creates a segment
>> 2) many worked threads acquire the segment
>> 2a) at this point master thread can no longer close the segment 
>> (since there's threads using it)
>> 3) worked threads do work on their segment (slice?) and then close 
>> their acquired segment - closing an acquired segment does NOT deallocate
>> 4) once all the acquired segments have been closed by the worked 
>> threads, the master thread can proceed closing the original segment 
>> for good (which will deallocate)
>>
>> The main reason for enforcing such a workflow (or _any_ workflow) is 
>> that we want to avoid race conditions where a thread accesses a 
>> segment while another closes it, which, again would lead potentially 
>> to a hard VM crash (which the API wants to avoid).
>>
>> Slight variations on this protocol have been suggested during the 
>> code review for JDK 14 integration where all threads are on an equal 
>> footing, and the last one closing the segment triggers deallocation. 
>> This is also possible to achieve, we need real world validation to 
>> guide us in picking which flavor would be more useful most of the times.
>>
>
> FWIW, all threads having equal footing sounds a lot better to me both 
> from a logical and technical standpoint. There is zero guarantee that 
> the main/master thread is going to be the first one that creates the 
> segments and to enforce such a model feels like Project Panama is 
> trying to dictate how people should write their own code for arbitrary 
> reasons.

We can have a discussion on which model should be best - I think there 
are use cases where master/workers pattern will emerge naturally, other 
cases in which that distinction will be less clear. After few months of 
experience writing code in anger against the API, we will surely have 
more experience as to which idioms work and which don't.

As for the "arbitrary reasons", having races between close/access is 
very very real, and it is what has driven the design where we are now - 
so that we can have both deterministic deallocation, performances 
(because each segment has a single, immutable, thread owner so checks 
are cheap) and safety (no hard VM crashes). Please do not underestimate 
the amount of discussions that went into this (some of which are 
captured in this mailing list, some other in documents such as [1]). The 
ByteBuffer API does not allow a 'close()' method precisely for the lack 
of such discipline in the API (there is no concept of ownership in the 
BB API) - while that might lead to a seemingly simpler API (no need to 
close things), that is also the #1 reason as to why BB are not a 
suitable API for e.g. allocating small structs and passing them to 
functions. The memory access API strikes a different balance in the 
design space, and trades some of the handiness of the BB API for 
predictable performances and deterministic deallocation regardless of 
whether access occurs within a single thread or not.

Again, I invite people to write real code, convert some BB code they 
might have to use the memory access API, and report back on what went 
wrong. We can tune things (the API is still in 'incubating' stage), but 
at the same time we should try not to lose sight of why we needed a new 
API in the first place - which means not _all_ the choices which seem 
no-brainers on the surface will actually be compatible with the design 
tenets of the new API.

>
>
> How can you even tell apart the master thread from a worker thread? No 
> methods AFAIK exists right now, only the ability to acquire the 
> segment and a boolean check to see if the current thread can access 
> the segment. Previously with the Scope API you had a hierarchy of 
> memory allocation units but with Memory Access you don't...

You can check what's the owner of the segment and compare against 
current thread.

Maurizio

[1] - http://cr.openjdk.java.net/~mcimadamore/panama/confinement.html

>
>
>> Maurizio
>>
>>>
>>> Regards,
>>>  Michael
>>>