[foreign-abi] RFR: JDK-8243669: Improve library loading for Panama libraries

[Maurizio Cimadamore]

On 01/05/2020 10:57, Samuel Audet wrote:
> On 4/30/20 7:07 PM, Maurizio Cimadamore wrote:
>>> Right, it's not perfect, but I think these kinds of issues are 
>>> solvable, if we're willing to spend time and work on them. For 
>>> example, if something like `PointerScope` could be integrated into 
>>> the Java language itself, we would be able to guarantee that what 
>>> you describe above never happens, making everything thread-safe. I 
>>> don't see any limitations in that regards, but I may be missing 
>>> something. Could you provide an example that fails? Or is there just 
>>> concern about the performance hit that could be incurred (in which 
>>> case I'd still say "let's work on it")?
>>
>> This is the very central issue we're trying to address with memory 
>> segments: how do you allow access to a segment from multiple threads 
>> while:
>>
>> * retaining access performances
>> * retaining deterministic deallocation guarantees
>>
>> It's not a theoretical problem. If you want to make your JavaCPP code 
>> safe you need to add a mutex so that threads can either access memory 
>> OR deallocate. Then I'm sure you won't be happy with the numbers that 
>> will come out of your benchmarks.
>>
>> Other solutions "include" something like reference counting, but not 
>> the same reference counting you do in your API. That is, if a thread 
>> wants to use a "pointer" (in your API) you must create a new instance 
>> just for that thread, you can't just increment some shared counter on 
>> the original pointer. That is, the _new_ thread must _not_ have 
>> access to the original pointer. Otherwise it is possible for people 
>> to write code where they don't call "retain" and they happily access 
>> the pointer from multiple threads, but the pointer doesn't know about 
>> it.
>>
>> While something like that might be made to work (we had something 
>> similar with our MemorySegment::acquire), it is not very appealing 
>> from an API perspective, as it creates "trees" of associated 
>> segments/pointers where the parent cannot be deallocated until all 
>> children are.
>>
>> All this to say what I was trying to say before: wrapping up 
>> AtomicInteger inside some ARC abstraction is _not_ a solution to the 
>> problem. First, it doesn't really take that long to implement,  but, 
>> most importantly, having a class which can do "retain"/"release" 
>> doesn't save you from uncooperative clients trying to use an instance 
>> from a different thread w/o calling "retain".
>>
>> So, I don't see a lot of value for providing such an abstraction in 
>> the JDK. The fact that there are libaries out there which might rely 
>> on reference counting to provide some sort of perceived safety 
>> doesn't automatically make this a candidate for providing something 
>> with the degree of safety that would (and should) be expected from a 
>> Java SE API.
>
> Thank you for bearing with me, but I must seriously be missing 
> something. Please explain why the following doesn't work:

Are you now proposing a _language_ feature - not just an API?

Maurizio

>
> Main Thread:
> {
> // some "global" scope
> Pointer p = // some constructor
> // code emitted by the compiler increments counter of p
> p.initSomeMore()
> // start threads, etc
> // code emitted by the compiler decrements counter of p
> }
>
> Thread 1:
> // some "local" scope
> {
> // code emitted by the compiler increments counter of p
> p.doSomething()
> // code emitted by the compiler decrements counter of p
> }
>
> Thread 2:
> // some other "local" scope
> {
> // code emitted by the compiler increments counter of p
> p.doSomethingElse()
> // code emitted by the compiler decrements counter of p
> }
>
> We have only one instance of the object, while locks, if they are 
> required, only need to happen when accessing the reference counter. 
> The user *does not* have access to the counter, it *cannot* be 
> manually incremented or decremented. I'm not saying this is easy to 
> implement into something meaningful, but I don't see where the 
> roadblocks are.
>
> Samuel