[foreign-abi] RFR: JDK-8243669: Improve library loading for Panama libraries

[Samuel Audet]

On 4/30/20 7:07 PM, Maurizio Cimadamore wrote:
>> Right, it's not perfect, but I think these kinds of issues are 
>> solvable, if we're willing to spend time and work on them. For 
>> example, if something like `PointerScope` could be integrated into the 
>> Java language itself, we would be able to guarantee that what you 
>> describe above never happens, making everything thread-safe. I don't 
>> see any limitations in that regards, but I may be missing something. 
>> Could you provide an example that fails? Or is there just concern 
>> about the performance hit that could be incurred (in which case I'd 
>> still say "let's work on it")?
> 
> This is the very central issue we're trying to address with memory 
> segments: how do you allow access to a segment from multiple threads while:
> 
> * retaining access performances
> * retaining deterministic deallocation guarantees
> 
> It's not a theoretical problem. If you want to make your JavaCPP code 
> safe you need to add a mutex so that threads can either access memory OR 
> deallocate. Then I'm sure you won't be happy with the numbers that will 
> come out of your benchmarks.
> 
> Other solutions "include" something like reference counting, but not the 
> same reference counting you do in your API. That is, if a thread wants 
> to use a "pointer" (in your API) you must create a new instance just for 
> that thread, you can't just increment some shared counter on the 
> original pointer. That is, the _new_ thread must _not_ have access to 
> the original pointer. Otherwise it is possible for people to write code 
> where they don't call "retain" and they happily access the pointer from 
> multiple threads, but the pointer doesn't know about it.
> 
> While something like that might be made to work (we had something 
> similar with our MemorySegment::acquire), it is not very appealing from 
> an API perspective, as it creates "trees" of associated 
> segments/pointers where the parent cannot be deallocated until all 
> children are.
> 
> All this to say what I was trying to say before: wrapping up 
> AtomicInteger inside some ARC abstraction is _not_ a solution to the 
> problem. First, it doesn't really take that long to implement,  but, 
> most importantly, having a class which can do "retain"/"release" doesn't 
> save you from uncooperative clients trying to use an instance from a 
> different thread w/o calling "retain".
> 
> So, I don't see a lot of value for providing such an abstraction in the 
> JDK. The fact that there are libaries out there which might rely on 
> reference counting to provide some sort of perceived safety doesn't 
> automatically make this a candidate for providing something with the 
> degree of safety that would (and should) be expected from a Java SE API.

Thank you for bearing with me, but I must seriously be missing 
something. Please explain why the following doesn't work:

Main Thread:
{
// some "global" scope
Pointer p = // some constructor
// code emitted by the compiler increments counter of p
p.initSomeMore()
// start threads, etc
// code emitted by the compiler decrements counter of p
}

Thread 1:
// some "local" scope
{
// code emitted by the compiler increments counter of p
p.doSomething()
// code emitted by the compiler decrements counter of p
}

Thread 2:
// some other "local" scope
{
// code emitted by the compiler increments counter of p
p.doSomethingElse()
// code emitted by the compiler decrements counter of p
}

We have only one instance of the object, while locks, if they are 
required, only need to happen when accessing the reference counter. The 
user *does not* have access to the counter, it *cannot* be manually 
incremented or decremented. I'm not saying this is easy to implement 
into something meaningful, but I don't see where the roadblocks are.

Samuel