[foreign-abi] RFR: JDK-8243669: Improve library loading for Panama libraries

[Samuel Audet]

On 4/28/20 11:15 AM, Samuel Audet wrote:
> On 4/28/20 10:39 AM, Maurizio Cimadamore wrote:
>> On 27/04/2020 23:13, Samuel Audet wrote:
>>> If reference counting gets used for both memory allocations and library
>>> loading, what about extracting that functionality into a public API that we
>>> could use with any other resources out there, such as GPU resources?
>>> JavaCPP is already doing that and it works great, but I don't see any
>>> reason why this kind of feature shouldn't be part of the JDK, something
>>> I've mentioned before...
>>
>> There's no _explicit_ reference counting mechanism anywhere - not in 
>> the memory segment API, not in the library lookup API (which is what 
>> changed here). Not sure what you have in mind - but with the foreign 
>> API we have been trying to stay clear of explicit 'retain'/'release' 
>> calls.  Is there a specific reason why you think that the JDK would be 
>> in a unique position to provide a better solution for something like 
>> this?
> 
> I understand that there is no "explicit reference counting", but it is 
> nonetheless exactly what it is doing "under the hood". Let's put aside 
> the question of offering a standard API for reference counting, and 
> start with a simpler problem first. Since you wish to abstract away 
> completely reference counting from users when it comes to memory 
> allocations and library loading, what are you planning to do about its 
> limitations, mainly when dealing with circular references?

After rereading your proposal a second time, it's clear that you are 
planning simply to fall back on the GC.

Well, I do agree that we shouldn't be doing reference counting manually, 
but as you know, since you've already started experimenting with it, 
there is smarter way of doing it, aka "scopes".

The idea with a standard API for reference counting would be to offer a 
framework that could be used for any native resources, not just memory 
segments or libraries or whatever next Panama is going to decide is 
"important", and that could be shared across any number of native 
libraries that are often used together to manage their resources in a 
sane way. Just for reference, here's an example with OpenCV and 
TensorFlow using JavaCPP's PointerScope:
http://bytedeco.org/news/2018/07/17/bytedeco-as-distribution/

It works well, and I do see the JDK offering something like that, but 
more ironed out, etc simply because it also works fine with C++, Python, 
Swift, etc as part of the language itself!

Do you disagree and if so, why?

Samuel