[foreign-abi] RFR: JDK-8243669: Improve library loading for Panama libraries

[Samuel Audet]

I don't feel that we're going in circles, I thought we were making 
progress actually. I fully disagree that this has nothing to do with the 
original RFR. I still firmly stand by my opinion that managing 
resources, whether they are loaded libraries or memory segments, or 
anything else, should be unified in a central framework of sorts. I'm 
sorry you feel differently. Your acrimony towards me though reinforces 
the feeling I have that OpenJDK is going the wrong way and that it is 
unwilling to deal with the needs of the community at large...

Samuel

On 5/12/20 7:13 PM, Maurizio Cimadamore wrote:
> I'm closing down this thread, sorry.
> 
> We're going in circles and this has absolutely nothing to do with the 
> original RFR (yes, you did it again).
> 
> Maurizio
> 
> 
> On 12/05/2020 02:54, Samuel Audet wrote:
>> On 5/11/20 7:11 PM, Maurizio Cimadamore wrote:
>>>> That looks like a good starting point, yes. Are saying that this is 
>>>> intended to be a public API that end users can use to replace 
>>>> mmap/munmap with not only cudaMalloc/cudaFree but whatever they 
>>>> might wish?
>>> That's the spirit, yes. We have to figure out how to make this piece 
>>> of "more unsafe API" cohexist with the rest of the API, but that's 
>>> the direction.
>>
>> Ok, good to hear that. Please do bounce off your ideas about that here 
>> if you can. Right now, for interop with native libraries, with the 
>> current state of MemorySegment, it still wouldn't bring anything more 
>> than what we can already do with sun.misc.Unsafe. It would be pretty 
>> sad if it stayed like that.
>>
>>>> Let's assume this is going to be all public. The next thing that 
>>>> worries me is about simultaneous access from multiple threads. We 
>>>> have no such restrictions in C++, so that is bound to cause issues 
>>>> down the road. Does OpenJDK intend to force this onto the Java 
>>>> community in a similar fashion to JPMS? Or are you open for debate 
>>>> on this, and other points?
>>> The above method already allows you to create unconfined segments. We 
>>> are also exploring (in parallel) very hard ways on how to make these 
>>> restrictions either disappear completely (by using some sort of 
>>> GC-based handhsake), or be less intrusive (by using a broader 
>>> definition of confinement which spans not across a single thread, but 
>>> across multiple, logically related, threads).
>>
>> What about offering an option to do more or less the same thing as 
>> what you've decided to do for library loading? That is the kind of 
>> thing that I was talking about unifying resource management a bit 
>> more. There are use cases when we would like to consider a large 
>> buffer the same way as a loaded library (that is, long lived, probably 
>> will never be deallocated), which is precisely how ByteBuffer is being 
>> handled. It sounds reasonable to me to surface that as an option to 
>> the user. For example, something like this:
>>
>> MemorySegment segment = MemorySegment.allocateCloseableNative(...);
>> // will get crappy performance trying to use that across threads
>> segment.close(); // OK
>>
>> MemorySegment segment = MemorySegment.allocateLongLivedNative(...);
>> // will get normal performance trying to use that across threads
>> segment.close(); // error
>> // ...
>> // the GC will clean that up eventually, maybe, maybe not...
>>
>> Not exactly perfect, but still a step in the right direction.
>>
>>> I didn't see that comment. In general you can attach whatever index 
>>> pre-processing capability you want with 
>>> MemoryHandles.filterCoordinates. Once you have a function that goes 
>>> from a logical index (or tuples of indices) into a index into the 
>>> basic memory segment you can insert that function as a filter of the 
>>> coordinate - and you will get back a var handle which features the 
>>> desired access coordinates, with the right behavior.
>>
>> Ok, thank you, I've replied there:
>> https://github.com/bytedeco/javacpp/issues/391#issuecomment-627044880
>>
>> It would be nice if you were able to make more information public 
>> about what you're planning to do. Leaving the community in the dark 
>> about things like potential avenues for resource management and the 
>> "rich VarHandle combinator API" isn't IMO the best way to build an API 
>> that's supposed to be useful to as many people as possible.
>>
>> Samuel