[foreign-abi] RFR: JDK-8243669: Improve library loading for Panama libraries

[Maurizio Cimadamore]

I'm closing down this thread, sorry.

We're going in circles and this has absolutely nothing to do with the 
original RFR (yes, you did it again).

Maurizio


On 12/05/2020 02:54, Samuel Audet wrote:
> On 5/11/20 7:11 PM, Maurizio Cimadamore wrote:
>>> That looks like a good starting point, yes. Are saying that this is 
>>> intended to be a public API that end users can use to replace 
>>> mmap/munmap with not only cudaMalloc/cudaFree but whatever they 
>>> might wish?
>> That's the spirit, yes. We have to figure out how to make this piece 
>> of "more unsafe API" cohexist with the rest of the API, but that's 
>> the direction.
>
> Ok, good to hear that. Please do bounce off your ideas about that here 
> if you can. Right now, for interop with native libraries, with the 
> current state of MemorySegment, it still wouldn't bring anything more 
> than what we can already do with sun.misc.Unsafe. It would be pretty 
> sad if it stayed like that.
>
>>> Let's assume this is going to be all public. The next thing that 
>>> worries me is about simultaneous access from multiple threads. We 
>>> have no such restrictions in C++, so that is bound to cause issues 
>>> down the road. Does OpenJDK intend to force this onto the Java 
>>> community in a similar fashion to JPMS? Or are you open for debate 
>>> on this, and other points?
>> The above method already allows you to create unconfined segments. We 
>> are also exploring (in parallel) very hard ways on how to make these 
>> restrictions either disappear completely (by using some sort of 
>> GC-based handhsake), or be less intrusive (by using a broader 
>> definition of confinement which spans not across a single thread, but 
>> across multiple, logically related, threads).
>
> What about offering an option to do more or less the same thing as 
> what you've decided to do for library loading? That is the kind of 
> thing that I was talking about unifying resource management a bit 
> more. There are use cases when we would like to consider a large 
> buffer the same way as a loaded library (that is, long lived, probably 
> will never be deallocated), which is precisely how ByteBuffer is being 
> handled. It sounds reasonable to me to surface that as an option to 
> the user. For example, something like this:
>
> MemorySegment segment = MemorySegment.allocateCloseableNative(...);
> // will get crappy performance trying to use that across threads
> segment.close(); // OK
>
> MemorySegment segment = MemorySegment.allocateLongLivedNative(...);
> // will get normal performance trying to use that across threads
> segment.close(); // error
> // ...
> // the GC will clean that up eventually, maybe, maybe not...
>
> Not exactly perfect, but still a step in the right direction.
>
>> I didn't see that comment. In general you can attach whatever index 
>> pre-processing capability you want with 
>> MemoryHandles.filterCoordinates. Once you have a function that goes 
>> from a logical index (or tuples of indices) into a index into the 
>> basic memory segment you can insert that function as a filter of the 
>> coordinate - and you will get back a var handle which features the 
>> desired access coordinates, with the right behavior.
>
> Ok, thank you, I've replied there:
> https://github.com/bytedeco/javacpp/issues/391#issuecomment-627044880
>
> It would be nice if you were able to make more information public 
> about what you're planning to do. Leaving the community in the dark 
> about things like potential avenues for resource management and the 
> "rich VarHandle combinator API" isn't IMO the best way to build an API 
> that's supposed to be useful to as many people as possible.
>
> Samuel