MappedMemorySegment & in-memory filesystems

[Maurizio Cimadamore]

On 22/06/2020 13:09, Ty Young wrote:
> Hi,
>
>
> With libraries such as NVML, the current "allocation scopes" API does 
> not properly model the relationship between a GPU and the 
> MemoryAddresses/MemorySegments that belong to it. Noticing this issue, 
> I've created a solution based on MappedMemorySegments which allow for 
> dynamic memory allocation by reusing the same persistent on-disk file 
> and simply creating new, larger MappedMemorySegment on it.

What is the problem you are trying to solve? In what way the NativeScope 
API is not good for your use case?

I'm surprised you ended up with a mapped segment - are you interested in 
persisting the memory contents to disk?

I guess that, before discussing the specifics, I'd be interested in 
learning more about what exactly you are trying to achieve.

Thanks
Maurizio

>
>
> This works really well and is pretty flexible: you can choose to 
> allocate memory in chunks or as-needed(with performance costs, of 
> course). It has, however, an issue in that the file needs to be stored 
> somewhere. on-disk, like I'm doing now, is an option but it of course 
> has disk I/O issues.
>
>
> The good news is that Java supports custom filesystems. The bad news 
> is that there is no standardized way to create one within the JDK 
> currently, without going with third-party libraries.
>
>
> So my main question here is, would this be something Panama could do?
>
>
> A few other questions:
>
>
> - can anything be to help with MemoryAddress alignment(e.g. a static 
> utility method)? I have to do this with my code but I feel like there 
> has to be a more standardized way of fixing a MemoryAddress's 
> alignment. Even if it's only specific to a MappedMemorySegment, it'd 
> be nice to have.
>
>
> - what is the relationship of a MappedMemorySegment and its underlying 
> file? It seems like all memory contents are periodically written to 
> the file as part of a write-cache or something but I've noticed 
> deleting the file on-disk doesn't affect my application in the 
> slightest while running. Is this supposed to happen?
>
>
> - could a new method be added to MappedMemorySegment to get an 
> expanded view of a file? Something like:
>
>
> public MappedMemorySegment expand(long bytes)
>
>
> ? This is already possible to do with the factory methods but having a 
> simple "expand" method would be shorter and easier to read.
>