MappedMemorySegment & in-memory filesystems

[Ty Young]

On 6/22/20 9:11 AM, Ty Young wrote:
>
>
>>
>> In other words, I think you should really go down the mapped segment 
>> path if you care about persistence - if that's not your primary use 
>> case, I think that's probably not the right tool for the job?
>
>
> There is no other option - realloc, as you've said before, makes zero 
> commitments in regards to whether the address you get back is purely 
> just expanded. If I went down that path, I'd have to create a messy 
> way to rebase all existing instances of MemoryAddress on the new 
> memory chunk.
>
>
> With MappedMemorySegment this is not the case as it's based on the 
> same file. All existing instances of MemoryAddress are valid since 
> they are backed by the file.
>
>

If the answer is no then whatever. There are in-memory filesystems 
available for Java, they just aren't apart of the JDK nor do they use 
JPMS. Given that a large part of FMA deals with memory I thought it was 
related.


Anyway, the fact that this works is kinda suspect. I wasn't able to 
determine whether or not this is already known about internally after 
gazing into my magical mind-reading crystal ball, so apologies I guess.


TL;DR, by using MemorySegment.ofNativeRestricted you can create a 
MemorySegment that is in a limbo state. This segment will not close when 
closing the parent MemorySegment from which it came from. This segment 
will always report that it is alive unless closed itself but if the File 
backed by the MappedMemorySegment is closed it will crash the JVM on 
read/write access. You can make it usable again by loading the File 
backed by the previous MappedMemorySegment into memory again. Of note is 
that the new MappedMemorySegment can be 1 byte even though, for example, 
8 bytes was carved out of the same file before.


If this isn't known about already then it would be nice if it was made 
part of the official API by adding an "isTethered()" method to 
MappedMemorySegment. I'm not 100% what's going on internally here in 
terms of memory state, my guess is that if the File backing a 
MappedMemorySegment isn't in memory then no memory is consumed even 
though the MemorySegment's return true for an alive state. The 
documentation for isAlive doesn't define what being "alive" means.


Here is code to produce this:


File file = new File("./test.txt");

         if(!file.exists())
             file.createNewFile();
         else
         {
             file.delete();
             file.createNewFile();

         }

         VarHandle handle = MemoryHandles.varHandle(long.class, 
ByteOrder.nativeOrder());

         MappedMemorySegment segment = 
MemorySegment.mapFromPath(file.toPath(), 0, 128, 
FileChannel.MapMode.READ_WRITE);

         MemorySegment sliceSegment = segment.asSlice(0, 8);

         handle.set(sliceSegment.baseAddress(), 20);

         MemorySegment unsafeSliceSegment = 
MemorySegment.ofNativeRestricted(
                 sliceSegment.baseAddress(),
                 CSupport.C_LONGLONG.byteSize(),
                 null,
                 null,
                 null
         );

         segment.force();
         segment.unload();
         segment.close();

         System.out.println("Big Segment Alive? " + segment.isAlive()); 
// prints false
         System.out.println("Sliced Segment Alive? " + 
sliceSegment.isAlive()); // prints false
         System.out.println("Unsafe Sliced Segment Alive? " + 
unsafeSliceSegment.isAlive()); // prints true

         // Removing this line gives a JVM crash on access of 
unsafeSliceSegment
         segment = MemorySegment.mapFromPath(file.toPath(), 0, 128, 
FileChannel.MapMode.READ_WRITE);

System.out.println(handle.get(unsafeSliceSegment.baseAddress()));