RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory

[Stuart Marks]

Hi Andrew,

I've been starting to look at some of the buffer-related issues and I've been 
discussing this issue with Alan.

On 9/25/18 2:01 AM, Andrew Haley wrote:
> On 09/24/2018 09:14 AM, Alan Bateman wrote:
> 
>> I'm not questioning the need to support NVM, instead I'm trying to
>> see whether MappedByteBuffer is the right way to expose this in the
>> standard API. Buffers were designed in JSR-51 with specific
>> use-cases in mind but they are problematic for many off-heap cases
>> as they aren't thread safe, are limited to 2GB, lack confinement,
>> only support homogeneous data (no layout support).
> 
> I'm baffled by this assertion. It's true that the 2Gb limit is turning
> into a real pain, but all of the rest are nothing to do with
> ByteBuffers, which are just raw memory. Adding structure is something
> that can be done by third-party libraries or by some future OpenJDK
> API.

If you look around Java SE for a public API to represent raw memory, then 
MappedByteBuffer is the obvious choice; there isn't any realistic alternative 
right now. By asking whether MBB is "the right way to expose" raw memory, I 
think Alan is really saying, is MBB the best API to use to expose raw memory in 
the long run? I think the answer is clearly No.

However, that's not an argument against proceeding with MBB. Rather, it's 
setting expectations that MBB has limitations that impose some pain in the short 
term, that possibly can be worked around, but which in the long term may prove 
to be insurmountable.

For an example of something that might be "insurmountable", I'll use the 2GB 
limitation. Doing something to raise the limit is certainly possible. The 
question is, after retrofitting this into the API, whether the result will be 
something that people want to program with, and whether it will perform well 
enough. It might not.

Another example would be a library layered on top that provides structured 
access. It's certainly possible have such a library that will get the job done. 
However, the Buffer API necessarily requires certain operations to be 
implemented using multiple method calls, or it might require copying in some 
cases. Either of these might impose unacceptable overhead.

There are, however, certain things that can be done with buffers in the short 
term to make things work better. For example, JDK-5029431 absolute bulk put/get 
methods. I suspect this will be quite helpful for the NVM case, and indeed it's 
been something that's been asked for repeatedly for quite some time.

If you (collectively) are aware of this and other limitations, then sure, let's 
proceed with this JEP.

>> So where does this leave us? If support for persistent memory is
>> added to FileChannel.map as we've been discussing then it may not be
>> too bad as the API surface is small. The API surface is just new map
>> modes and a MappedByteBuffer::isPersistent method. The force method
>> that specify a range is something useful to add to MBB anyway.
> 
> Yeah, that's right, it is. While something not yet planned might be an
> alternative, even a better one, the purpose of our faster release
> cadence is to "evolve the Java SE Platform and the JDK at a more rapid
> pace, so that new features [can] be delivered in timelier
> manner". This is timely; waiting for Panama to think of what might be
> possible, not so much.

Agree, "waiting for Panama" is certainly not something that anyone wants to do.

s'marks