RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory

[Andrew Dinn]

Hi Alan,

On 26/09/18 11:35, Alan Bateman wrote:

> I'm reasonably happy with the approach that we converged on to introduce
> new map modes and use the existing FileChannel.map method. Ideally new
> map modes wouldn't need to be exposed but you've looked into that (to my
> satisfaction at least). One detail that I think may need another
> iteration or two on is whether we need one or two modes. This will
> become clearer once the javadoc is fleshed out a bit further. It maybe
> that one new map mode, "SYNC" for example, that works with the existing
> READ_WRITE mode may be clearer and easier to specify. I think that would
> be consistent with how copy-on-write mappings are exposed with the
> PRIVATE mode. It also provides a 1-1 mapping to the underlying MAP_SYNC
> flag too.

I'm not clear why we should only use one flag. The two flags I specified
reflect two independent use cases, one where data stored in an NVM
device is accessed read-only and another where it is accessed
read-write. Are you suggesting that the read-only case is redundant? I'm
not sure I agree. For example, a utility which might want to review the
state of persistent data while a service is off-line would really want
to pass flag READ_ONLY_PERSISTENT. Of course, it could employ
READ_WRITE_PERSISTENT (or equivalently, SYNC) and just not write the
data but, mutatis mutandis, that same argument would remove the case for
flag READ_ONLY.

> As regards the bigger topic on what the right API is for "memory" then I
> don't think ByteBuffer is the right answer. You've touched on a few of
> the issues in your mail but there are bigger issues around thread safety
> and confinement, also the issue of the buffer position/limit that get in
> the way and the reason why several libraries use Unsafe. There isn't any
> concrete proposal or discussion to point at around splitting out this
> aspect of Project Panama. Stuart and I just pointing out that a better
> solution could emerge which could lead to have an alternative API to map
> a region of NVM as "memory" rather than a mapped byte buffer. If I were
> developing a file system backed by NVM then that is probably the raw API
> that I would want, not MBB.

I really don't understand how thread safety comes into the argument
here. How is some other mechanism going to avoid the need for client
threads -- or, rather, the applications which create them them -- to
manage concurrent updates of NVM? Are you perhaps thinking of some form
of software transactional memory? I'm really struggling to understand
why you keep raising this point without any further detail to explain
how the lack of exclusion and synchronization primitives constitutes a
problem this API that can be bypassed by rolling some equivalent
alternative into another API.

Also, can you explain what you mean by confinement? (thread confinement?).

Also, I don't think I would label this API an attempt to develop a file
system. I think that's rather and overblown characterisation of what it
does. This is an attempt to provide an intermediate storage tier
somewhere between a file system and volatile memory to
create/access/update data across program runs, without incurring the
costs associated with implementing that sort of capability on top of
existing file system implementations.

The use of a byte array layout at the base level is indeed, as the
success of Unix/Linux/MS Windows file systems makes clear, a helpful way
of enabling a variety of application-defined data layouts to be
implemented on top of this storage tier. But I don't really see how that
makes this a file system.

> As regards introducing an API that we could deprecate then that musing
> was about introducing a JDK-specific API. If MapMode were an interface
> then we could have introduce a JDK-specific map mode that wouldn't have
> required rev'ing the standard API. Introducing a completely separate map
> method in a JDK-specific module doesn't seem to be worth it as I think
> we can be confident that the proposed and possible-new.future approaches
> will not conflict.
Ok, so no need for a change there then I guess.

I'm still not quite sure where this reply leaves the JEP though. Shall I
update the Risks and Assumptions section to include mention of
JDK-5029431 as suggested to Stuart? Is there anything else I can do to
progress things?

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander