JDK 9 proposal: allocating ByteBuffers on heterogeneous memory

[Paul Sandoz]

Hi Steve,

> On 8 Apr 2016, at 00:03, Dohrmann, Steve <steve.dohrmann at intel.com> wrote:
> 
> Hi Paul,
> 
> We would like to have an an API for Intel's 3D XPoint memory sooner than the JDK 10 timeframe and proposed this API because it seems simple enough to consider for JDK 9.  As you suggest, we will participate in the Panama discussions  in this area.  Any additional guidance you have would be appreciated.
> 

e.g. clone/build the panama forest, join the pamana-dev email list, and start asking questions :-) I can send you links etc. off-line if need be.


> Just to clarify, it is incidental that the proposed Memory interface has only one method.  We see the value of the interface as nominative; a new type that can be passed around to abstract various sources of ByteBuffer memory.
> 

I suspected as much, but would prefer that we gain more experience on what this interface should be, and how it intersects with other efforts, rather than introducing a skeletal version now.

I suppose it’s possible for such an interface to extend from IntFunction for compatibility if existing 8 or 9 dependent libraries use IntFunction for abstracting buffer allocation.

FWIW at one point i did postulate and prototyped a MemoryRegion class but after some thought and feedback made a hasty retreat :-)


> Regarding construction and allocation, our current Memory implementation allocates ByteBuffers by calling the NewDirectByteBuffer JNI function with a pointer to 3D XPoint memory allocated via a supporting native library.

Ok, that’s what we thought when some of us had an off-line discussion about this. It slots in quite nicely, and can be easily abstracted by IntFunction<ByteBuffer> i.e. i don’t think there is anything fundamentally stopping you providing something that would work on Java 8 or 9.


Is there any documentation on the memory ordering properties of 3D XPoint memory? how would it differ from say normal memory? can one access the memory using AVX instructions? does it support unaligned loads/stores? (i am guessing the latter is yes).

(Separately there is also the question of whether this kind of memory is something HotSpot itself could leverage.)


>  The Linux libraries we have worked with are NVML (https://github.com/pmem/nvml/) and memkind (https://github.com/memkind/memkind).  We recently also became aware of the NVM-Direct library (https://github.com/oracle/NVM-Direct).

So do you create JNI bindings to NVML?

Opportunistically perhaps this is also somewhere Panama might be able to help with, since it will provide good improvements over the current JNI experience.

Paul.

> We currently don't need our own subclass and return the ByteBuffer returned by the JNI call.
> 
> Thanks,
> Steve
>