RFC: Experiment in accessing/managing persistent memory from Java

[Andrew Dinn]

I have been helping one of my Red Hat colleagues, Jonathan Halliday, to
investigate provision of a Java equivalent to Intel's libpmem suite of C
libraries [1]. This approach avoids the significant cost of using the
Intel libraries from Java via JNI (or, worse, as a virtual driver for a
persistent memory device).

  Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be
mapped over persistent memory, providing equivalent function to libpmem
itself.

  On top of this he implemented a Java journaled log class providing
equivalent functionality to one of the Intel client libs, libpmemlog,
built over libpmem.

  The modified MappedByteBuffer can be configured to use either i) a
registered native method or ii) a JIT intrinsic to perform the critical
task of cache line writeback i.e. the persistence step (the intrinsic is
my contribution).

Jonathan's tests compare use of JNI, registered native and intrinsic
with an equivalent C program to write a large swathe of records to a
journaled log file stored in persistent memory. Performance is worse
than C when relying on JNI and significantly better with JVM/JDK
support. Indeed, as one might reasonably expect, use of the JIT
intrinsic almost completely eliminates writeback costs.

The journaled log code, jdk dev tree patch, build instructions, test
code plus C equivalent and test results are all available from
Jonathan's git repo [2].

For those who do not want to look at the actual code, the README file
[3] provides background to use of persistent memory, an overview of the
design, and summary details of the test process and results.

[1] https://pmem.io/pmdk/
[2] https://github.com/jhalliday/pmem
[3] http://github.com://jhalliday/pmem/README.md

n.b. Jonathan has experimented with using this same prototype to replace
the journaled log used in the Red Hat Narayana transaction manager. It
provides a significant improvement on the current disk file based log,
both for throughput and latency (the code is not yet available as
getting it to work involved some horrible hacking of the build to
migrate up to jdk11).


regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander