[foreign-memaccess] musing on the memory access API

[leerho]

Maurizio, Jorn,

Thank you very much for your thoughtful comments and observations!

* At the beginning, the doc claims protection from use after free even
> in concurrent use - looking at the code that doesn't seem to be the case
> though? E.g. it's true that updates to the "valid" bit of the memory
> state are atomic, but that doesn't rule out the possibility of multiple
> threads seeing a "true" value, then being interleaved with a memory
> released, which would ultimately result in access free? I the Java 16
> iteration of the API we address this problem too, but at a much lower
> level (we needed some VM/GC black magic to pull this off).


You are absolutely right about the multi-threading issue!  I wrote this a
couple
years ago and on my re-read I caught that as well!  Our library is strictly
single-threaded, which we mention in other places in the documentation.
I need to correct that statement.  Nonetheless, your solving this problem
at a much lower level is precisely what I would hope you would do! And
at the same time you offer much stronger multithreading guarantees!

* The main differences between the memory access API and your API seem
> to be in how dereference is done - you opted for virtual methods, while
> we go all in on var handles (and then we provide a bunch of static
> accessors on the side). I think the two are similar, although I think
> I'm happy where we landed with our API, since using the pre-baked
> statics is not any harder than using an instance method, but in exchange
> we get a lot of capabilities of out the var handle API (such as atomic
> access and adaptation). This decision has repercussions on the API, of
> course: the fact that we use MemorySegment as a VarHandle coordinate
> means we cannot get too crazy with hierarchies on the MemorySegment
> front - in fact, when we tried to do that (at some point we had
> MappedMemorySegment <: MemorySegment) we ran into performance issues, as
> memory access var handle need exact type information to be fast.


Two comments.
1. I chose virtual methods because as of JDK8, that was the only tool in
the toolbox.
The main advantage of virtual methods is that I can create an API hierarchy
(driven by the needs of the application) that effectively collapses down to
one
class at runtime (as long as it is single inheritance).
I'm not yet sure how I would do it differently with the MemoryAccess API.

...we ran into performance issues, as
> memory access var handles need exact type information to be fast.


This relates to an issue that I'm concerned about, but perhaps because
I don't fully understand why  "memory access var handles *need* exact type
information to be *fast*" or is this just a convention?  At the CPU level,
it
ingests chunks of bytes and then extracts whatever type specified by the
assembly instruction whether it be a 32-bit integer (signed or unsigned),
short, long, float, double or whatever.  I would like the ability to create
a
MemorySegment allocated as bytes, load it with longs (for speed) and
then read it with a MemoryLayout that describes some complex multi-type
data structure (because I know what the bytes represent!).  In other words,
MemorySegment should act like a blob of bytes and reading and writing
from it should behave like a *C union* overlayed with a *C struct.*
I realize this violates the Java principles of strict typing, but if we
really
are interested in speed, we need this ability (even if you force us to
declare it as *unsafe*).  I'm sure you have thought
about this, but I'm not sure, yet, if this is a reality in the code.

This already appears in Java in a few very limited cases.  E.g., I can view
a
*double* as raw bits, perform operations on the raw bits as a long, and
convert it back to a double.  We have some math routines that take
advantage of this.  What is unfortunate is the lack of being able to
convert a double (or long, etc) into bytes and back at an intrinsic level,
which should be very fast.

I looked at your spliterator and it is not clear how I would use it to view
the same blob of bytes with two different layouts.  I must be missing
something :(.

* I believe/hope that the main gripes you had with the byte buffer API
> (which seem to be endianness related) are gone with the memory access
> API. There we made the decision of leaving endianness outside of the
> MemorySegment - e.g. endianness is a property of the VarHandle doing the
> access, not a property of the segment per se. I believe this decision
> paid off (e.g. our segments are completely orthogonal w.r.t. layout
> decisions), and avoids a lot of confusion as to "what's the default" etc.


I have a number of gripes about the ByteBuffer.

1. The most serious issue is the handling of endianness.
First, the default is BigEndian, which today makes no sense as nearly all
CPUs are LE.  And, some byte compression algorithms only work with a given
endianness.  Perhaps I could live with this, but if I am interested in
performance
I would like to match my CPU, so I dutifully set endianness to LE.

ByteBuffer bb = ByteBuffer.allocate(16);

bb.order(ByteOrder.LITTLE_ENDIAN);


Later, suppose I need to do any one of the following common operations:
slice(), duplicate() or asReadOnlyBuffer().

*    The ByteBuffer silently reverts back to BigEndian!*

So the engineer must magically know to always reset the desired endianness
after
every one of those common operations.  And, by the way, this is not
documented
in the Javadocs anywhere I could find.

This is the cause of many difficult to find bugs!  In fact we have cases
where
in large segments of data that have been stored into historical archives,
the
same segment will have different parts of it encoded with LE and other
parts
in BE!  This is a maintenance nightmare.

This bug is easy to find in the ByteBuffer source code.  The calls to
slice(),
duplicate() and asReadOnlyBuffer() return a new ByteBuffer without copying
over the current state of Endianness.

This is why in our Memory Package implementation we made endianness
immutable, once it is chosen, and all equivalent calls to slice(),
duplicate(),
etc() retain the state of endianness.

2. ByteBuffer array handling is clumsy.  It was designed strictly from an
IO
streaming use-case with no alternative for absolute addressing like the
single primitive methods.  The BB API is

ByteBuffer put(<type>[] src, int srcOffset, int length);


Our use case has the need to put or get an array at an absolute offset
from the beginning of the buffer. For example,


> ByteBuffer put(long bufferOffset, <type>[] src, int srcOffset, int length);


Attempting to replicate this method with the current BB API requires:

   - Saving the current setting of position and limit (if used)
   - Setting the position, computing and perhaps checking the limit
   - executing the put() above,
   - restoring position and limit.

This is a real PITA, and could be so easily solved with a few easy to add
methods.

3.  There is no method that allows a high-performance (system level)
copy of a region of one ByteBuffer to another ByteBuffer without going
through the heap.  This is so easy to do with Unsafe, I hope you have
the ability to do this with MemorySegments.  What we need is something like

static void copy(MemorySegment src, long srcOffsetBytes,

MemorySegment dst, long dstOffsetBytes, long lengthBytes)


Since there are no java arrays involved, the length could be a long.
Under the covers, you could easily go parallel with multiple threads if
the size is big.

4. The handling of the positional values is also clumsy IMHO where, for
example,
the Mark is silently invalidated.   Agreed this is documented, but
remembering
the rules where the positionals are suddenly silently changed can be
difficult
unless you do it all the time.  I designed a different positional system
<https://datasketches.apache.org/api/memory/snapshot/apidocs/index.html> (see
BaseBuffer) where there is no need to invalidate them.

I hope you find this of interest.

Cheers,

Lee.