Interactions between memory segments and byte buffers

[Chris Vest]

Hi,

In our (Netty) experimentation with foreign-memaccess, we have looked into
how memory segments and byte buffers interact, because byte buffers are
still the currency for doing IO, and we have found some bugs, and also some
restrictions that we would prefer to have lifted.

There are two issues we've identified. The less controversial one is that
when you create a ByteBuffer from a slice of a heap MemorySegments, the
buffer will get the base and capacity of the root (non-sliced) memory
segment. The created buffer is effectively not sliced.

This test fails for heap memory segments, but passes for native memory
segments:

@Test
public void buffersAndArraysFromSlices() {
  try (MemorySegment segment = MemorySegment.ofArray(new byte[16])) {
    int newSize = 8;
    var slice = segment.asSlice(4, newSize);

    var bytes = slice.toByteArray();
    assertEquals(newSize, bytes.length);

    var buffer = slice.asByteBuffer();
    // Fails for heap segments, but passes for native segments:
    assertEquals(newSize, buffer.capacity());
  }
}

The second issue is probably more controversial because it relates to this
passage in the MemorySegment.asByteBuffer javadoc:



*If this segment is shared, calling certain I/O operations on the resulting
buffer might result in an unspecified exception being thrown. Examples of
such problematic operations
are FileChannel.read(ByteBuffer), FileChannel.write(ByteBuffer),
java.nio.channels.SocketChannel.read(ByteBuffer) and
java.nio.channels.SocketChannel.write(ByteBuffer).*
This is problematic for us because, considering our usage patterns we have
decided to use shared segments by default, and we need to create
ByteBuffers for doing IO via the safe JDK APIs.

There is also some additional fall-out from how this restriction is
implemented. For instance, if you get a ByteBuffer from a shared, native
memory segment, you cannot call asReadOnlyBuffer on that buffer. You also
cannot create slices of it. Perhaps there are other things as well. This
test demonstrates the problem:

@Test
public void readOnlyByteBufferFromSharedNativeSegment() {
  try (MemorySegment segment = MemorySegment.allocateNative(8).share()) {
    var byteBuffer = segment.asByteBuffer();
    // This fails for native segments, but passes for heap segments.
    // Throws UOE: ByteBuffer derived from shared segments not supported
    byteBuffer.asReadOnlyBuffer();
  }
}

I think this is an aspect of shared segments that is easy to miss. It would
be desirable if shared and confined segments behaved the same in this
regard; both for what ByteBuffer methods are available, and for the ability
to use buffers from shared native segments for IO.

Thanks,
Chris