compose MemorySegments

[Douglas Surber]

I'm a newcomer to Panama. My use case seems a little different from what Panama currently supports. While Panama has most of what I need, there's one thing missing, the ability to compose MemorySegments into a single composite MemorySegment that seamlessly spans all the components.

My code reads data from a network. The size and number of fields can be large so the total amount of data read at one time can be large. The code does not know the length of the data in advance, finding it only by reading all the data. Also the code does not fully interpret the data as it reads it. That's too expensive. Instead the code copies the data from the network into a sequence of fixed size blocks, byte[]s. When the user application needs a field the code uses the metadata included in the data to locate the desired field and extract it from the blocks.

Currently the code has to manipulate the bytes individually to construct the Java object the app requested. Keep in mind that a field value can begin at any offset in a block and can span block boundaries. This byte manipulation is relatively expensive.

With the existing Panama API I can create a MemorySegment for an individual block, offset it to the start of a field and extract a value so long as it does not span a block boundary. But if the field spans a block boundary that no longer works. 

What I need is a way to compose MemorySegments, one for each block, into a single MemorySegment that spans them all hiding the block boundaries. 

	public static MemorySegment of(MemorySegment ... subsegments);

I also would like to append a MemorySegment to an existing composite MemorySegment.

	public MemorySegment append(MemorySegment tail);

Given this I could compose all the blocks into a single MemorySegment and then ignore the block boundaries.

	List<byte[]> blocks = ... ;

	MemorySegment[] individualBlocks = new MemorySegment[blocks.length()];
	int i = 0;
	for (byte[] b : blocks) individualBlocks[i++] = MemorySegment.ofArray(b);
	MemorySegment allBlocks = MemorySegment.of(individualBlocks);
	
	protected String getString(long byteOffset, int charLength) {
		MemorySegment dataPointer = allBlocks.asSlice(byteOffset, charLength * 2);
		System.out.println(new String(dataPointer.toCharArray()));
	}

Naively toCharArray would allocate a new char[] and copy the bits into that array then the String constructor would copy the bits from the char[] into its internal byte[]. At least one copy is necessary but it would be nice if this could be optimized to avoid allocation of the temporary char[] and the copy into it.

I do have a question on the toXxxArray methods. How is endianness accounted for? The data I get from the network may have a different endianness from the platform.

Also, it would be convenient to add methods such as getInt to MemorySegment analogous to toIntArray. I know this can be done with a VarHandle but getting a single primitive value is likely to be such a common use case that encapsulating it into a method would be helpful.

Douglas