<div dir="ltr"><div>Thanks Maurizio, this is definitely helpful. I'll be waiting for the updated constants, meanwhile using my own :)<br></div><div>I also think some of these points could be reflected in documentation, especially the part about MemorySegment dereferencing.<br></div><div><br></div><div>Best regards,</div><div>Alexander Biryukov<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 13 июн. 2022 г. в 17:35, Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com">maurizio.cimadamore@oracle.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Alexander,<br>
MemorySegment and ByteBuffer are a bit different when it comes to <br>
dereferencing. ByteBuffers always allow you to dereference a value, <br>
regardless of its starting offset. So you can read an `int` value at <br>
offset 3 in a buffer - which might or might not be aligned (but of <br>
course I do realized that, when working with packed representation, <br>
alignment is not a concern).<br>
<br>
On the other hand, when a memory segment is dereferenced, the <br>
ValueLayout used for the dereference operation is consulted. This layout <br>
might have alignment information attached (all layout constants provided <br>
by default such as JAVA_INT and JAVA_LONG do).<br>
<br>
So if you try something like<br>
<br>
segment.get(JAVA_INT, 3);<br>
<br>
This will likely fail with an alignment error.<br>
<br>
If this is not the behavior you want, because you work on packed <br>
layouts, or you simply wants the same behavior you get with the <br>
ByteBuffer API, the solution is to declare unaligned layout constants - <br>
like this:<br>
<br>
final static ValueLayout.OfInt JAVA_INT_UNALIGNED = <br>
JAVA_INT.withBitAlignment(8);<br>
<br>
And then:<br>
<br>
segment.get(JAVA_INT_UNALIGNED, 3);<br>
<br>
This will work w/o issues.<br>
<br>
I note that this tends to come up quite a bit - which probably suggests <br>
that working with unaligned layouts is not uncommon - we might consider <br>
adding unaligned layout constants in the ValueLayout API, so that both <br>
aligned and unaligned use cases are supported "out-of-the-box". But the <br>
capability to support unaligned access is there.<br>
<br>
I hope this helps<br>
Maurizio<br>
<br>
On 13/06/2022 14:06, Alexander Biryukov wrote:<br>
> Hi, I'm testing MemorySegment and stumbled across some inconsistent<br>
> behavior in the API.<br>
><br>
> I have the following task:<br>
> * read binary file to RAM, file structure is fixed, but no<br>
> padding/alignment is present (lots of strings of variable length)<br>
> * read file content depending on the user input, e.g. sometimes I need to<br>
> read offset 13, sometimes offset 24 and so on<br>
> * Stored values are typically privitives (byte, long, double etc) or arrays<br>
> (byte[], short[], int[] and so on)<br>
> * Primitives are decoded on the fly to variables, same is true for arrays<br>
> (random access) or sometimes arrays are copied as a blob to another<br>
> MemorySegment<br>
><br>
> I'm trying to create a VarHandle for MemorySegment:<br>
><br>
>> @Test<br>
>> fun sample() {<br>
>> val mem = MemorySegment.allocateNative(100,<br>
>> MemorySession.openImplicit())<br>
>> val buf = ByteBuffer.allocateDirect(100).order(ByteOrder.LITTLE_ENDIAN)<br>
>> buf.put(-20) // 0<br>
>> buf.putDouble(5.0) // 1<br>
>> buf.put(2) // 9<br>
>> buf.putInt(5) // 10<br>
>> buf.putInt(10) // 14<br>
>> buf.flip()<br>
>> mem.copyFrom(MemorySegment.ofBuffer(buf))<br>
>><br>
> val mh = MethodHandles.memorySegmentViewVarHandle(ValueLayout.JAVA_INT)<br>
>> val bh = MethodHandles.byteBufferViewVarHandle(IntArray::class.java,<br>
>> ByteOrder.LITTLE_ENDIAN)<br>
><br>
>> println(mh.get(mem, 14L) as Int) // <-- throws<br>
>> "Misaligned access at address"<br>
>> println(bh.get(mem.asByteBuffer(), 14) as Int) // works, prints "10"<br>
>> }<br>
><br>
> The variant with memorySegmentViewVarHandle throws, while variant with<br>
> byteBufferViewVarHandle works.<br>
> I checked the source code and found that for some reason MemorySegment-variant<br>
> uses offsetNoVMAlignCheck, while ByteBuffer-variant doesn't.<br>
> Here's the code from the source:<br>
><br>
> ByteBuffer-variant<br>
><br>
>> return UNSAFE.getLongUnaligned(<br>
>> ba,<br>
>> ((long) index(ba, index)) + Unsafe.ARRAY_BYTE_BASE_OFFSET,<br>
>> <a href="http://handle.be" rel="noreferrer" target="_blank">handle.be</a>);<br>
>><br>
> MemorySegment-variant<br>
><br>
>> return SCOPED_MEMORY_ACCESS.getIntUnaligned(bb.sessionImpl(),<br>
>> bb.unsafeGetBase(),<br>
>> offsetNoVMAlignCheck(bb, base, handle.alignmentMask),<br>
>> <a href="http://handle.be" rel="noreferrer" target="_blank">handle.be</a>);<br>
>><br>
><br>
> I'm not sure, but it seems like offsetNoVMAlignCheck shouldn't be present<br>
> or there must be an alternative API for querying unaligned data?<br>
> If nothing is wrong, is there a way to query data in MemorySegment without<br>
> alignment?<br>
> ByteBuffer is technically fine for the time being, but I'm expecting a high<br>
> load on the service, and since I'm not using ByteBuffers I'd rather not pay<br>
> for them (GC).<br>
><br>
><br>
> Best regards,<br>
> Alexander Biryukov<br>
</blockquote></div>