Memory segment and unaligned access
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Tue Jun 14 08:09:39 UTC 2022
On 14/06/2022 00:51, Alexander Biryukov wrote:
> Thanks Maurizio, this is definitely helpful. I'll be waiting for the
> updated constants, meanwhile using my own :)
> I also think some of these points could be reflected in documentation,
> especially the part about MemorySegment dereferencing.
Hi,
the documentation can be improved perhaps. I note that there is already
a section on alignment:
https://download.java.net/java/early_access/jdk19/docs/api/java.base/java/lang/foreign/MemorySegment.html#segment-alignment
Which also explains in details how alignment w.r.t. heap segments is
enforced. But perhaps what's missing is a section which says: ok, if you
don't care about alignment, just do this.
Maurizio
>
> Best regards,
> Alexander Biryukov
>
> пн, 13 июн. 2022 г. в 17:35, Maurizio Cimadamore
> <maurizio.cimadamore at oracle.com>:
>
> Hi Alexander,
> MemorySegment and ByteBuffer are a bit different when it comes to
> dereferencing. ByteBuffers always allow you to dereference a value,
> regardless of its starting offset. So you can read an `int` value at
> offset 3 in a buffer - which might or might not be aligned (but of
> course I do realized that, when working with packed representation,
> alignment is not a concern).
>
> On the other hand, when a memory segment is dereferenced, the
> ValueLayout used for the dereference operation is consulted. This
> layout
> might have alignment information attached (all layout constants
> provided
> by default such as JAVA_INT and JAVA_LONG do).
>
> So if you try something like
>
> segment.get(JAVA_INT, 3);
>
> This will likely fail with an alignment error.
>
> If this is not the behavior you want, because you work on packed
> layouts, or you simply wants the same behavior you get with the
> ByteBuffer API, the solution is to declare unaligned layout
> constants -
> like this:
>
> final static ValueLayout.OfInt JAVA_INT_UNALIGNED =
> JAVA_INT.withBitAlignment(8);
>
> And then:
>
> segment.get(JAVA_INT_UNALIGNED, 3);
>
> This will work w/o issues.
>
> I note that this tends to come up quite a bit - which probably
> suggests
> that working with unaligned layouts is not uncommon - we might
> consider
> adding unaligned layout constants in the ValueLayout API, so that
> both
> aligned and unaligned use cases are supported "out-of-the-box".
> But the
> capability to support unaligned access is there.
>
> I hope this helps
> Maurizio
>
> On 13/06/2022 14:06, Alexander Biryukov wrote:
> > Hi, I'm testing MemorySegment and stumbled across some inconsistent
> > behavior in the API.
> >
> > I have the following task:
> > * read binary file to RAM, file structure is fixed, but no
> > padding/alignment is present (lots of strings of variable length)
> > * read file content depending on the user input, e.g. sometimes
> I need to
> > read offset 13, sometimes offset 24 and so on
> > * Stored values are typically privitives (byte, long, double
> etc) or arrays
> > (byte[], short[], int[] and so on)
> > * Primitives are decoded on the fly to variables, same is true
> for arrays
> > (random access) or sometimes arrays are copied as a blob to another
> > MemorySegment
> >
> > I'm trying to create a VarHandle for MemorySegment:
> >
> >> @Test
> >> fun sample() {
> >> val mem = MemorySegment.allocateNative(100,
> >> MemorySession.openImplicit())
> >> val buf =
> ByteBuffer.allocateDirect(100).order(ByteOrder.LITTLE_ENDIAN)
> >> buf.put(-20) // 0
> >> buf.putDouble(5.0) // 1
> >> buf.put(2) // 9
> >> buf.putInt(5) // 10
> >> buf.putInt(10) // 14
> >> buf.flip()
> >> mem.copyFrom(MemorySegment.ofBuffer(buf))
> >>
> > val mh =
> MethodHandles.memorySegmentViewVarHandle(ValueLayout.JAVA_INT)
> >> val bh =
> MethodHandles.byteBufferViewVarHandle(IntArray::class.java,
> >> ByteOrder.LITTLE_ENDIAN)
> >
> >> println(mh.get(mem, 14L) as Int) // <-- throws
> >> "Misaligned access at address"
> >> println(bh.get(mem.asByteBuffer(), 14) as Int) // works,
> prints "10"
> >> }
> >
> > The variant with memorySegmentViewVarHandle throws, while
> variant with
> > byteBufferViewVarHandle works.
> > I checked the source code and found that for some reason
> MemorySegment-variant
> > uses offsetNoVMAlignCheck, while ByteBuffer-variant doesn't.
> > Here's the code from the source:
> >
> > ByteBuffer-variant
> >
> >> return UNSAFE.getLongUnaligned(
> >> ba,
> >> ((long) index(ba, index)) + Unsafe.ARRAY_BYTE_BASE_OFFSET,
> >> handle.be
> <https://urldefense.com/v3/__http://handle.be__;!!ACWV5N9M2RV99hQ!LiFa43Fuc1M0ZIi2xsVdjDyRPIsqkSPux6u3cqQXwzuopEHcKeuZHYrniUbTV6Bu73hNjNbEA5fVv14EOp9dOgdnxA$>);
> >>
> > MemorySegment-variant
> >
> >> return SCOPED_MEMORY_ACCESS.getIntUnaligned(bb.sessionImpl(),
> >> bb.unsafeGetBase(),
> >> offsetNoVMAlignCheck(bb, base, handle.alignmentMask),
> >> handle.be
> <https://urldefense.com/v3/__http://handle.be__;!!ACWV5N9M2RV99hQ!LiFa43Fuc1M0ZIi2xsVdjDyRPIsqkSPux6u3cqQXwzuopEHcKeuZHYrniUbTV6Bu73hNjNbEA5fVv14EOp9dOgdnxA$>);
> >>
> >
> > I'm not sure, but it seems like offsetNoVMAlignCheck shouldn't
> be present
> > or there must be an alternative API for querying unaligned data?
> > If nothing is wrong, is there a way to query data in
> MemorySegment without
> > alignment?
> > ByteBuffer is technically fine for the time being, but I'm
> expecting a high
> > load on the service, and since I'm not using ByteBuffers I'd
> rather not pay
> > for them (GC).
> >
> >
> > Best regards,
> > Alexander Biryukov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20220614/deca3842/attachment.htm>
More information about the panama-dev
mailing list