Different treatments of offsets in MemorySegment.set and copy methods.

[Maurizio Cimadamore]

On 22/06/2023 08:03, Andrii Lomakin wrote:

> Hi team.
> I am working with foreign API and noticed that offsets are treated 
> differently for the "set" and "copy" methods of MemorySegment in the 
> case of heap-based segments.
>
> If I create a segment from let's say, byte[] and call "set(OfDouble, 
> 0)" on such a segment it doesn't take array offset into account during 
> the calculation of the applied offset value. So it processes this 
> offset as is, and I get an exception about address misalignment.

The alignment exception is by design. If you want to work with unaligned 
values (because we can’t really guarantee any alignment when working on 
a byte[]), then you can use the JAVA_DOUBLE_UNALIGNED constant.

> But if I pass offsets into the "copy" method, they are automatically 
> corrected.

There are several copy methods in memory segments, which all bottom out 
here:

|static void copy(MemorySegment srcSegment, ValueLayout 
srcElementLayout, long srcOffset, MemorySegment dstSegment, ValueLayout 
dstElementLayout, long dstOffset, long elementCount) { ... } |

Now, as you can see this method takes two layouts. The layouts are used 
to determine alignment of the copy, as well as byte swap.

If you pass JAVA_DOUBLE to this copy method you should get exactly the 
same exception you get with the get/set method. But note that the copy 
method that doesn’t take a layout just uses JAVA_BYTE underneath, so 
that might be why you get the impression that the copy is working. See 
below:

|jshell> var seg = MemorySegment.allocateNative(10, 
SegmentScope.auto()); seg ==> MemorySegment{ array: Optional.empty 
address:140000700364496 limit: 10 } jshell> var byteSeg = 
MemorySegment.ofArray(new byte[10]); byteSeg ==> MemorySegment{ array: 
Optional[[B at 7e6cbb7a] address:0 limit: 10 } jshell> 
MemorySegment.copy(byteSeg, 0, seg, 0, 10); // ok, because it uses 
JAVA_BYTE jshell> MemorySegment.copy(byteSeg, JAVA_DOUBLE, 0, seg, 
JAVA_DOUBLE, 0, 1); // fails | Exception 
java.lang.IllegalArgumentException: Source segment incompatible with 
alignment constraints | at MemorySegment.copy (MemorySegment.java:1352) 
| at (#6:1) jshell> MemorySegment.copy(byteSeg, JAVA_DOUBLE_UNALIGNED, 
0, seg, JAVA_DOUBLE_UNALIGNED, 0, 1); // ok again |

The treatment of alignment w.r.t. heap segments is explained in great 
details here:

https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/lang/foreign/MemorySegment.html

(Java 21 has a similar section).

Note that this has nothing to do with methods not treating offsets 
uniformly - offsets are in fact treated uniformly, and always checked 
against the alignment constraints of the layout being used for the 
dereference operation.

If you don’t care about alignment, use unaligned layouts (for which 
constants are also provided), or use segments that wrap long[], which 
allow for more alignment.

The current behavior is kind of a forced move, driven by the fact that 
heap segments have different alignment properties based on platform and 
JVM implementation. So the alignment of heap segment is defined in terms 
of the lowest common denominator. If the API didn’t do this, you might 
write some code using memory segments that would work well in one 
platform and then fail on another because of different alignment 
constraints. While platform-dependent behavior is fair game when 
interacting with native code, in this case you would be able to get 
platform-dependent behavior only when accessing memory, and we didn’t 
want that (as this API will likely be used as a replacement for 
ByteBuffer in many cases, e.g. even when no native interop is needed).

Cheers
Maurizio

> Is it done by design?
> IMHO, both methods should treat offsets uniformly, and that is not 
> taking into account that such treatments of offsets in "set" methods 
> make it very hard to work with heap-based segments.
>
> P.S. I am using JDK 20.
> -- 
> Best regards,
> Andrii Lomakin.
>
​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230622/956a854a/attachment-0001.htm>