Aligning the source of a FloatVector?

[Peter Kessler OS]

Thanks.  At least you understand the problems.

sun.misc.Unsafe seems ... unsafe, and as you say it only works until the next garbage collection.  Off-heap memory is probably beyond the customers I am working with.  Maybe not beyond their technical skills, but beyond the effort they are willing to go to, considering the contortions they would have to go to in the rest of their code.  Even your proposed arrays of Vector elements (and the necessity of two-level indexing) seems a bridge too far.  It would be great if panama had some hint one could put on an ordinary array to align it with the caches.

`-XX:ObjectAlignmentInBytes=64` works fairly well for benchmarks (on machines where cache lines are 64 bytes :-).  In practice it might waste a lot of space.

How do you explain, to users, the run-to-run variation caused by the lack of alignment?

                                                ... peter

From: Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date: Friday, September 26, 2025 at 08:05
To: Peter Kessler OS <peter.kessler at os.amperecomputing.com>, panama-dev at openjdk.org <panama-dev at openjdk.org>
Subject: Re: Aligning the source of a FloatVector?
Hi Peter,

Indeed, alignment has clear performance impact on vector accesses.

There was a proposal to introduce an API point to hint what's the
closest offset for a vector access of particular size to be naturally
aligned. For now, it's straightforward to implement it on top of
sun.misc.Unsafe (by computing absolute address of oop + element offset).
Then, the offset can be used in pre-loop to align vector accesses in
main loop.

Off-heap memory provides more opportunities to properly control
alignment property. (In particular, MemorySegment features asSlice()
method to enforce arbitrary alignment.)

Unfortunately, it doesn't completely solve the problem, partly because
GC is in play, but also because it is notoriously hard to align accesses
when multiple sources/destinations are in play.

So, for now the best option is to work with off-heap memory (e.g.,
through MemorySegment) and use alignment hints (coded in ad-hoc manner).

Also, for on-heap memory, hyper-alignment was discussion for on-heap
arrays of Vector elements, but it's way too far away in the future to
discuss anything concrete about it (requires flattened arrays).

Best regards,
Vladimir Ivanov

On 9/25/25 16:42, Peter Kessler OS wrote:
> Is there way to control the alignment of `float[]` so that it loads
> nicely into a FloatVector?
>
> `jdk.incubator.vector.FloatVector.fromArray(VectorSpecies<Float>,
> float[], int)` creates a `FloatVector` from some elements of a
> `float[]`.  On a machine with 4 float vector lanes, that call reads 4
> float values from the `float[]`.  If the 4 values from the array are
> properly aligned relative to a cache line boundary -- and if the cache
> line width is a multiple of the number of lanes -- then everything goes
> smoothly because every `FloatVector.fromArray` call will read from only
> one cache line.  If the values in the array are not aligned relative to
> a cache line boundary, then some of the `FloatVector.fromArray` calls
> will have to read from 2 cache lines.  On some processors, reads from 2
> cache lines will take longer than reads from a single cache line.
>
> The JVM promises to align objects on 8-byte boundaries (and a user can
> ask for coarser alignment with `-XX:ObjectAlignmentInBytes=` in powers
> of 2, at the expense of wasting memory and cache space to "dark matter"
> between objects).  But the JVM makes no promises about the alignment of
> elements within an array, other than that they will be "naturally
> aligned" for the type of the element.
>
> I can discover the alignment of the elements of a `float[]` by noting
> the time taken to read them into `FloatVector` instances.  And I can
> make use of that alignment information to pad the array to get
> advantageous alignment for the elements.  But any garbage collection
> could move the array to a different alignment (maintaining
> `ObjectAlignmentInBytes`) which would negate any padding I might have added.
>
> Is there a way (for example, an annotation) that will align the elements
> of an array such that vector reads will not cross cache line
> boundaries?  The alignment mechanism will have to take into account the
> size of the object header (with and without compressed class pointers),
> the length field for the array, and any padding added to naturally align
> the elements of the array.  A Java programmer will have no idea about
> those details.  But they might be willing to annotate an array that they
> were going to use with `FloatVector`.
>
>                                                  ... peter
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20250927/e1947dcf/attachment-0001.htm>