<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Aptos",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Thanks. At least you understand the problems.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">sun.misc.Unsafe seems ... unsafe, and as you say it only works until the next garbage collection. Off-heap memory is probably beyond the customers I am working with. Maybe not beyond their technical skills,
but beyond the effort they are willing to go to, considering the contortions they would have to go to in the rest of their code. Even your proposed arrays of Vector elements (and the necessity of two-level indexing) seems a bridge too far. It would be great
if panama had some hint one could put on an ordinary array to align it with the caches.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">`-XX:ObjectAlignmentInBytes=64` works fairly well for benchmarks (on machines where cache lines are 64 bytes :-). In practice it might waste a lot of space.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">How do you explain, to users, the run-to-run variation caused by the lack of alignment?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ... peter<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Vladimir Ivanov <vladimir.x.ivanov@oracle.com><br>
<b>Date: </b>Friday, September 26, 2025 at 08:05<br>
<b>To: </b>Peter Kessler OS <peter.kessler@os.amperecomputing.com>, panama-dev@openjdk.org <panama-dev@openjdk.org><br>
<b>Subject: </b>Re: Aligning the source of a FloatVector?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
<span style="font-size:11.0pt">Hi Peter,<br>
<br>
Indeed, alignment has clear performance impact on vector accesses.<br>
<br>
There was a proposal to introduce an API point to hint what's the <br>
closest offset for a vector access of particular size to be naturally <br>
aligned. For now, it's straightforward to implement it on top of <br>
sun.misc.Unsafe (by computing absolute address of oop + element offset). <br>
Then, the offset can be used in pre-loop to align vector accesses in <br>
main loop.<br>
<br>
Off-heap memory provides more opportunities to properly control <br>
alignment property. (In particular, MemorySegment features asSlice() <br>
method to enforce arbitrary alignment.)<br>
<br>
Unfortunately, it doesn't completely solve the problem, partly because <br>
GC is in play, but also because it is notoriously hard to align accesses <br>
when multiple sources/destinations are in play.<br>
<br>
So, for now the best option is to work with off-heap memory (e.g., <br>
through MemorySegment) and use alignment hints (coded in ad-hoc manner).<br>
<br>
Also, for on-heap memory, hyper-alignment was discussion for on-heap <br>
arrays of Vector elements, but it's way too far away in the future to <br>
discuss anything concrete about it (requires flattened arrays).<br>
<br>
Best regards,<br>
Vladimir Ivanov<br>
<br>
On 9/25/25 16:42, Peter Kessler OS wrote:<br>
> Is there way to control the alignment of `float[]` so that it loads <br>
> nicely into a FloatVector?<br>
> <br>
> `jdk.incubator.vector.FloatVector.fromArray(VectorSpecies<Float>, <br>
> float[], int)` creates a `FloatVector` from some elements of a <br>
> `float[]`. On a machine with 4 float vector lanes, that call reads 4 <br>
> float values from the `float[]`. If the 4 values from the array are <br>
> properly aligned relative to a cache line boundary -- and if the cache <br>
> line width is a multiple of the number of lanes -- then everything goes <br>
> smoothly because every `FloatVector.fromArray` call will read from only <br>
> one cache line. If the values in the array are not aligned relative to <br>
> a cache line boundary, then some of the `FloatVector.fromArray` calls <br>
> will have to read from 2 cache lines. On some processors, reads from 2 <br>
> cache lines will take longer than reads from a single cache line.<br>
> <br>
> The JVM promises to align objects on 8-byte boundaries (and a user can <br>
> ask for coarser alignment with `-XX:ObjectAlignmentInBytes=` in powers <br>
> of 2, at the expense of wasting memory and cache space to "dark matter" <br>
> between objects). But the JVM makes no promises about the alignment of <br>
> elements within an array, other than that they will be "naturally <br>
> aligned" for the type of the element.<br>
> <br>
> I can discover the alignment of the elements of a `float[]` by noting <br>
> the time taken to read them into `FloatVector` instances. And I can <br>
> make use of that alignment information to pad the array to get <br>
> advantageous alignment for the elements. But any garbage collection <br>
> could move the array to a different alignment (maintaining <br>
> `ObjectAlignmentInBytes`) which would negate any padding I might have added.<br>
> <br>
> Is there a way (for example, an annotation) that will align the elements <br>
> of an array such that vector reads will not cross cache line <br>
> boundaries? The alignment mechanism will have to take into account the <br>
> size of the object header (with and without compressed class pointers), <br>
> the length field for the array, and any padding added to naturally align <br>
> the elements of the array. A Java programmer will have no idea about <br>
> those details. But they might be willing to annotate an array that they <br>
> were going to use with `FloatVector`.<br>
> <br>
> ... peter<br>
> <o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>