Vector API - Non Temporal Hints

[Paul Sandoz]

Hi Michaël,

My apologies for not replying sooner, this got buried in my inbox. Thank you for the suggestion.

IIUC, on x64, such non-temporal access instructions require the memory address be aligned to the vector size, otherwise a GP exception will occur. Further, I believe after such accesses a memory fence is required, so I would disagree with you about the change of semantics. This is complex and requires a high-degree of expertise to use correctly. It’s also not clear to me how this behaves on other CPU architectures and whether the API can work in a cross platform manner.

With MemorySegments we can align the memory appropriately [*], but we would need more (as you say) and that adds more complexity to an API that already requires expertise to use. So for now I am inclined not to add such support.

Paul.

[*] The Vector API has no concept of aligned access. The implementation, on x64, will generated unaligned instructions, which have the same performance cost as equivalent aligned instructions if the memory address is aligned to the vector size. 

> On Sep 29, 2023, at 1:05 PM, Michaël Figuière <michael.figuiere at gmail.com> wrote:
> 
> Hi,
> 
> The Vector API currently doesn't offer a way to add a non temporal hint when storing a vector into memory whereas both x86 and ARM instruction sets offer such capability, at least for continuous, non-scattered storage. As the API has been introduced with large dataset manipulation in mind, having such an option could be a valuable way to grab some performance improvement without much effort for both the end developer and the OpenJDK developers.
> 
> An obvious way to add such a capability would be to overload the relevant intoXXX() methods in each XXXVector class to add an extra boolean nonTemporal argument. A convenient aspect of this addition is that it doesn't alter the semantic of the storage method as it's just a performance hint. Thus supporting it for each present and future architecture can be done in a best effort manner.
> 
> Best,
> Michaël