Support for Matrix Extension

[Paul Sandoz]

Hi Stéphane,

Thanks for sharing that information.

Relatedly, the auto-vectorizer was enhanced to support some VNNI instructions:

  https://bugs.openjdk.java.net/browse/JDK-8214751

And, we are exploring half float support here:

  https://github.com/openjdk/panama-vector/pull/164

We are aware of the AMX extensions but have not given much deep thought into how it might fit into the Vector API or be some separate abstraction belonging to the same larger family. I was not aware of the similar functionality for ARM.

The AMX tile configuration management could make it challenging, as if there is a session of tile-based computation with tile sizes known in advance for some computation.

I think one primary use-case for AMX is to replace vector micro-kernels in a highly optimized gemm-like operations, where data movement and package are equally important.

Paul.
 


> On Nov 19, 2021, at 7:18 AM, Stéphane Dubedat <stephanedubedat666 at gmail.com> wrote:
> 
> Intel next server CPU iteration Sapphire rappids will introduce a new x86
> extension name AMX.
> This potentially disruptive extension allows for hardware accelerated SIMD
> Matrix operations!
> https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-extension-amx-brings-matrix-operations-to-debut-with-sapphire-rapids/
> https://en.wikichip.org/wiki/x86/amx
> 
> Note that ARM also has an analogue extension:
> https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/scalable-matrix-extension-armv9-a-architecture
> 
> So
> 1) The vector API should expose an explicit API for leveraging those new,
> majorly useful, SIMD instructions
> 2) The JVM/Vector API could leverage sometimes those instructions
> automatically even when not specified by the user, if supported at runtime
> and detecting patterns of matrices operations?
> 
> Maybe this could be useful for more things that matrix operations such as N
> dimensional arrays in general? ->
> https://github.com/eclipse/deeplearning4j/tree/master/nd4j
> Ndarray are very useful as can be seen by their widespread use on the
> Python ecosystem through Numpy
> see also this SIMD version https://github.com/JetBrains-Research/viktor