RFR: 8264973: AArch64: Optimize vector max/min/add reduction of two integers with NEON pairwise instructions

Dong Bo dongbo at openjdk.java.net
Mon Apr 26 05:58:52 UTC 2021


On aarch64, current implementations of vector reduce_add2I, reduce_max2I, reduce_min2I can be optimized with NEON pairwise instructions:


## reduce_add2I, before
mov    w10, v19.s[0]
mov    w2, v19.s[1]
add    w10, w0, w10
add    w10, w10, w2
## reduce_add2I, optimized
addp   v23.2s, v24.2s, v24.2s
mov    w10, v23.s[0]
add    w10, w10, w2

## reduce_max2I, before
dup    v16.2d, v23.d[0]
sminv  s16, v16.4s
mov    w10, v16.s[0]
cmp    w10, w0
csel   w10, w10, w0, lt
## reduce_max2I, optimized
sminp  v16.2s, v23.2s, v23.2s
mov    w10, v16.s[0]
cmp    w10, w0
csel   w10, w10, w0, lt


I don't expect this to change anything of SuperWord, vectorizing of length 2 reductions is disabled by [1].
This is useful for VectorAPI, tested benchmarks in [2], performance can improve ~51% and ~8% for `Int64Vector.ADD` and `Int64Vector.MAX` respectively.


Benchmark                   (size)   Mode  Cnt     Score    Error   Units
# optimized
Int64Vector.ADDLanes          1024  thrpt   10  2492.123 ± 23.561  ops/ms
Int64Vector.ADDMaskedLanes    1024  thrpt   10  1825.882 ±  5.261  ops/ms
Int64Vector.MAXLanes          1024  thrpt   10  1921.028 ±  3.253  ops/ms
Int64Vector.MAXMaskedLanes    1024  thrpt   10  1588.575 ±  3.903  ops/ms
Int64Vector.MINLanes          1024  thrpt   10  1923.913 ±  2.117  ops/ms
Int64Vector.MINMaskedLanes    1024  thrpt   10  1596.875 ±  2.163  ops/ms
# default
Int64Vector.ADDLanes          1024  thrpt   10  1644.223 ±  1.885  ops/ms
Int64Vector.ADDMaskedLanes    1024  thrpt   10  1491.502 ± 26.436  ops/ms
Int64Vector.MAXLanes          1024  thrpt   10  1784.066 ±  3.816  ops/ms
Int64Vector.MAXMaskedLanes    1024  thrpt   10  1494.750 ±  3.451  ops/ms
Int64Vector.MINLanes          1024  thrpt   10  1785.266 ±  8.893  ops/ms
Int64Vector.MINMaskedLanes    1024  thrpt   10  1499.233 ±  3.498  ops/ms


Verified correctness with tests `test/jdk/jdk/incubator/vector/`. Also tested linux-aarch64-server-fastdebug tier1-3.

[1] https://github.com/openjdk/jdk/blob/3bf4c904fbbd87d4db18db22c1be384616483eed/src/hotspot/share/opto/superword.cpp#L2004
[2] https://github.com/openjdk/panama-vector/blob/vectorIntrinsics/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int64Vector.java

-------------

Commit messages:
 - 8264973: AArch64: Optimize vector max/min/add reduction of two integers with NEON pairwise instructions

Changes: https://git.openjdk.java.net/jdk/pull/3683/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3683&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8264973
  Stats: 40 lines in 3 files changed: 2 ins; 12 del; 26 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3683.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3683/head:pull/3683

PR: https://git.openjdk.java.net/jdk/pull/3683


More information about the hotspot-compiler-dev mailing list