RFR: 8264973: AArch64: Optimize vector max/min/add reduction of two integers with NEON pairwise instructions
Dong Bo
dongbo at openjdk.java.net
Mon Apr 26 05:58:52 UTC 2021
On aarch64, current implementations of vector reduce_add2I, reduce_max2I, reduce_min2I can be optimized with NEON pairwise instructions:
## reduce_add2I, before
mov w10, v19.s[0]
mov w2, v19.s[1]
add w10, w0, w10
add w10, w10, w2
## reduce_add2I, optimized
addp v23.2s, v24.2s, v24.2s
mov w10, v23.s[0]
add w10, w10, w2
## reduce_max2I, before
dup v16.2d, v23.d[0]
sminv s16, v16.4s
mov w10, v16.s[0]
cmp w10, w0
csel w10, w10, w0, lt
## reduce_max2I, optimized
sminp v16.2s, v23.2s, v23.2s
mov w10, v16.s[0]
cmp w10, w0
csel w10, w10, w0, lt
I don't expect this to change anything of SuperWord, vectorizing of length 2 reductions is disabled by [1].
This is useful for VectorAPI, tested benchmarks in [2], performance can improve ~51% and ~8% for `Int64Vector.ADD` and `Int64Vector.MAX` respectively.
Benchmark (size) Mode Cnt Score Error Units
# optimized
Int64Vector.ADDLanes 1024 thrpt 10 2492.123 ± 23.561 ops/ms
Int64Vector.ADDMaskedLanes 1024 thrpt 10 1825.882 ± 5.261 ops/ms
Int64Vector.MAXLanes 1024 thrpt 10 1921.028 ± 3.253 ops/ms
Int64Vector.MAXMaskedLanes 1024 thrpt 10 1588.575 ± 3.903 ops/ms
Int64Vector.MINLanes 1024 thrpt 10 1923.913 ± 2.117 ops/ms
Int64Vector.MINMaskedLanes 1024 thrpt 10 1596.875 ± 2.163 ops/ms
# default
Int64Vector.ADDLanes 1024 thrpt 10 1644.223 ± 1.885 ops/ms
Int64Vector.ADDMaskedLanes 1024 thrpt 10 1491.502 ± 26.436 ops/ms
Int64Vector.MAXLanes 1024 thrpt 10 1784.066 ± 3.816 ops/ms
Int64Vector.MAXMaskedLanes 1024 thrpt 10 1494.750 ± 3.451 ops/ms
Int64Vector.MINLanes 1024 thrpt 10 1785.266 ± 8.893 ops/ms
Int64Vector.MINMaskedLanes 1024 thrpt 10 1499.233 ± 3.498 ops/ms
Verified correctness with tests `test/jdk/jdk/incubator/vector/`. Also tested linux-aarch64-server-fastdebug tier1-3.
[1] https://github.com/openjdk/jdk/blob/3bf4c904fbbd87d4db18db22c1be384616483eed/src/hotspot/share/opto/superword.cpp#L2004
[2] https://github.com/openjdk/panama-vector/blob/vectorIntrinsics/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int64Vector.java
-------------
Commit messages:
- 8264973: AArch64: Optimize vector max/min/add reduction of two integers with NEON pairwise instructions
Changes: https://git.openjdk.java.net/jdk/pull/3683/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3683&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8264973
Stats: 40 lines in 3 files changed: 2 ins; 12 del; 26 mod
Patch: https://git.openjdk.java.net/jdk/pull/3683.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/3683/head:pull/3683
PR: https://git.openjdk.java.net/jdk/pull/3683
More information about the hotspot-dev
mailing list