Integrated: 8356760: VectorAPI: Optimize VectorMask.fromLong for all-true/all-false cases
erifan
duke at openjdk.org
Sat Aug 2 07:58:07 UTC 2025
On Fri, 13 Jun 2025 08:33:09 GMT, erifan <duke at openjdk.org> wrote:
> If the input long value `l` of `VectorMask.fromLong(SPECIES, l)` would set or unset all lanes, `VectorMask.fromLong(SPECIES, l)` is equivalent to `maskAll(true)` or `maskAll(false)`. But the cost of the `maskAll` is
> relative smaller than that of `fromLong`. So this patch does the conversion for these cases.
>
> The conversion is done in C2's IGVN phase. And on platforms (like Arm NEON) that don't support `VectorLongToMask`, the conversion is done during intrinsiication process if `MaskAll` or `Replicate` is supported.
>
> Since this optimization requires the input long value of `VectorMask.fromLong` to be specific compile-time constants, and such expressions are usually hoisted out of the loop. So we can't see noticeable performance change.
>
> This conversion also enables further optimizations that recognize maskAll patterns, see [1]. And we can observe a performance improvement of about 7% on both aarch64 and x64.
>
> As `VectorLongToMask` is converted to `MaskAll` or `Replicate`, some existing optimizations recognizing the `VectorLongToMask` will be affected, like
>
> VectorMaskToLong (VectorLongToMask x) => x
>
>
> Hence, this patch also added the following optimizations:
>
> VectorMaskToLong (MaskAll x) => (x & (-1ULL >> (64 - vlen))) // x is -1 or 0
> VectorMaskToLong (VectorStoreMask (Replicate x)) => (x & (-1ULL >> (64 - vlen))) // x is -1 or 0
>
> VectorMaskCast (VectorMaskCast x) => x
>
> And we can see noticeable performance improvement with the above optimizations for floating-point types.
>
> Benchmarks on Nvidia Grace machine with option `-XX:UseSVE=2`:
>
> Benchmark Unit Before Error After Error Uplift
> microMaskFromLongToLong_Double128 ops/s 1522384.986 1324881.46 2835774480 403575069.7 1862.71
> microMaskFromLongToLong_Double256 ops/s 4275.415598 28.560622 4285.587451 27.633101 1
> microMaskFromLongToLong_Double512 ops/s 3702.171936 9.528497 3692.747579 18.47744 0.99
> microMaskFromLongToLong_Double64 ops/s 4624.452243 37.388427 4616.320519 23.455954 0.99
> microMaskFromLongToLong_Float128 ops/s 1239661.887 1286803.852 2842927993 360468218.3 2293.3
> microMaskFromLongToLong_Float256 ops/s 3681.64954 15.153633 3685.411771 21.737124 1
> microMaskFromLongToLong_Float512 ops/s 3007.563025 10.189944 3022.002986 14.137287 1
> microMaskFromLongToLong_Float64 ops/s 1646664.258 1375451.279 2948453900 397472562.4 1790.56
>
>
> Benchmarks on AMD EPYC 9124 16-Core Processor with option `-XX:UseAVX=3`:
>
> Benchmark Unit Before Error After Error Uplift
> microMaskFromLongToLong_Double...
This pull request has now been integrated.
Changeset: f40381e4
Author: erfang <erfang at nvidia.com>
Committer: Jatin Bhateja <jbhateja at openjdk.org>
URL: https://git.openjdk.org/jdk/commit/f40381e41d1356f92546a21c0d24060f8606b9b3
Stats: 1093 lines in 9 files changed: 1080 ins; 0 del; 13 mod
8356760: VectorAPI: Optimize VectorMask.fromLong for all-true/all-false cases
Reviewed-by: xgong, jbhateja
-------------
PR: https://git.openjdk.org/jdk/pull/25793
More information about the hotspot-compiler-dev
mailing list