Integrated: 8284981: Support the vectorization of some counting-down loops in SLP

[Fei Gao]

On Tue, 19 Apr 2022 02:12:09 GMT, Fei Gao <fgao at openjdk.org> wrote:

> SLP can vectorize basic counting-down or counting-up loops. But for the counting-down loop below, in which array index scale
> is negative and index starts from a constant value, SLP can't succeed in vectorizing.
> 
> 
>   private static final int SIZE = 2345;
>   private static int[] a = new int[SIZE];
>   private static int[] b = new int[SIZE];
> 
>   public static void bar() {
>       for (int i = 1000; i > 0; i--) {
>           b[SIZE - i] = a[SIZE - i];
>       }
>   }
> 
> 
> Generally, it's necessary to find adjacent memory operations, i.e. load/store, after unrolling in SLP. Constructing SWPointers[1] for all memory operations is a key step to determine if these memory operations are adjacent. To construct a SWPointer successfully, SLP should first recognize the pattern of the memory address and normalize it. The address pattern of the memory operations in the case above can be visualized as:
> ![image](https://user-images.githubusercontent.com/39403138/163905008-e9d62a4a-74f1-4d05-999b-8c4d5fc84d2b.png)
> which is equivalent to `(N - (long) i) << 2`. SLP recursively resolves the address mode by SWPointer::scaled_iv_plus_offset(). When arriving at the `SubL` node, it accepts `SubI` only and finally rejects the pattern of the case above[2]. In this way, SLP can't construct effective SWPointers for these memory operations and the process of vectorization breaks off.
> 
> The pattern like `(N - (long) i) << 2` is formal and easy to resolve. We add the pattern of SubL in the patch to vectorize counting-down loops like the case above.
> 
> After the patch, generated loop code for above case is like below on
> aarch64:
> 
> LOOP: mov   w10, w12
>       sxtw  x12, w10
>       neg   x0, x12
>       lsl   x0, x0, #2
>       add   x1, x17, x0
>       ldr   q16, [x1, x2]
>       add   x0, x18, x0
>       str   q16, [x0, x2]
>       ldr   q16, [x1, x13]
>       str   q16, [x0, x13]
>       ldr   q16, [x1, x14]
>       str   q16, [x0, x14]
>       ldr   q16, [x1, x15]
>       sub   x12, x11, x12
>       lsl   x12, x12, #2
>       add   x3, x17, x12
>       str   q16, [x0, x15]
>       ldr   q16, [x3, x2]
>       add   x12, x18, x12
>       str   q16, [x12, x2]
>       ldr   q16, [x1, x16]
>       str   q16, [x0, x16]
>       ldr   q16, [x3, x14]
>       str   q16, [x12, x14]
>       ldr   q16, [x3, x15]
>       str   q16, [x12, x15]
>       sub   w12, w10, #0x20
>       cmp   w12, #0x1f
>       b.gt  LOOP
> 
> 
> This patch also works on x86 simd machines. We tested full jtreg on both aarch64 and x86 platforms. All tests passed.
> 
> [1] https://github.com/openjdk/jdk/blob/b56df2808d79dcc1e2d954fe38dd84228c683e8b/src/hotspot/share/opto/superword.cpp#L3826
> [2] https://github.com/openjdk/jdk/blob/b56df2808d79dcc1e2d954fe38dd84228c683e8b/src/hotspot/share/opto/superword.cpp#L3953

This pull request has now been integrated.

Changeset: df7fba1c
Author:    Fei Gao <fgao at openjdk.org>
Committer: Jie Fu <jiefu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/df7fba1cda336c3d9940f0496082bff715711b68
Stats:     81 lines in 3 files changed: 76 ins; 0 del; 5 mod

8284981: Support the vectorization of some counting-down loops in SLP

Reviewed-by: roland, kvn

-------------

PR: https://git.openjdk.java.net/jdk/pull/8289