RFR: 8286177: C2: "failed: non-reduction loop contains reduction nodes" assert failure

[Christian Hagedorn]

On Fri, 20 May 2022 09:45:19 GMT, Roberto Castañeda Lozano <rcastanedalo at openjdk.org> wrote:

> [JDK-8279622](https://bugs.openjdk.java.net/browse/JDK-8279622) introduced an assertion in the SLP analysis verifying that the examined loop does not contain inconsistent reduction information. The assertion is placed at the beginning of the SLP analysis, and can fail even for loops that are not vectorizable by SLP (and hence without risk of being miscompiled). This is the case for the [reported issue](https://bugs.openjdk.java.net/browse/JDK-8286177) and for many other recent failures triggered by JavaFuzzer-generated test cases. This changeset postpones the assertion to the SLP output phase to ensure that it only fails when the program really risks being miscompiled.
> 
> Given that many transformations can invalidate reduction information (see for example the JBS reports for [JDK-8261147](https://bugs.openjdk.java.net/browse/JDK-8261147), [JDK-8279622](https://bugs.openjdk.java.net/browse/JDK-8279622), and [JDK-8286177](https://bugs.openjdk.java.net/browse/JDK-8286177)), a more fundamental fix would be to remove the possibility of inconsistent reduction information by construction, by running the reduction analysis on-demand. I will file a RFE to explore this idea after JDK 19.
> 
> #### Testing
> 
>   - hs-tier1-5 (windows-x64, linux-x64, linux-aarch64, and macosx-x64; debug mode).
>   - Tested that the assertion, in its new placement, would still have caught the bug reported in [JDK-8279622](https://bugs.openjdk.java.net/browse/JDK-8279622) in the original scenario.
>   - Tested that the assertion, in its new placement, does not fail for 27 JavaFuzzer-generated test cases that trigger the assertion in its original placement.

That looks reasonable to go with this lower-risk fix for JDK 19 since the assertion warns about a problem that has been around for quite some time. It makes sense to revisit this again later and propose a full fix in JDK 20 to avoid similar problems with wrongly marked reduction nodes in the future.

-------------

Marked as reviewed by chagedorn (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/8805