RFR: 8294816: C2: Math.min/max vectorization miscompilation [v2]

[Nick Gasson]

On Wed, 2 Nov 2022 14:42:20 GMT, Bhavana Kilambi <bkilambi at openjdk.org> wrote:

>> C2 miscompiles during auto-vectorization of MinI/MaxI nodes when "short" type operands are involved. When a short and an integer value is compared, C2 generates vector min/max nodes for "short" types which does not result in correct output as it disregards the higher order bits of the integer input. Java API for Math.min/max also only supports the int, long, float and double types but not the subword integer types namely - char, byte and short. Hence, char/short/byte min/max vector instructions should not be generated.
>> This patch ensures that MaxV and MinV vector nodes are only generated for the "int" type for MaxI and MinI nodes during auto-vectorization.
>
> Bhavana Kilambi has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Updated string names for Min and Max IR nodes

Marked as reviewed by ngasson (Reviewer).

-------------

PR: https://git.openjdk.org/jdk/pull/10944