RFR: 8367333: C2: Vector math operation intrinsification failure

[Emanuel Peter]

On Fri, 12 Sep 2025 19:14:18 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> As part of [JDK-8353786](https://bugs.openjdk.org/browse/JDK-8353786), C2 support for operations backed by the vector math library was completely removed. On JDK side, there is a special dispatching logic added to avoid intrinsic calls in `jdk.internal.vm.vector.VectorSupport`. But it's still possible to observe such paradoxical situations (intrinsic calls with obsolete operation IDs) when processing effectively dead code.
> 
> Consider `FloatVector::lanewiseTemplate`:
> 
>     FloatVector lanewiseTemplate(VectorOperators.Unary op) {
>         if (opKind(op, VO_SPECIAL)) {
>             ...                             
>             else if (opKind(op, VO_MATHLIB)) {
>                 return unaryMathOp(op);
>             }
>         }
>         int opc = opCode(op);
>         return VectorSupport.unaryOp(opc, ...);
>     }
> 
> 
> At runtime, `unaryMathOp` is unconditionally invoked, but during compilation it's possible to end up with an intrinsification attempt of `VectorSupport.unaryOp()` before `opKind(op, VO_SPECIAL)` is inlined. 
> 
> It can be reliably reproduced `-XX:+StressIncrementalInlining` flag.
> 
> The fix is to fail-fast intrinsification rather than crashing the VM.
> 
> Testing: tier1 - tier4

Changes requested by epeter (Reviewer).

test/hotspot/jtreg/compiler/vectorapi/TestVectorMathLib.java line 33:

> 31:  * @test
> 32:  * @bug 8367333
> 33:  * @requires vm.compiler2.enabled

Do you need this `@requires`? It might be nice to be able to run this with other compilers too.

test/hotspot/jtreg/compiler/vectorapi/TestVectorMathLib.java line 40:

> 38:  *                   -XX:CompileCommand=compileonly,compiler.vectorapi.TestVectorMathLib::test*
> 39:  *                   -XX:+StressIncrementalInlining
> 40:  *                       compiler.vectorapi.TestVectorMathLib

Like @jatin-bhateja mentioned: alignment is off.
I'd also like to see a run without flags, maybe with only `-XX:CompileCommand=compileonly,compiler.vectorapi.TestVectorMathLib::test*`

-------------

PR Review: https://git.openjdk.org/jdk/pull/27263#pullrequestreview-3227803098
PR Review Comment: https://git.openjdk.org/jdk/pull/27263#discussion_r2351063466
PR Review Comment: https://git.openjdk.org/jdk/pull/27263#discussion_r2351069465