RFR: 8364305: Support AVX10 saturating floating point conversion instructions [v12]

[Sandhya Viswanathan]

On Thu, 11 Sep 2025 23:10:44 GMT, Mohamed Issa <missa at openjdk.org> wrote:

>> Intel® AVX10 ISA [1] extensions added new saturating floating point conversion instructions which comply with definitions in section 5.8 of the 2019 IEEE-754 standard. They can compute floating point to integral type conversions while also handling special inputs such as NaN, +Infinity, and -Infinity.
>> 
>> Without AVX10.2, the current approach starts by converting the floating point value(s) in the source register to the desired integral value(s) in the destination register. In the scalar case, the CVTTSS2SI (single precision) or CVTTSD2SI (double precision) instruction is used. In the vector case, the CVTTPS2DQ (single precision) or CVTTPD2DQ (double precision) is used. However, if the source contains a special value (NaN, -Infinity, +Infinity, <= Integer.MIN_VALUE, or >= Integer.MAX_VALUE), extra handling is required. The specific sequence of instructions involved depends on the source (single precision vs double precision), destination (long, integer, short, or byte), level of parallelization (scalar vs vector), and supported AVX extension type. Essentially though, the special values are mapped to values (NaN -> 0, -Infinity, <= Integer.MIN_VALUE -> Integer.MIN_VALUE, +Infinity, >= Integer.MAX_VALUE -> Integer.MAX_VALUE) in the integer range with the help of a few temporary regis
 ters to store intermediate results.
>> 
>> This change uses the new AVX10.2 scalar (VCVTTSS2SIS or  VCVTTSD2SIS) and vector (VCVTTPS2QQS, VCVTTPS2DQS, VCVTTPD2QQS, and VCVTTPD2DQS) instructions on supported platforms to avoid the extra handling described above. Also, the JTREG tests listed below were used to verify correctness with `-XX:-UseSuperWord` / `-XX:+UseSuperWord` options to exercise both scalar and vector paths. The baseline build used is [OpenJDK v26-b11](https://github.com/openjdk/jdk/releases/tag/jdk-26%2B11).
>> 
>> 1. `jtreg:test/hotspot/jtreg/compiler/codegen/TestByteDoubleVect.java`
>> 2. `jtreg:test/hotspot/jtreg/compiler/codegen/TestByteFloatVect.java`
>> 3. `jtreg:test/hotspot/jtreg/compiler/codegen/TestIntDoubleVect.java`
>> 4. `jtreg:test/hotspot/jtreg/compiler/codegen/TestIntFloatVect.java`
>> 5. `jtreg:test/hotspot/jtreg/compiler/codegen/TestLongDoubleVect.java`
>> 6. `jtreg:test/hotspot/jtreg/compiler/codegen/TestLongFloatVect.java`
>> 7. `jtreg:test/hotspot/jtreg/compiler/codegen/TestShortDoubleVect.java`
>> 8. `jtreg:test/hotspot/jtreg/compiler/codegen/TestShortFloatVect.java`
>> 9. `jtreg:test/hotspot/jtreg/compiler/vectorapi/VectorFPtoIntCastTest.java`
>> 1...
>
> Mohamed Issa has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Change the floating point conversion instruction, IR nodes, and test rules to make them clearer
>  - Change debug text format of AVX 10.2 vector conversion instructions

src/hotspot/cpu/x86/x86.ad line 7669:

> 7667:   predicate(!VM_Version::supports_avx10_2() &&
> 7668:             !VM_Version::supports_avx512vl() &&
> 7669:             Matcher::vector_length_in_bytes(n->in(1)) < 64 &&

Good to add "is_integral_type(Matcher::vector_element_basic_type(n)) &&" here.

test/hotspot/jtreg/compiler/vectorapi/VectorFPtoIntCastTest.java line 26:

> 24: /**
> 25: * @test
> 26: * @bug 8287835 8320347

Did you mean 8364305 here?

test/hotspot/jtreg/compiler/vectorization/runner/ArrayTypeConvertTest.java line 364:

> 362:         applyIfCPUFeatureAnd = {"avx2", "true", "avx10_2", "false"})
> 363:     @IR(counts = {IRNode.X86_VCAST_F2X_AVX10, "> 0"},
> 364:         applyIfCPUFeature = {"avx10_2", "true"})

Need to add the following for X86_VCAST_F2X as well as X86_VCAST_F2X_AVX10.
applyIfOr = {"AlignVector", "false", "UseCompactObjectHeaders", "false"},

test/hotspot/jtreg/compiler/vectorization/runner/ArrayTypeConvertTest.java line 387:

> 385:         applyIfCPUFeatureAnd = {"avx2", "true", "avx10_2", "false"})
> 386:     @IR(counts = {IRNode.X86_VCAST_F2X_AVX10, "> 0"},
> 387:         applyIfCPUFeature = {"avx10_2", "true"})

Need to add the following for X86_VCAST_F2X as well as X86_VCAST_F2X_AVX10.
applyIfOr = {"AlignVector", "false", "UseCompactObjectHeaders", "false"},

test/hotspot/jtreg/compiler/vectorization/runner/ArrayTypeConvertTest.java line 413:

> 411:         applyIfCPUFeatureAnd = {"avx", "true", "avx10_2", "false"})
> 412:     @IR(counts = {IRNode.X86_VCAST_D2X_AVX10, "> 0"},
> 413:         applyIfCPUFeature = {"avx10_2", "true"})

Need to add the following for X86_VCAST_D2X and X86_VCAST_D2X_AVX10:
applyIf = {"MaxVectorSize", ">=16"},

test/hotspot/jtreg/compiler/vectorization/runner/ArrayTypeConvertTest.java line 432:

> 430:         applyIfCPUFeatureAnd = {"avx", "true", "avx10_2", "false"})
> 431:     @IR(counts = {IRNode.X86_VCAST_D2X_AVX10, "> 0"},
> 432:         applyIfCPUFeature = {"avx10_2", "true"})

Need to add the following for X86_VCAST_D2X and X86_VCAST_D2X_AVX10:
applyIf = {"MaxVectorSize", ">=16"},

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342571300
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342620816
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342615073
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342615727
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342618205
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342618455