RFR: 8364305: Support AVX10 saturating floating point conversion instructions [v11]

[Sandhya Viswanathan]

On Thu, 11 Sep 2025 02:19:54 GMT, Mohamed Issa <missa at openjdk.org> wrote:

>> Intel® AVX10 ISA [1] extensions added new saturating floating point conversion instructions which comply with definitions in section 5.8 of the 2019 IEEE-754 standard. They can compute floating point to integral type conversions while also handling special inputs such as NaN, +Infinity, and -Infinity.
>> 
>> Without AVX10.2, the current approach starts by converting the floating point value(s) in the source register to the desired integral value(s) in the destination register. In the scalar case, the CVTTSS2SI (single precision) or CVTTSD2SI (double precision) instruction is used. In the vector case, the CVTTPS2DQ (single precision) or CVTTPD2DQ (double precision) is used. However, if the source contains a special value (NaN, -Infinity, +Infinity, <= Integer.MIN_VALUE, or >= Integer.MAX_VALUE), extra handling is required. The specific sequence of instructions involved depends on the source (single precision vs double precision), destination (long, integer, short, or byte), level of parallelization (scalar vs vector), and supported AVX extension type. Essentially though, the special values are mapped to values (NaN -> 0, -Infinity, <= Integer.MIN_VALUE -> Integer.MIN_VALUE, +Infinity, >= Integer.MAX_VALUE -> Integer.MAX_VALUE) in the integer range with the help of a few temporary regis
 ters to store intermediate results.
>> 
>> This change uses the new AVX10.2 scalar (VCVTTSS2SIS or  VCVTTSD2SIS) and vector (VCVTTPS2QQS, VCVTTPS2DQS, VCVTTPD2QQS, and VCVTTPD2DQS) instructions on supported platforms to avoid the extra handling described above. Also, the JTREG tests listed below were used to verify correctness with `-XX:-UseSuperWord` / `-XX:+UseSuperWord` options to exercise both scalar and vector paths. The baseline build used is [OpenJDK v26-b11](https://github.com/openjdk/jdk/releases/tag/jdk-26%2B11).
>> 
>> 1. `jtreg:test/hotspot/jtreg/compiler/codegen/TestByteDoubleVect.java`
>> 2. `jtreg:test/hotspot/jtreg/compiler/codegen/TestByteFloatVect.java`
>> 3. `jtreg:test/hotspot/jtreg/compiler/codegen/TestIntDoubleVect.java`
>> 4. `jtreg:test/hotspot/jtreg/compiler/codegen/TestIntFloatVect.java`
>> 5. `jtreg:test/hotspot/jtreg/compiler/codegen/TestLongDoubleVect.java`
>> 6. `jtreg:test/hotspot/jtreg/compiler/codegen/TestLongFloatVect.java`
>> 7. `jtreg:test/hotspot/jtreg/compiler/codegen/TestShortDoubleVect.java`
>> 8. `jtreg:test/hotspot/jtreg/compiler/codegen/TestShortFloatVect.java`
>> 9. `jtreg:test/hotspot/jtreg/compiler/vectorapi/VectorFPtoIntCastTest.java`
>> 1...
>
> Mohamed Issa has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Check for instructions that shouldn't appear in vector floating point conversion tests
>  - Correctly calculate vector lengths and don't rely on VectorReinterpret in cast2F2X and cast2D2X memory instructions

src/hotspot/cpu/x86/x86.ad line 7715:

> 7713: %}
> 7714: 
> 7715: instruct cast2FtoX_reg_evex(vec dst, vec src) %{

Could be named as castFtoX_reg_avx10.

src/hotspot/cpu/x86/x86.ad line 7728:

> 7726: %}
> 7727: 
> 7728: instruct cast2FtoX_mem_evex(vec dst, memory src) %{

Could be named as  castFtoX_mem_avx10.

src/hotspot/cpu/x86/x86.ad line 7789:

> 7787: %}
> 7788: 
> 7789: instruct cast2DtoX_reg_evex(vec dst, vec src) %{

Could be named as castDtoX_reg_avx10.

src/hotspot/cpu/x86/x86.ad line 7802:

> 7800: %}
> 7801: 
> 7802: instruct cast2DtoX_mem_evex(vec dst, memory src) %{

Could be named as castDtoX_mem_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11728:

> 11726: %}
> 11727: 
> 11728: instruct conv2F2I_reg_reg(rRegI dst, regF src)

Could be named as convF2I_reg_reg_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11739:

> 11737: %}
> 11738: 
> 11739: instruct conv2F2I_reg_mem(rRegI dst, memory src)

Could be named as convF2I_reg_mem_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11762:

> 11760: %}
> 11761: 
> 11762: instruct conv2F2L_reg_reg(rRegL dst, regF src)

Could be named as convF2L_reg_reg_avx10

src/hotspot/cpu/x86/x86_64.ad line 11773:

> 11771: %}
> 11772: 
> 11773: instruct conv2F2L_reg_mem(rRegL dst, memory src)

Could be named as convF2L_reg_mem_avx10

src/hotspot/cpu/x86/x86_64.ad line 11796:

> 11794: %}
> 11795: 
> 11796: instruct conv2D2I_reg_reg(rRegI dst, regD src)

Could be named as convD2I_reg_reg_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11807:

> 11805: %}
> 11806: 
> 11807: instruct conv2D2I_reg_mem(rRegI dst, memory src)

Could be named as convD2I_reg_mem_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11830:

> 11828: %}
> 11829: 
> 11830: instruct conv2D2L_reg_reg(rRegL dst, regD src)

Could be named as convD2L_reg_reg_avx10.

src/hotspot/cpu/x86/x86_64.ad line 11841:

> 11839: %}
> 11840: 
> 11841: instruct conv2D2L_reg_mem(rRegL dst, memory src)

Could be named as convD2L_reg_mem_avx10.

test/hotspot/jtreg/compiler/lib/ir_framework/IRNode.java line 494:

> 492:     }
> 493: 
> 494:     public static final String CAST_F2X = PREFIX + "CAST_F2X" + POSTFIX;

May be we can name CAST_F2X as X86_VCAST_F2X and CAST2_F2X as X86_VCAST_F2X_AVX10.
Then we can use the similar theme for other names below as well.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342294949
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342295729
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342298160
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342300778
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342302012
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342304990
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342305755
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342306364
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342307288
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342308076
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342309412
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342310391
PR Review Comment: https://git.openjdk.org/jdk/pull/26919#discussion_r2342326871