RFR: 8288047: Accelerate Poly1305 on x86_64 using AVX512 instructions

[Vladimir Kozlov]

On Wed, 5 Oct 2022 21:28:26 GMT, vpaprotsk <duke at openjdk.org> wrote:

> Handcrafted x86_64 asm for Poly1305. Main optimization is to process 16 message blocks at a time. For more details, left a lot of comments in `macroAssembler_x86_poly.cpp`.
> 
> - Added new KAT test for Poly1305 and a fuzz test to compare intrinsic and java.
>   - Would like to add an `InvalidKeyException` in `Poly1305.java` (see commented out block in that file), but that conflicts with the KAT. I do think we should detect (R==0 || S ==0) so would like advice please.
> - Added a JMH perf test.
>    - JMH test had to use reflection (instead of existing `MacBench.java`), since Poly1305 is not 'properly' registered with the provider.
> 
> Perf before:
> 
> Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score        Error  Units
> Poly1305DigestBench.digest          64              thrpt    8  2961300.661 ± 110554.162  ops/s
> Poly1305DigestBench.digest         256              thrpt    8  1791912.962 ±  86696.037  ops/s
> Poly1305DigestBench.digest        1024              thrpt    8   637413.054 ±  14074.655  ops/s
> Poly1305DigestBench.digest       16384              thrpt    8    48762.991 ±    390.921  ops/s
> Poly1305DigestBench.digest     1048576              thrpt    8      769.872 ±      1.402  ops/s
> 
> and after:
> 
> Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score        Error  Units
> Poly1305DigestBench.digest          64              thrpt    8  2841243.668 ± 154528.057  ops/s
> Poly1305DigestBench.digest         256              thrpt    8  1662003.873 ±  95253.445  ops/s
> Poly1305DigestBench.digest        1024              thrpt    8  1770028.718 ± 100847.766  ops/s
> Poly1305DigestBench.digest       16384              thrpt    8   765547.287 ±  25883.825  ops/s
> Poly1305DigestBench.digest     1048576              thrpt    8    14508.458 ±     56.147  ops/s

Test: jdk/incubator/vector/VectorMaxConversionTests.java#id1
Flags: `-ea -esa -XX:UseAVX=3 -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+UseKNLSetting -XX:+UseZGC`
CPU: Intel 8358 (all AVX512 features).

I think the problem is this subtest runs with ` -XX:+UseKNLSetting`[VectorMaxConversionTests.java#L50](https://github.com/openjdk/jdk/blob/master/test/jdk/jdk/incubator/vector/VectorMaxConversionTests.java#L50) which limits AVX512 features.

Call stack:

V  [libjvm.so+0x6e3bf0]  Assembler::vpslldq(XMMRegister, XMMRegister, int, int)+0x190  (assembler_x86.cpp:5358)
V  [libjvm.so+0x152a23b]  MacroAssembler::poly1305_process_blocks_avx512(Register, Register, Register, Register, Register, Register, Register, Register)+0xc7b  (macroAssembler_x86_poly.cpp:590)
V  [libjvm.so+0x152c23d]  MacroAssembler::poly1305_process_blocks(Register, Register, Register, Register)+0x3ad  (macroAssembler_x86_poly.cpp:849)
V  [libjvm.so+0x192dc00]  StubGenerator::generate_poly1305_processBlocks()+0x170  (stubGenerator_x86_64.cpp:2069)
V  [libjvm.so+0x1936a89]  StubGenerator::generate_initial()+0x419  (stubGenerator_x86_64.cpp:3798)
V  [libjvm.so+0x1937b78]  StubGenerator_generate(CodeBuffer*, int)+0xf8  (stubGenerator_x86_64.hpp:526)
V  [libjvm.so+0x198e695]  StubRoutines::initialize1() [clone .part.0]+0x155  (stubRoutines.cpp:229)
V  [libjvm.so+0xfc4342]  init_globals()+0x32  (init.cpp:123)
V  [libjvm.so+0x1a7268f]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x37f

-------------

PR: https://git.openjdk.org/jdk/pull/10582