RFR: 8288047: Accelerate Poly1305 on x86_64 using AVX512 instructions [v5]

[Jatin Bhateja]

On Mon, 24 Oct 2022 22:09:29 GMT, vpaprotsk <duke at openjdk.org> wrote:

>> Handcrafted x86_64 asm for Poly1305. Main optimization is to process 16 message blocks at a time. For more details, left a lot of comments in `macroAssembler_x86_poly.cpp`.
>> 
>> - Added new KAT test for Poly1305 and a fuzz test to compare intrinsic and java.
>>   - Would like to add an `InvalidKeyException` in `Poly1305.java` (see commented out block in that file), but that conflicts with the KAT. I do think we should detect (R==0 || S ==0) so would like advice please.
>> - Added a JMH perf test.
>>    - JMH test had to use reflection (instead of existing `MacBench.java`), since Poly1305 is not 'properly' registered with the provider.
>> 
>> Perf before:
>> 
>> Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score        Error  Units
>> Poly1305DigestBench.digest          64              thrpt    8  2961300.661 ± 110554.162  ops/s
>> Poly1305DigestBench.digest         256              thrpt    8  1791912.962 ±  86696.037  ops/s
>> Poly1305DigestBench.digest        1024              thrpt    8   637413.054 ±  14074.655  ops/s
>> Poly1305DigestBench.digest       16384              thrpt    8    48762.991 ±    390.921  ops/s
>> Poly1305DigestBench.digest     1048576              thrpt    8      769.872 ±      1.402  ops/s
>> 
>> and after:
>> 
>> Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score        Error  Units
>> Poly1305DigestBench.digest          64              thrpt    8  2841243.668 ± 154528.057  ops/s
>> Poly1305DigestBench.digest         256              thrpt    8  1662003.873 ±  95253.445  ops/s
>> Poly1305DigestBench.digest        1024              thrpt    8  1770028.718 ± 100847.766  ops/s
>> Poly1305DigestBench.digest       16384              thrpt    8   765547.287 ±  25883.825  ops/s
>> Poly1305DigestBench.digest     1048576              thrpt    8    14508.458 ±     56.147  ops/s
>
> vpaprotsk has updated the pull request incrementally with one additional commit since the last revision:
> 
>   extra whitespace character

Few other non-algorithm change set comments.

src/hotspot/cpu/x86/macroAssembler_x86_poly.cpp line 22:

> 20:  * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
> 21:  * or visit www.oracle.com if you need additional information or have any
> 22:  * questions.

Of late  stub code has been re-organized, to comply with it you may want to remove this file and merge macro-assembly code into a new file stubGenerator_x86_64_poly.cpp on the lines of src/hotspot/cpu/x86/stubGenerator_x86_64_aes.cpp

src/hotspot/cpu/x86/macroAssembler_x86_poly.cpp line 849:

> 847:   jcc(Assembler::less, L_process16Loop);
> 848: 
> 849:   poly1305_process_blocks_avx512(input, length,

Since entire code is based on 512 bit encoding misalignment penalty may be costly here. A scalar peel handling (as done in tail) for input portion before a 64 byte aligned address  could further improve the performance for large block sizes.

src/hotspot/cpu/x86/stubGenerator_x86_64.cpp line 2040:

> 2038: 
> 2039: address StubGenerator::generate_poly1305_processBlocks() {
> 2040:   __ align64();

This can be replaced by  __ align(CodeEntryAlignment);

src/java.base/share/classes/com/sun/crypto/provider/Poly1305.java line 175:

> 173:             // Choice of 1024 is arbitrary, need enough data blocks to amortize conversion overhead
> 174:             // and not affect platforms without intrinsic support
> 175:             int blockMultipleLength = (len/BLOCK_LENGTH) * BLOCK_LENGTH;

Since Poly processes 16 byte chunks, a strength reduced version of above expression could be len & (~(BLOCK_LEN-1)

test/micro/org/openjdk/bench/javax/crypto/full/Poly1305DigestBench.java line 94:

> 92:             throw new RuntimeException(ex);
> 93:         }
> 94:     }

On CLX patch shows performance regression of about 10% for block size 1024-2048+.

CLX (Non-IFMA target)

Baseline (JDK-20):-

Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score   Error  Units
Poly1305DigestBench.digest          64              thrpt    2  3128928.978          ops/s
Poly1305DigestBench.digest         256              thrpt    2  1526452.083          ops/s
Poly1305DigestBench.digest        1024              thrpt    2   509267.401          ops/s
Poly1305DigestBench.digest        2048              thrpt    2   305784.922          ops/s
Poly1305DigestBench.digest        4096              thrpt    2   142175.885          ops/s
Poly1305DigestBench.digest        8192              thrpt    2    72142.906          ops/s
Poly1305DigestBench.digest       16384              thrpt    2    36357.000          ops/s
Poly1305DigestBench.digest     1048576              thrpt    2      676.142          ops/s


Withopt:
Benchmark                   (dataSize)  (provider)   Mode  Cnt        Score   Error  Units
Poly1305DigestBench.digest          64              thrpt    2  3136204.416          ops/s
Poly1305DigestBench.digest         256              thrpt    2  1683221.124          ops/s
Poly1305DigestBench.digest        1024              thrpt    2   457432.172          ops/s
Poly1305DigestBench.digest        2048              thrpt    2   277563.817          ops/s
Poly1305DigestBench.digest        4096              thrpt    2   149393.357          ops/s
Poly1305DigestBench.digest        8192              thrpt    2    79463.734          ops/s
Poly1305DigestBench.digest       16384              thrpt    2    41083.730          ops/s
Poly1305DigestBench.digest     1048576              thrpt    2      705.419          ops/s

-------------

PR: https://git.openjdk.org/jdk/pull/10582