RFR: 8371864: GaloisCounterMode.implGCMCrypt0 AVX512/AVX2 intrinsics stubs cause AES-GCM encryption failure for certain payload sizes [v3]

[Aleksey Shipilev]

On Thu, 20 Nov 2025 05:06:58 GMT, Jiangli Zhou <jiangli at openjdk.org> wrote:

>> Please review the fix in StubGenerator::aesgcm_avx512 and StubGenerator::aesgcm_avx2 to handle some edge cases with input sizes that are not multiple of the block size. 
>> 
>> Thanks to Thomas Holenstein and Lukas Zobernig for analyzing the issue and providing the test case!
>
> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix Whitespace error.

test/jdk/com/sun/crypto/provider/Cipher/AES/TestGCMSplitBound.java line 59:

> 57:     private static final int TAG_SIZE_IN_BYTES = 16;
> 58: 
> 59:     private Cipher getCipher(final byte[] key, final byte[] aad, byte[] nonce)

Suggestion:

    private Cipher getCipher(final byte[] key, final byte[] aad, final byte[] nonce)

test/jdk/com/sun/crypto/provider/Cipher/AES/TestGCMSplitBound.java line 65:

> 63:             new GCMParameterSpec(8 * TAG_SIZE_IN_BYTES, nonce, 0, nonce.length);
> 64:         Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
> 65:         cipher.init(Cipher.ENCRYPT_MODE, keySpec, params);

Er. This is used from `gcmDecrypt`? How does it work without `Cipher.DECRYPT_MODE`?

test/jdk/com/sun/crypto/provider/Cipher/AES/TestGCMSplitBound.java line 89:

> 87:       System.arraycopy(ciphertext, 0, nonce, 0, IV_SIZE_IN_BYTES);
> 88:       Cipher cipher = getCipher(key, aad, nonce);
> 89:       return cipher.doFinal(ciphertext, IV_SIZE_IN_BYTES, ciphertext.length - IV_SIZE_IN_BYTES);

Indenting is still 2-space here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28363#discussion_r2547113559
PR Review Comment: https://git.openjdk.org/jdk/pull/28363#discussion_r2547438961
PR Review Comment: https://git.openjdk.org/jdk/pull/28363#discussion_r2547112755