RFR: 8371864: GaloisCounterMode.implGCMCrypt0 AVX512/AVX2 intrinsics stubs cause AES-GCM encryption failure for certain payload sizes [v3]

[Jiangli Zhou]

On Thu, 20 Nov 2025 19:37:11 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix Whitespace error.
>
> test/jdk/com/sun/crypto/provider/Cipher/AES/TestGCMSplitBound.java line 65:
> 
>> 63:             new GCMParameterSpec(8 * TAG_SIZE_IN_BYTES, nonce, 0, nonce.length);
>> 64:         Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
>> 65:         cipher.init(Cipher.ENCRYPT_MODE, keySpec, params);
> 
> Er. This is used from `gcmDecrypt`? How does it work without `Cipher.DECRYPT_MODE`?

Good catch. Interestingly the test passed for me on my local machine. Fixed to use Cipher.DECRYPT_MODE when doing gcmDecrypt.

Also an interesting new finding, with the decrypted message verification, I see there are 2 failures out of 200 runs with AVX512. I'm filing a new issue on the specifically, so it can be investigated.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28363#discussion_r2548112676