RFR: 8330611: AES-CTR vector intrinsic may read out of bounds (x86_64, AVX-512) [v2]

[Charles Connell]

On Wed, 24 Apr 2024 00:21:40 GMT, Martin Balao <mbalao at openjdk.org> wrote:

>> We would like to propose a fix for 8330611.
>> 
>> To avoid an out of bounds memory read when the input's size is not multiple of the block size, we read the plaintext/ciphertext tail in 8, 4, 2 and 1 byte batches depending on what it is guaranteed to be available by 'len_reg'. This behavior replaces the read of 16 bytes of input upfront and later discard of spurious data.
>> 
>> While we add 3 extra instructions + 3 extra memory reads in the worst case —to the same cache line probably—, the performance impact of this fix should be low because it only occurs at the end of the input and when its length is not multiple of the block size.
>> 
>> A reliable test case for this bug is hard to develop because we would need accurate heap allocation. The fact that spuriously read data is silently discarded most of the time makes this bug harder to observe. No regressions have been observed in the compiler/codegen/aes jtreg category. Additionally, we verified the fix manually with the debugger.
>> 
>> This work is in collaboration with @franferrax .
>
> Martin Balao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Avoid register conflict in Windows.
>   
>   Co-authored-by: Francisco Ferrari Bihurriet <fferrari at redhat.com>
>   Co-authored-by: Martin Balao <mbalao at redhat.com>

I am planning a blog post about discovering and reporting this bug that will appear on my company's website. I plan to credit everyone involved with the names I see on your profiles here on Github and/or on the OpenJDK JIRA instance. I also may link to github profiles or personal websites if I come across them. If you would like to have your name or link changed or omitted, please let me know. My email address is cconnell at hubspot.com if you would like to contact me there.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18849#issuecomment-2083601585