[jdk17] RFR: 8267652: c2 loop unrolling by 8 results in reading memory past array

[Vladimir Kozlov]

On Mon, 21 Jun 2021 08:52:23 GMT, Nils Eliasson <neliasso at openjdk.org> wrote:

> I moved this PR to be on top of JDK-17. (The old one can be found here: https://github.com/openjdk/jdk/pull/4527)
> 
> Currently there are a bunch of AVX instructions on x86 that operate on memory that read a full 16-bytes even though only 8 are used. This means we can read out of bounds. This can be reproduced by using -XX:MaxLoopUnrollFactor=8 or -XX:MaxVectorLength=8.
> 
> I've tried creating test cases where a complete unroll results in a 8 byte vector. Then we will choose none-AVX instructions.
> 
> I've tried to patch x86.ad, looking for all uses of LoadVector on instructions that require AVX. I add a predicate that the vector length must be more than 8 bytes. This forces the use the reg-reg variants when the vector length is 8.
> 
> What I am missing is some kind of verification that the fix covers all cases.
> 
> Another additional complexity is that we are using the same instructions in assembler_x86.cpp. I've seen no obvious out-of-bounds reads, but they might be there.
> 
> Best regards,
> Nils Eliasson

okay

-------------

Marked as reviewed by kvn (Reviewer).

PR: https://git.openjdk.java.net/jdk17/pull/108