RFR: JDK-8267652: c2 loop unrolling by 8 results in reading memory past array

[Vladimir Kozlov]

On Fri, 18 Jun 2021 09:01:11 GMT, Nils Eliasson <neliasso at openjdk.org> wrote:

> Hi,
> 
> Currently there are a bunch of AVX instructions on x86 that operate on memory that read a full 16-bytes even though only 8 are used. This means we can read out of bounds. This can be reproduced by using -XX:MaxLoopUnrollFactor=8 or -XX:MaxVectorLength=8.
> 
> I've tried creating test cases where a complete unroll results in a 8 byte vector. Then we will choose none-AVX instructions. 
> 
> I've tried to patch x86.ad, looking for all uses of LoadVector on instructions that require AVX. I add a predicate that the vector length must be more than 8 bytes. This forces the use the reg-reg variants when the vector length is 8. 
> 
> What I am missing is some kind of verification that the fix covers all cases.
> 
> Another additional complexity is that we are using the same instructions in assembler_x86.cpp. I've seen no obvious out-of-bounds reads, but they might be there. 
> 
> Best regards,
> Nils Eliasson

Nils, your changes are based on JDK 18 but bug's fix version is JDK 17. I think you need to rebase changes on JDK 17 and file new PR.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4527