RFR: 8346664: C2: Optimize mask check with constant offset [v24]

[Emanuel Peter]

On Tue, 25 Feb 2025 06:09:54 GMT, Matthias Ernst <duke at openjdk.org> wrote:

> Related: https://bugs.openjdk.org/browse/JDK-8288683
> 
> I'm still trying to understand:
> 
> * why this is intermittent, it seems to me that it should trigger deterministically

Sometimes that is due to timing, i.e. differences in profiling, what code gets inlined etc. That can change the shape of the IR and the order in which it is processed. That can change the order in which things get folded and propagated, and so in some cases the bug triggers and in others not. It would be good to extract a smaller reproducer where it triggers more reliably. Here `StressIGVN` and `StressCCP` with `RepeatCompilation` can be your friends.

> * what's the severity of this? do we need to back out or is this merely an indicator for a missed downstream optimization. 8288683 has a case where execution would be incorrect, also not sure I understand how that can happen.

Let me explain:
- In IGVN, we only narrow types. If you miss an optimization, then the result is suboptimal, but still correct.
- In CCP, we start with empty types and widen them until the types are correct. If we miss a step, then the type is too narrow and the result is incorrect. Imagine a type is too narrow, and then we have some condition based on it. Imagine the range is wrongly `[0,5]` but should be `[0,10]`. If we check if the value could be `9`, we think it is `false`, but it could actually be `true`. We wrongly constant fold a condition and get wrong results.

Not sure if any of this applies in that example, but it might.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/22856#issuecomment-2680916268