RFR: 8354282: C2: more crashes in compiled code because of dependency on removed range check CastIIs

[Quan Anh Mai]

On Thu, 10 Apr 2025 15:15:54 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> This is a variant of 8332827. In 8332827, an array access becomes
> dependent on a range check `CastII` for another array access. When,
> after loop opts are over, that RC `CastII` was removed, the array
> access could float and an out of bound access happened. With the fix
> for 8332827, RC `CastII`s are no longer removed.
> 
> With this one what happens is that some transformations applied after
> loop opts are over widen the type of the RC `CastII`. As a result, the
> type of the RC `CastII` is no longer narrower than that of its input,
> the `CastII` is removed and the dependency is lost.
> 
> There are 2 transformations that cause this to happen:
> 
> - after loop opts are over, the type of the `CastII` nodes are widen
>   so nodes that have the same inputs but a slightly different type can
>   common.
>   
> - When pushing a `CastII` through an `Add`, if of the type both inputs
>   of the `Add`s are non constant, then we end up widening the type
>   (the resulting `Add` has a type that's wider than that of the
>   initial `CastII`).
>   
> There are already 3 types of `Cast` nodes depending on the
> optimizations that are allowed. Either the `Cast` is floating
> (`depends_only_test()` returns `true`) or pinned. Either the `Cast`
> can be removed if it no longer narrows the type of its input or
> not. We already have variants of the `CastII`:
> 
> - if the Cast can float and be removed when it doesn't narrow the type
> of its input.
> 
> - if the Cast is pinned and be removed when it doesn't narrow the type
> of its input.
> 
> - if the Cast is pinned and can't be removed when it doesn't narrow
> the type of its input.
> 
> What we need here, I think, is the 4th combination:
> 
> - if the Cast can float and can't be removed when it doesn't narrow
> the type of its input.
> 
> Anyway, things are becoming confusing with all these different
> variants named in ways that don't always help figure out what
> constraints one of them operate under. So I refactored this and that's
> the biggest part of this change. The fix consists in marking `Cast`
> nodes when their type is widen in a way that prevents them from being
> optimized out.
> 
> Tobias ran performance testing with a slightly different version of
> this change and there was no regression.

If a `CastII` that does not narrow its input has its type being a constant, do you think GVN should transform it into a constant, or such nodes should return the bottom type so that it is not folded into a floating `ConNode`?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24575#issuecomment-2801452694