RFR: 8354282: C2: more crashes in compiled code because of dependency on removed range check CastIIs [v10]

[Christian Hagedorn]

On Fri, 5 Dec 2025 14:05:06 GMT, Roland Westrelin <roland at openjdk.org> wrote:

>> This is a variant of 8332827. In 8332827, an array access becomes
>> dependent on a range check `CastII` for another array access. When,
>> after loop opts are over, that RC `CastII` was removed, the array
>> access could float and an out of bound access happened. With the fix
>> for 8332827, RC `CastII`s are no longer removed.
>> 
>> With this one what happens is that some transformations applied after
>> loop opts are over widen the type of the RC `CastII`. As a result, the
>> type of the RC `CastII` is no longer narrower than that of its input,
>> the `CastII` is removed and the dependency is lost.
>> 
>> There are 2 transformations that cause this to happen:
>> 
>> - after loop opts are over, the type of the `CastII` nodes are widen
>>   so nodes that have the same inputs but a slightly different type can
>>   common.
>>   
>> - When pushing a `CastII` through an `Add`, if of the type both inputs
>>   of the `Add`s are non constant, then we end up widening the type
>>   (the resulting `Add` has a type that's wider than that of the
>>   initial `CastII`).
>>   
>> There are already 3 types of `Cast` nodes depending on the
>> optimizations that are allowed. Either the `Cast` is floating
>> (`depends_only_test()` returns `true`) or pinned. Either the `Cast`
>> can be removed if it no longer narrows the type of its input or
>> not. We already have variants of the `CastII`:
>> 
>> - if the Cast can float and be removed when it doesn't narrow the type
>> of its input.
>> 
>> - if the Cast is pinned and be removed when it doesn't narrow the type
>> of its input.
>> 
>> - if the Cast is pinned and can't be removed when it doesn't narrow
>> the type of its input.
>> 
>> What we need here, I think, is the 4th combination:
>> 
>> - if the Cast can float and can't be removed when it doesn't narrow
>> the type of its input.
>> 
>> Anyway, things are becoming confusing with all these different
>> variants named in ways that don't always help figure out what
>> constraints one of them operate under. So I refactored this and that's
>> the biggest part of this change. The fix consists in marking `Cast`
>> nodes when their type is widen in a way that prevents them from being
>> optimized out.
>> 
>> Tobias ran performance testing with a slightly different version of
>> this change and there was no regression.
>
> Roland Westrelin has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review

I had a look and it seems that the internal test is relying on a `CastII` node to be removed after loop opts, when we widen `CastII` nodes, to trigger an ideal optimization. That is no longer the case with this patch because we keep the `CastII` node in the graph. The fix would be to improve the ideal optimization to look through cast nodes. However, this feels out of scope, especially since this PR is a bug fix for JDK 26.

I therefore propose to fix the internal test before integrating this PR and then follow up with an RFE to fix the ideal optimization. I can take care of this and let you know once this is done.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24575#issuecomment-3632449257