RFR: 8354282: C2: more crashes in compiled code because of dependency on removed range check CastIIs

[Roland Westrelin]

On Mon, 14 Apr 2025 11:50:27 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> This is a variant of 8332827. In 8332827, an array access becomes
>> dependent on a range check `CastII` for another array access. When,
>> after loop opts are over, that RC `CastII` was removed, the array
>> access could float and an out of bound access happened. With the fix
>> for 8332827, RC `CastII`s are no longer removed.
>> 
>> With this one what happens is that some transformations applied after
>> loop opts are over widen the type of the RC `CastII`. As a result, the
>> type of the RC `CastII` is no longer narrower than that of its input,
>> the `CastII` is removed and the dependency is lost.
>> 
>> There are 2 transformations that cause this to happen:
>> 
>> - after loop opts are over, the type of the `CastII` nodes are widen
>>   so nodes that have the same inputs but a slightly different type can
>>   common.
>>   
>> - When pushing a `CastII` through an `Add`, if of the type both inputs
>>   of the `Add`s are non constant, then we end up widening the type
>>   (the resulting `Add` has a type that's wider than that of the
>>   initial `CastII`).
>>   
>> There are already 3 types of `Cast` nodes depending on the
>> optimizations that are allowed. Either the `Cast` is floating
>> (`depends_only_test()` returns `true`) or pinned. Either the `Cast`
>> can be removed if it no longer narrows the type of its input or
>> not. We already have variants of the `CastII`:
>> 
>> - if the Cast can float and be removed when it doesn't narrow the type
>> of its input.
>> 
>> - if the Cast is pinned and be removed when it doesn't narrow the type
>> of its input.
>> 
>> - if the Cast is pinned and can't be removed when it doesn't narrow
>> the type of its input.
>> 
>> What we need here, I think, is the 4th combination:
>> 
>> - if the Cast can float and can't be removed when it doesn't narrow
>> the type of its input.
>> 
>> Anyway, things are becoming confusing with all these different
>> variants named in ways that don't always help figure out what
>> constraints one of them operate under. So I refactored this and that's
>> the biggest part of this change. The fix consists in marking `Cast`
>> nodes when their type is widen in a way that prevents them from being
>> optimized out.
>> 
>> Tobias ran performance testing with a slightly different version of
>> this change and there was no regression.
>
> If a `CastII` that does not narrow its input has its type being a constant, do you think GVN should transform it into a constant, or such nodes should return the bottom type so that it is not folded into a floating `ConNode`?

@merykitty @eme64 @chhagedorn thanks for the reviews
Does testing need to be run on this before I integrate?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24575#issuecomment-3627292197