RFR: 8372634: C2: Materialize type information from instanceof checks [v4]

[Vladimir Ivanov]

On Wed, 3 Dec 2025 10:40:52 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

>> Yes, it's a good idea and the right direction to move. While experimenting with a different enhancement, I noticed that a subtype check leaves a null check behind irrespective of whether the check goes away or not.  
>> 
>> Unfortunately, there are some engineering considerations which complicates the change. `SubTypeCheck` is shared across all the places where subtype checks are performed, but `checkcast` and `instanceof` differ in the way `null` is handled. So, the proper way to fix it is to introduce a higher-level representation which implicitly handles nulls and then eventually lower it to `SubTypeCheck` and materialize null check if needed.
>
> There are multiple ways without having to have yet another higher-level representation. The first one is that since `SubTypeCheck` does not accept `null` now, we can just choose one result for `null`. Choosing the `instanceof` approach may be a little more desirable, as it removes the need to perform this complicated match, and for `checkcast` we can manually insert a `CheckCastPP` anyway. Another solution is to have another input to `SubTypeCheck` which gives the result when the `obj` is `null`. On a whim, I kind of like this, as we can match both the `checkcast` and the `instanceof` pattern here, it also simplifies `GraphKit::gen_checkcast`, as we do not have to worry about "the cast that always succeeds will leave behind a null check".
> 
> Just a suggestion, though. This PR is fine as it is to me.

I agree it can be implemented without introducing new fancy IR nodes. The open question to me though is whether we can live without materializing null check until `SubTypeCheck` nodes are macro expanded. Otherwise, it'll turn into a gradual lowering though different representations.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28517#discussion_r2594317538