RFR: 8372634: C2: Materialize type information from instanceof checks [v2]

[Quan Anh Mai]

On Mon, 1 Dec 2025 19:49:29 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> src/hotspot/share/opto/parse2.cpp line 1739:
>> 
>>> 1737:   }
>>> 1738: 
>>> 1739:   // Match an instanceof check.
>> 
>> We seem to require that the input of `SubTypeCheck` is not `null`. What do you think about allowing `SubTypeCheck` to accept `null` and return `false`?
>
> Yes, it's a good idea and the right direction to move. While experimenting with a different enhancement, I noticed that a subtype check leaves a null check behind irrespective of whether the check goes away or not.  
> 
> Unfortunately, there are some engineering considerations which complicates the change. `SubTypeCheck` is shared across all the places where subtype checks are performed, but `checkcast` and `instanceof` differ in the way `null` is handled. So, the proper way to fix it is to introduce a higher-level representation which implicitly handles nulls and then eventually lower it to `SubTypeCheck` and materialize null check if needed.

There are multiple ways without having to have yet another higher-level representation. The first one is that since `SubTypeCheck` does not accept `null` now, we can just choose one result for `null`. Choosing the `instanceof` approach may be a little more desirable, as it removes the need to perform this complicated match, and for `checkcast` we can manually insert a `CheckCastPP` anyway. Another solution is to have another input to `SubTypeCheck` which gives the result when the `obj` is `null`. On a whim, I kind of like this, as we can match both the `checkcast` and the `instanceof` pattern here, it also simplifies `GraphKit::gen_checkcast`, as we do not have to worry about "the cast that always succeeds will leave behind a null check".

Just a suggestion, though. This PR is fine as it is to me.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28517#discussion_r2584574037