RFR: 8372634: C2: Materialize type information from instanceof checks [v5]

[Emanuel Peter]

On Tue, 9 Dec 2025 01:25:40 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> Even though `instanceof` check (and reflective `Class.isInstance` call) narrows operand's type, sharpened type information is not explicitly materialized in the IR.
>> 
>> There's a `SubTypeCheck` node present, but it is not a substitute for a `CheckCastPP` node with a proper type.
>> 
>> The difference can be illustrated with the following simple cases:
>> 
>>    class A { void m() {} }
>>    class B extends A { void m() {} }
>> 
>>     void testInstanceOf(A obj) {
>>         if (obj instanceof B) {
>>             obj.m();
>>         }
>>     }
>> 
>> InstanceOf::testInstanceOf (12 bytes)
>>   @ 8 InstanceOf$A::m (0 bytes) failed to inline: virtual call
>> 
>> vs
>> 
>>     void testInstanceOfCast(A obj) {
>>         if (obj instanceof B) {
>>             B b = (B)obj;
>>             b.m();
>>         }
>>     }
>> 
>> InstanceOf::testInstanceOfCast (17 bytes)
>>   @ 13 InstanceOf$B::m (1 bytes) inline (hot)
>> 
>> 
>> Proposed fix annotates operands of subtype checks with proper type information which reflects the effects of subtype check. Not-yet-canonicalized IR shape poses some challenges, but I decided to match it early so information is available right away, rather than waiting for IGVN pass and delay inlining to post-parse phase.
>> 
>> FTR it is not a complete fix. It works for trivial cases, but for more complex conditions the IR shape becomes too complex during parsing (as illustrated by some test cases). I experimented with annotating subtype checks after initial parsing pass is over, but the crucial simplification step happens as part of split-if transformation which happens when no more inlining is possible. So, the only possible benefit (without forcing split-if optimization earlier) is virtual-to-direct call strength reduction. I plan to explore it separately.
>> 
>> Testing: hs-tier1 - hs-tier5
>
> Vladimir Ivanov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Revert Compile::should_delay_inlining unification

test/hotspot/jtreg/compiler/inlining/TestSubtypeCheckTypeInfo.java line 363:

> 361:     // Sample:
> 362:     //    213   42    b        compiler.inlining.TestSubtypeCheckTypeInfo::testIsInstanceCondLatePost (13 bytes)
> 363:     static final Pattern TEST_CASE = Pattern.compile("^\\d+\\s+\\d+\\s+b\\s+" + TEST_CLASS_NAME + "::(\\w+) .*");

Drive by comment, no need to change things here now:
@iwanowww @chhagedorn Would it not be nice if we could do this kind of matching with the `TestFramework`? Instead of `IR` matching, just match the output of any compilation tracing / printing.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28517#discussion_r2601929785