RFR: 8372634: C2: Materialize type information from instanceof checks

[Roland Westrelin]

On Thu, 27 Nov 2025 00:53:54 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Even though `instanceof` check (and reflective `Class.isInstance` call) narrows operand's type, sharpened type information is not explicitly materialized in the IR.
> 
> There's a `SubTypeCheck` node present, but it is not a substitute for a `CheckCastPP` node with a proper type.
> 
> The difference can be illustrated with the following simple cases:
> 
>    class A { void m() {} }
>    class B extends A { void m() {} }
> 
>     void testInstanceOf(A obj) {
>         if (obj instanceof B) {
>             obj.m();
>         }
>     }
> 
> InstanceOf::testInstanceOf (12 bytes)
>   @ 8 InstanceOf$A::m (0 bytes) failed to inline: virtual call
> 
> vs
> 
>     void testInstanceOfCast(A obj) {
>         if (obj instanceof B) {
>             B b = (B)obj;
>             b.m();
>         }
>     }
> 
> InstanceOf::testInstanceOfCast (17 bytes)
>   @ 13 InstanceOf$B::m (1 bytes) inline (hot)
> 
> 
> Proposed fix annotates operands of subtype checks with proper type information which reflects the effects of subtype check. Not-yet-canonicalized IR shape poses some challenges, but I decided to match it early so information is available right away, rather than waiting for IGVN pass and delay inlining to post-parse phase.
> 
> FTR it is not a complete fix. It works for trivial cases, but for more complex conditions the IR shape becomes too complex during parsing (as illustrated by some test cases). I experimented with annotating subtype checks after initial parsing pass is over, but the crucial simplification step happens as part of split-if transformation which happens when no more inlining is possible. So, the only possible benefit (without forcing split-if optimization earlier) is virtual-to-direct call strength reduction. I plan to explore it separately.
> 
> Testing: hs-tier1 - hs-tier5

I tried the patch and the test case out of curiosity but when I removed the change to `Parse::sharpen_type_after_if()`, the test still passed. I made the change below and it now fails without the change to `Parse::sharpen_type_after_if()` and passes with it. 


diff --git a/test/hotspot/jtreg/compiler/inlining/TestSubtypeCheckTypeInfo.java b/test/hotspot/jtreg/compiler/inlining/TestSubtypeCheckTypeInfo.java
index cc292dc0900..bdb79966a2a 100644
--- a/test/hotspot/jtreg/compiler/inlining/TestSubtypeCheckTypeInfo.java
+++ b/test/hotspot/jtreg/compiler/inlining/TestSubtypeCheckTypeInfo.java
@@ -346,13 +346,14 @@ static boolean lateInlineIsInstanceCondPost(A o, boolean cond) {
     // Parse compilation log (-XX:+PrintCompilation -XX:+PrintInlining output).
     static boolean parseOutput(List<String> output) {
         boolean result = true;
-        Pattern compilation = Pattern.compile("^\\d+\\s+\\d+\\s+b\\s+");
+        Pattern compilation = Pattern.compile("^\\d+\\s+\\d+\\s+b\\s+.*");
         StringBuilder inlineTree = new StringBuilder();
         for (String line : output) {
             // Detect start of next compilation.
             if (compilation.matcher(line).matches()) {
                 // Parse output for previous compilation.
                 result &= validateInliningOutput(inlineTree.toString());
+                inlineTree.setLength(0);
             }
             inlineTree.append(line);
         }

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28517#issuecomment-3585853817