RFR: 8316533: C2 compilation fails with assert(verify(phase)) failed: missing Value() optimization

[Roberto Castañeda Lozano]

On Wed, 25 Oct 2023 14:29:07 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

> **Problem**
> We have a `abstract` class `A` with no subtype. Hence, a reference of type `A` must always be `null` (unless a subclass were to be loaded, which we guard against with a compile dependency).
> 
> But there are at least these two ways a `A:NotNull` can be created:
> - Null-Check: CastPP after null-check improves type from `A` to `A:NotNull`.
> - Forced compilation (eg CTW) of a member method of `A`. Then `Parm0` has type `A`, which is improved to `A:NotNull` because the `this/self` pointer cannot be `null`.
> 
> This means we are now left with an impossible type `A:NotNull`, a path that uses this type will never be executed.
> 
> The question is now what should happen at a `SubTypeCheck` if we do:
> `SubTypeCheck(  oop #A:NotNull   ,   constant-classptr-A-exact )`
> 
> The verification happens because we do these two different things:
> - `SubTypeCheck`: we first detect that we have a constant classptr of a class `A`, which is abstract and has no subtype. Hence, we conclude that any oop compared to it cannot be a subtype (there are no subtypes), and it cannot be of the same type (class is abstract). Hence, any oop must be a supertype (TypeInt::CC_GT).
> - The verification code computes the subtype check by computing the klass of the oop via `LoadKlass` (this constant folds to `constant-classptr-A-exact`, because the type of the oop is `A:NotNull`). The `CmpP` node compares the two klasses, and sees that they are identical, and returns an `TypeInt::CC_EQ`.
> 
> **Alternatives**
> 
> Both results are reasonable, but they are in fact both supersets of the true result. We should take the intersection of the two and get `Type:TOP`, since the input type is already impossible. In fact, it would be best if the impossible type was never created. We could do that by improving `CmpP` to detect the impossible type and constant fold towards the `null` path, removing the `A:NotNull` path. It is harder to deal with the forced-compilation of non-static methods of an abstract class with no subclasses - here we would basically have to forbid compilation or replace the compilation with a `Halt`.
> 
> **Solution**
> 
> Instead, I have now decided to change the logic in `SubTypeCheckNode` to return `EQ` in case the oop has the same klass and is `NotNull`.
> 
> **Testing**
> 
> Tier1-6 and stress testing. Running...

Looks good, I just have a few minor comments and style suggestions.

src/hotspot/share/opto/subtypenode.cpp line 51:

> 49:         // situation since super_t is also abstract, and hence sub_t cannot have the
> 50:         // same type and be non-null.
> 51:         // Still, if the non-static method of a abstract class without subclasses is

Suggestion:

        // Still, if the non-static method of an abstract class without subclasses is

src/hotspot/share/opto/subtypenode.cpp line 60:

> 58:         // subtype.
> 59:         return TypeInt::CC_GT;
> 60:       }

Style suggestion:
Suggestion:

      }
      // subk is either a supertype of superk, or null. In either case, superk is a subtype.
      return TypeInt::CC_GT;

-------------

Marked as reviewed by rcastanedalo (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16361#pullrequestreview-1710565752
PR Review Comment: https://git.openjdk.org/jdk/pull/16361#discussion_r1380314236
PR Review Comment: https://git.openjdk.org/jdk/pull/16361#discussion_r1380324238