RFR: 8353565: Javac throws "inconsistent stack types at join point" exception

[Jan Lahoda]

Consider code like:

public class T {
    private Object computeTypeAtMergePoint1(int i) {
        return (Object) switch (i) {
            case 0 -> switch (i) {
                case 0 -> { yield new A(); }
                default -> { yield new B(); }
            };
            default -> switch (i) {
                case 0 -> { yield new C(); }
                default -> { yield new D(); }
            };
        };
    }
    enum E {A, B}
    interface I1 {}
    class A implements I1 {}
    class B implements I1 {}
    interface I2 {}
    class C implements I2 {}
    class D implements I2 {}

}


Compiling this leads to a crash:

$ .../jdk-24/bin/javac  /tmp/T.java 
An exception has occurred in the compiler (24-internal). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.
java.lang.AssertionError: inconsistent stack types at join point
        at jdk.compiler/com.sun.tools.javac.jvm.Code$State.error(Code.java:1824)
        at jdk.compiler/com.sun.tools.javac.jvm.Code$State.join(Code.java:1814)
        at jdk.compiler/com.sun.tools.javac.jvm.Code.resolve(Code.java:1525)
        at jdk.compiler/com.sun.tools.javac.jvm.Code.resolvePending(Code.java:1556)
        at jdk.compiler/com.sun.tools.javac.jvm.Code.emitop(Code.java:382)
        at jdk.compiler/com.sun.tools.javac.jvm.Code.emitop0(Code.java:511)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.visitReturn(Gen.java:1905)
        at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCReturn.accept(JCTree.java:1773)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genDef(Gen.java:588)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genStat(Gen.java:623)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genStat(Gen.java:609)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genStats(Gen.java:660)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.internalVisitBlock(Gen.java:1121)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.visitBlock(Gen.java:1085)
        at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1137)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genDef(Gen.java:588)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genStat(Gen.java:623)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genMethod(Gen.java:949)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.visitMethodDef(Gen.java:912)
        at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:961)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genDef(Gen.java:588)
        at jdk.compiler/com.sun.tools.javac.jvm.Gen.genClass(Gen.java:2494)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.genCode(JavaCompiler.java:771)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1710)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1678)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:978)
        at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:319)
        at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:178)
        at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:66)
        at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:52)
printing javac parameters to: /tmp/javac.20250414_101233.args


The reason is that all of the yields "jump" to the same point in the bytecode, and javac is unable to compute the top-of-stack type for that point. javac expects that there's subtyping relation for the types on the join point.

In a little bit more detail:
- the `(Object)` cast is important, as that makes the switch expressions standalone expressions, whose types are inferred from the content, not from the target type
-if we had only:

    private Object computeTypeAtMergePoint1(int i) {
        return (Object) switch (i) {
            case 0 -> { yield new A(); }
            default -> { yield new D(); }
        };
    }


then this would compile, as the type of the switch expression would be `Object`, and inside `visitYield`, we use `forceStackTop` to change the top-stack type to the type of the switch expression. This works on one level, but does not work well for nested level switch expressions - while the outer switch expression in the original example has type `Object`, the nested ones have `I1` and `I2`, respectively, and the nested `yield` statements set `I1` or `I2` as the stack-top type. And neither of `I1` or `I2` is a subtype of the other, leading to the failure.

The proposal here is to use `Types.lub` to compute the common supertype of the types that join. The subtyping checks are kept, to help with some corner cases, in particular with `null`/`BOT` type handling. And also to limit potential performance impact.

I also think the `forceStackTop` is more a workaround that permitted the use of simply subtyping rather than `lub` at the join points. And it only works for one level of nesting for switch expressions. The proposal here is to drop the `forceStackTop` for both switch expressions and conditional expressions, so that if there are some cases where the new code would not work, we would find out faster/easier. But I can keep the `forceStackTop` calls in that is preferred.

(In any case, there's one call to `forceStackTop` in `Gen.visitAssign`. That one seems legitimate to me, so that one is kept.)

-------------

Commit messages:
 - 8353565: Javac throws "inconsistent stack types at join point" exception

Changes: https://git.openjdk.org/jdk/pull/24617/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=24617&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8353565
  Stats: 122 lines in 3 files changed: 112 ins; 3 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/24617.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/24617/head:pull/24617

PR: https://git.openjdk.org/jdk/pull/24617