RFR: 8369654: javac OutOfMemoryError for complex intersection type [v2]

[Jan Lahoda]

On Wed, 29 Oct 2025 22:34:58 GMT, Vicente Romero <vromero at openjdk.org> wrote:

>> Javac is throwing an OOME for the code like:
>> 
>> class Test {
>>     interface WithMixin01<T> {}
>>     // ...
>>     // 20 interfaces here
>>     interface WithMixin20<T> {}
>> 
>>     interface ClientA extends
>>             WithMixin02<ClientA>,
>>             WithMixin01<ClientA>,
>>             WithMixin03<ClientA>,
>>             WithMixin04<ClientA>,
>>             WithMixin05<ClientA>,
>>             WithMixin06<ClientA>,
>>             WithMixin07<ClientA>,
>>             WithMixin08<ClientA>,
>>             WithMixin09<ClientA>,
>>             WithMixin10<ClientA>,
>>             WithMixin11<ClientA>,
>>             WithMixin12<ClientA>,
>>             WithMixin13<ClientA>,
>>             WithMixin14<ClientA>,
>>             WithMixin15<ClientA>,
>>             WithMixin16<ClientA>,
>>             WithMixin17<ClientA>,
>>             WithMixin18<ClientA>,
>>             WithMixin19<ClientA>,
>>             WithMixin20<ClientA> {
>>     }
>> 
>>     interface ClientB extends
>>             WithMixin01<ClientB>,
>>             WithMixin02<ClientB>,
>>             WithMixin03<ClientB>,
>>             WithMixin04<ClientB>,
>>             WithMixin05<ClientB>,
>>             WithMixin06<ClientB>,
>>             WithMixin07<ClientB>,
>>             WithMixin08<ClientB>,
>>             WithMixin09<ClientB>,
>>             WithMixin10<ClientB>,
>>             WithMixin11<ClientB>,
>>             WithMixin12<ClientB>,
>>             WithMixin13<ClientB>,
>>             WithMixin14<ClientB>,
>>             WithMixin15<ClientB>,
>>             WithMixin16<ClientB>,
>>             WithMixin17<ClientB>,
>>             WithMixin18<ClientB>,
>>             WithMixin19<ClientB>,
>>             WithMixin20<ClientB> {
>>     }
>> 
>>     public static void main(String... args) {
>>         ClientA a = null;
>>         ClientB b = null;
>>         String selector = "a";
>>         Object o = switch (selector) {
>>             case "a" -> a;
>>             case "b" -> b;
>>             default -> null;
>>         };
>>     }
>> }
>> 
>> the reason is that after [JDK-8353565](https://bugs.openjdk.org/browse/JDK-8353565) we are using Types::lub when joining jump chains in Code. The result of the lub will be erased anyways so the proposal here is to cut in the result given by lub types that will be erased anyways. The reason for the slow lub invocation here is that in order to obtain the lub in this case, the lub method is invoked several times. Each invocation, minus the first one, with very complex types which makes them very ...
>
> Vicente Romero has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor changes

I am not the expert on types, and leveraging the fact that we will erase the type later is clever.

But, when I was trying to debug this, it seemed the reason is that we are repeatedly and recursively asking `lub(ClientB, ClientA)`, from different paths (e.g. from `WithMixin02<ClientA>, WithMixin01<ClientB>`, which will then ask the same question based on different mixins one level deeper in the recursion). Caching the results helps in this case[1], and I wonder if that could be a viable solution (i.e. is this mostly about the recursion?).

I also tried `var v = java.util.List.of(a, b);` - that takes a long time as well, and without the cache, it takes a long time in `lub`. But the cache is not enough for that case, as with the cache, it still spends a lot of time elsewhere.

[1]

diff --git a/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java b/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
index 0c155caa56d..3da5c0703bd 100644
--- a/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
+++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
@@ -3984,6 +3984,45 @@ public Type lub(List<Type> ts) {
      * does not exist return the type of null (bottom).
      */
     public Type lub(Type... ts) {
+        TypeListKey key = new TypeListKey(this, ts);
+        Type result = lubCache.get(key);
+
+        if (result == null) {
+            lubCache.put(key, result = computeLub(ts));
+        }
+
+        return result;
+    }
+    private record TypeListKey(Types thisTypes, Type... types) {
+        @Override
+        public int hashCode() {
+            int hashCode = 0;
+            for (Type t : types) {
+                hashCode = 127 * hashCode + thisTypes.hashCode(t);
+            }
+            return hashCode;
+        }
+        @Override
+        public boolean equals(Object obj) {
+            if (!(obj instanceof TypeListKey other)) {
+                return false;
+            }
+            if (thisTypes != other.thisTypes) {
+                return false;
+            }
+            if (types.length != other.types.length) {
+                return false;
+            }
+            for (int i = 0; i < types.length; i++) {
+                if (!thisTypes.isSameType(types[i], other.types[i])) {
+                    return false;
+                }
+            }
+            return true;
+        }
+    }
+    private Map<TypeListKey, Type> lubCache = new HashMap<>();
+    private Type computeLub(Type... ts) {
         final int UNKNOWN_BOUND = 0;
         final int ARRAY_BOUND = 1;
         final int CLASS_BOUND = 2;

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28050#issuecomment-3467331832