RFR: 8337142: StackOverflowError in Types.containsTypeRecursive with deeply nested type hierarchy

[Vicente Romero]

javac can crash with SOE for some deeply nested type hierarchies. See the related JBS issue for the code example. Here we have another example of infinite recursion in the compiler. This is what is happening:

- method Types."isSubtype"::containsTypeRecursive uses a cache, which is a Set, the elements in that cache are of type Types.TypePair. Method TypePair::equals invoke Types::isSameType
- Types::isSameType for this test case ends up invoking Types."isSubtype"::containsTypeRecursive, closing the infinite loop

The proposed solution is to define current anonymous class isSameTypeVisitor as a named class and derive from it a visitor that will be the one used by Types.TypePair and thus invoked by TypePair::equals every time instances of TypePair are added or removed from a Map, Set, etc.

This new visitor will compare types looking for exactness which is something desirable in a map key.

TIA

-------------

Commit messages:
 - 8337142: StackOverflowError in Types.containsTypeRecursive with deeply nested type hierarchy

Changes: https://git.openjdk.org/jdk/pull/28101/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=28101&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8337142
  Stats: 96 lines in 2 files changed: 90 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/28101.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/28101/head:pull/28101

PR: https://git.openjdk.org/jdk/pull/28101