[lworld] RFR: 8282616: [lworld] C2 does not properly handle circular .ref or value class fields

[Tobias Hartmann]

The current C2 implementation recurses infinitely when scalarizing circular .ref or value class fields, leading to a segmentation fault (stack overflow) during compilation. I added code to detect such circularity and limit the scalarization depth accordingly. Unfortunately, we can now observe nodes of the same type but with different scalarization depth during GVN and IGVN. Here's an example:


value class V1 {
    V2 v2 = V2.default;
}

value class V2 {
    V1 v1 = V1.default;
}

V1 a = new V1(); // Scalarized as InlineTypeNodeV1(InlineTypeNodeV2(oop))
V2 b = new V2(); // Scalarized as InlineTypeNodeV2(InlineTypeNodeV1(oop))
V1 c = __ withfield(a, v2, b); // Scalarized as InlineTypeNodeV1(InlineTypeNodeV2(InlineTypeNodeV1(oop)))


Nodes for `a` and `c` are of the same type but scalarized with different depth. That's a problem for `InlineTypeNode::merge_with` which relies on Phi nodes to merge field values at the exact same depth. I added code to adjust the scalarization depth during parsing/GVN and limited the creation of Phi nodes during IGVN. All of that is dependent on the detection of circularity. I filed [JDK-8302308](https://bugs.openjdk.org/browse/JDK-8302308) to address some limitations of the current implementation in a low-priority, follow-up enhancement.

Thanks,
Tobias

-------------

Commit messages:
 - Removed useless check
 - Added missing check
 - Removed extra asserts
 - Removed code added for testing
 - Improved detection of circularity
 - Some more fixes/cleanups
 - First prototype

Changes: https://git.openjdk.org/valhalla/pull/824/files
 Webrev: https://webrevs.openjdk.org/?repo=valhalla&pr=824&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8282616
  Stats: 364 lines in 14 files changed: 288 ins; 15 del; 61 mod
  Patch: https://git.openjdk.org/valhalla/pull/824.diff
  Fetch: git fetch https://git.openjdk.org/valhalla pull/824/head:pull/824

PR: https://git.openjdk.org/valhalla/pull/824