RFR: 8324345: Stack overflow during C2 compilation when splitting memory phi

Daniel Lundén dlunden at openjdk.org
Thu Jul 18 15:29:49 UTC 2024


`ConnectionGraph::find_inst_mem` contains recursive calls that can lead to a native C++ stack overflow in some cases.

### Changeset

- Add a simple recursion depth limit to `ConnectionGraph::find_inst_mem`. When `find_inst_mem` reaches the limit, we bail out from the current compilation and retry without escape analysis. The depth limit is currently set to a fixed value of 1000, based on ad hoc experiments. We want a small enough limit to ensure we never get a stack overflow, but still high enough to allow completing the escape analysis in most compilations. I'm open to other suggestions for setting the limit (perhaps adaptively based on the execution environment?).
- Add a regression test `TestFindInstMemRecursion.java`.
- Temporarily remove the implicit memory limit (due to [JDK-8331185](https://bugs.openjdk.org/browse/JDK-8331185)) from `VectorReplicateLongSpecialImmTest.java`. The memory limit triggers after this changeset, but the (potential) problem is most likely unrelated to this issue. I will create a new issue after merging this PR which we can look into separately.

### Testing

- [GitHub Actions](https://github.com/dlunde/jdk/actions/runs/9976490176)
- tier1 to tier4 on Windows x64, Linux x64, Linux aarch64, macOS x64, and macOS aarch64.
- Standard performance benchmarking. I found no regressions with the proposed depth limit of 1000.

-------------

Commit messages:
 - Remove MemLimit from VectorReplicateLongSpecialImmTest
 - Update copyright
 - Add recursion depth limit and regression test

Changes: https://git.openjdk.org/jdk/pull/20238/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=20238&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8324345
  Stats: 62 lines in 4 files changed: 53 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/20238.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/20238/head:pull/20238

PR: https://git.openjdk.org/jdk/pull/20238


More information about the hotspot-compiler-dev mailing list