Integrated: 8324345: Stack overflow during C2 compilation when splitting memory phi

[Daniel Lundén]

On Thu, 18 Jul 2024 15:24:33 GMT, Daniel Lundén <dlunden at openjdk.org> wrote:

> `ConnectionGraph::find_inst_mem` contains recursive calls that can lead to a native C++ stack overflow in some cases.
> 
> ### Changeset
> 
> - Add a simple recursion depth limit to `ConnectionGraph::find_inst_mem`. When `find_inst_mem` reaches the limit, we bail out from the current compilation and retry without escape analysis. The depth limit is currently set to a fixed value of 1000, based on ad hoc experiments. We want a small enough limit to ensure we never get a stack overflow, but still high enough to allow completing the escape analysis in most compilations. I'm open to other suggestions for setting the limit (perhaps adaptively based on the execution environment?).
> - Add a regression test `TestFindInstMemRecursion.java`.
> - Temporarily remove the implicit memory limit (due to [JDK-8331185](https://bugs.openjdk.org/browse/JDK-8331185)) from `VectorReplicateLongSpecialImmTest.java`. The memory limit triggers after this changeset, but the (potential) problem is most likely unrelated to this issue. I will create a new issue after merging this PR which we can look into separately.
> 
> ### Testing
> 
> - [GitHub Actions](https://github.com/dlunde/jdk/actions/runs/9976490176)
> - tier1 to tier4 on Windows x64, Linux x64, Linux aarch64, macOS x64, and macOS aarch64.
> - Standard performance benchmarking. I found no regressions with the proposed depth limit of 1000.

This pull request has now been integrated.

Changeset: fdb4350f
Author:    Daniel Lundén <dlunden at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/fdb4350fcecef1915cdbc27ece24153a1b6c884d
Stats:     63 lines in 4 files changed: 54 ins; 0 del; 9 mod

8324345: Stack overflow during C2 compilation when splitting memory phi

Reviewed-by: thartmann, kvn

-------------

PR: https://git.openjdk.org/jdk/pull/20238