RFR: 8324345: Stack overflow during C2 compilation when splitting memory phi [v2]

[duke]

On Fri, 19 Jul 2024 12:27:04 GMT, Daniel Lundén <dlunden at openjdk.org> wrote:

>> `ConnectionGraph::find_inst_mem` contains recursive calls that can lead to a native C++ stack overflow in some cases.
>> 
>> ### Changeset
>> 
>> - Add a simple recursion depth limit to `ConnectionGraph::find_inst_mem`. When `find_inst_mem` reaches the limit, we bail out from the current compilation and retry without escape analysis. The depth limit is currently set to a fixed value of 1000, based on ad hoc experiments. We want a small enough limit to ensure we never get a stack overflow, but still high enough to allow completing the escape analysis in most compilations. I'm open to other suggestions for setting the limit (perhaps adaptively based on the execution environment?).
>> - Add a regression test `TestFindInstMemRecursion.java`.
>> - Temporarily remove the implicit memory limit (due to [JDK-8331185](https://bugs.openjdk.org/browse/JDK-8331185)) from `VectorReplicateLongSpecialImmTest.java`. The memory limit triggers after this changeset, but the (potential) problem is most likely unrelated to this issue. I will create a new issue after merging this PR which we can look into separately.
>> 
>> ### Testing
>> 
>> - [GitHub Actions](https://github.com/dlunde/jdk/actions/runs/9976490176)
>> - tier1 to tier4 on Windows x64, Linux x64, Linux aarch64, macOS x64, and macOS aarch64.
>> - Standard performance benchmarking. I found no regressions with the proposed depth limit of 1000.
>
> Daniel Lundén has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Limit test to only compile problematic method

@dlunde 
Your change (at version 4463ba7a82aa8973a44d0a99550b4c5f0c69935d) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20238#issuecomment-2259885230