RFR: 8362394: C2: Repeated stacked string concatenation fails with "Hit MemLimit" and other resourcing errors

[Galder Zamarreño]

On Fri, 8 Aug 2025 06:10:56 GMT, Daniel Skantz <dskantz at openjdk.org> wrote:

> This PR addresses a bug in the stringopts phase. During string concatenation, repeated stacking of concatenations can lead to excessive compilation resource use and generation of questionable code as the merging of two StringBuilder-append-toString links sc1 and sc2 can result in a new StringBuilder with the size sc1->num_arguments() * sc2->num_arguments().
> 
> In the attached test, the size of the successively merged StringBuilder doubles on each merge -- there's 24 of them -- as the toString result of the first component is used twice in the second component [1], etc. Not only does the compiler hang on this test case, but the string concat optimization seems to give an arbitrary amount of back-to-back stores in the generated code depending on the number of stacked concatenations.
> 
> The proposed solution is to put an upper bound on the size of a merged concatenation, which guards against this case of repeated concatenations on the same string variable, and potentially other edge cases. 100 seems like a generous limit, and higher limits could be insufficient as each argument corresponds to about 20 new nodes later in replace_string_concat [2].
> 
> [1] https://github.com/openjdk/jdk/blob/0ceb366dc26e2e4f6252da9dd8930b016a5d46ba/src/hotspot/share/opto/stringopts.cpp#L303
> 
> [2] https://github.com/openjdk/jdk/blob/0ceb366dc26e2e4f6252da9dd8930b016a5d46ba/src/hotspot/share/opto/stringopts.cpp#L1806
> 
> Testing: T1-4.
> 
> Extra testing: verified that no method in T1-4 is being compiled with a merged concat candidate exceeding the suggested limit of 100 aguments, regardless of whether or not the later checks verify_control_flow() and verify_mem_flow pass.

test/hotspot/jtreg/compiler/stringopts/TestStackedConcatsMany.java line 28:

> 26:  * @bug 8357105
> 27:  * @summary Test that repeated stacked string concatenations do not
> 28:  *          consume too many compilation resources.

Is there a reasonable way to enhance the test to validate excessive resources? I'm not sure if the following example would work, but I'm wondering if there is something that can be measured deterministically. E.g. before with the given test there would be ~N IR nodes produced but now it would be a max of ~M, assuming that M is deterministically smaller than N.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26685#discussion_r2266284078