RFR: 8370519: C2: Hit MemLimit when running with +VerifyLoopOptimizations [v3]

[Emanuel Peter]

On Tue, 2 Dec 2025 11:21:05 GMT, Roland Westrelin <roland at openjdk.org> wrote:

>> For this failure memory stats are:
>> 
>> 
>> Total Usage: 1095525816 
>>     --- Arena Usage by Arena Type and compilation phase, at arena usage peak of 1095525816 ---
>>         Phase                         Total        ra      node      comp      type    states   reglive  regsplit   regmask superword     cienv        ha     other
>>         none                        5976032    331560   5402064    197512     33712     10200         0         0       984         0         0         0         0
>>         parse                       2716464     65456   1145480    196408   1112752         0         0         0         0         0    196368         0         0
>>         optimizer                     98184         0     32728         0     65456         0         0         0         0         0         0         0         0
>>         connectionGraph               32728         0         0     32728         0         0         0         0         0         0         0         0         0
>>         iterGVN                       32728         0     32728         0         0         0         0         0         0         0         0         0         0
>>         idealLoop                 918189632         0  38687056 872824784    392776         0         0         0         0         0   6285016         0         0
>>         idealLoopVerify             2228144         0         0   2228144         0         0         0         0         0         0         0         0         0
>>         macroExpand                   32728         0     32728         0         0         0         0         0         0         0         0         0         0
>>         graphReshape                  32728         0     32728         0         0         0         0         0         0         0         0         0         0
>>         matcher                    20135944   3369848   9033208   7536400     65456    131032         0         0         0         0         0         0         0
>>         postselect_cleanup           294872    294872         0         0         0         0         0         0         0         0         0         0         0
>>         scheduler                    752944    196488    556456         0         0         0         0         0         0         0         0         0         0
>>         regalloc                     388736    388736         0         0         0         0         0         0         0         0         0         0         0
>>        ...
>
> Roland Westrelin has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - review
>  - review

Changes requested by epeter (Reviewer).

src/hotspot/share/opto/loopnode.hpp line 672:

> 670:   bool  _allow_optimizations;   // Allow loop optimizations
> 671: 
> 672:   IdealLoopTree( PhaseIdealLoop* phase, Node *head, Node *tail );

Suggestion:

  IdealLoopTree(PhaseIdealLoop* phase, Node* head, Node* tail);

src/hotspot/share/opto/loopnode.hpp line 1217:

> 1215:     PhaseTransform(Ideal_Loop),
> 1216:     _arena(mtCompiler, Arena::Tag::tag_idealloop),
> 1217:     _loop_or_ctrl(&_arena),

How about some of the other data structures? For example `_idom`?

test/hotspot/jtreg/compiler/c2/TestVerifyLoopOptimizationsHighMemUsage.java line 32:

> 30:  *                   -XX:+UnlockDiagnosticVMOptions -XX:+StressLoopPeeling -XX:+VerifyLoopOptimizations
> 31:  *                   -XX:StressSeed=3106998670 TestVerifyLoopOptimizationsHighMemUsage
> 32:  * @run main TestVerifyLoopOptimizationsHighMemUsage

Suggestion:

 * @run main ${test.main.class}

Also: your test does not have a `package` declaration.

-------------

PR Review: https://git.openjdk.org/jdk/pull/28581#pullrequestreview-3562480981
PR Review Comment: https://git.openjdk.org/jdk/pull/28581#discussion_r2606610438
PR Review Comment: https://git.openjdk.org/jdk/pull/28581#discussion_r2606618710
PR Review Comment: https://git.openjdk.org/jdk/pull/28581#discussion_r2606620987