RFR: 8373820: C2: Robust Node::uncast_helper infinite loop check [v2]

[Christian Hagedorn]

On Wed, 17 Dec 2025 10:19:16 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Current check in `Node::uncast_helper` checks for "infinite loop", but really checks for the depth of 1K nodes when searching through the graph:
>> 
>> 
>>     assert(depth_count++ < K, "infinite loop in Node::uncast_helper");
>> 
>> 
>> I suppose it is plausible to have a legit chain of 1K nodes with very deep inlining and/or optimization, which is _not_ an infinite loop. This might be the cause for some CTW failures in deeper stress modes: I have been running CTW stress tests for 12+ hours without ever hitting the new check, and I usually hit the old one within that timeframe in current mainline.
>> 
>> Given how we basically walk through `in(1)`, i.e. moving as if through the linked list of nodes, I think we can check against the number of nodes we have. If we walk more nodes than we have, that would mean we visited some node twice, which necessarily means there is a infinite loop in this walk. This makes the check more robust. 
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, `applications/ctw/modules` in different stress modes, 10x passes
>>  - [x]  Linux x86_64 server fastdebug, `hotspot_compiler`
>>  - [x]  Linux x86_64 server fastdebug, `tier1`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8373820-c2-robust-uncast-helper-check
>  - Fix

Looks good!

-------------

Marked as reviewed by chagedorn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/28861#pullrequestreview-3587252749