RFR: 8372451: C2 SuperWord: "endless loop" assert. Need to implement proper worklist mechanism [v3]

[Manuel Hässig]

On Tue, 2 Dec 2025 08:13:47 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Context**: `VTransform::optimize`. Works a bit like IGVN, it allows each node to perform optimizations. Recently introduced during JDK26.
>> 
>> **Problem**: I made the assumption that we don't need a worklist mechanism, we can just do multiple passes over all nodes. The assumption was that there would not be any "trickling" of updates over the graph. But that is wrong: for example we can have a long chain of dead nodes, and we need to progressively remove the last node and mark it as dead.
>> 
>> **Solution**: Implement proper worklist mechanism, so that updates can trickle over the graph.
>
> Emanuel Peter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8372451-too-many-dead-vector-reduction-vtnodes
>  - Apply suggestions from code review
>    
>    Co-authored-by: Christian Hagedorn <christian.hagedorn at oracle.com>
>  - limit steps of optimize, for Manuel
>  - Merge branch 'master' into JDK-8372451-too-many-dead-vector-reduction-vtnodes
>  - rm old documentation
>  - git move to new test
>  - streamline
>  - refactor and verify
>  - unique worklist
>  - wip solution
>  - ... and 1 more: https://git.openjdk.org/jdk/compare/69927660...54881ff4

Marked as reviewed by mhaessig (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/28512#pullrequestreview-3528909043