RFR: 8372451: C2 SuperWord: "endless loop" assert. Need to implement proper worklist mechanism [v2]

[Christian Hagedorn]

On Thu, 27 Nov 2025 15:42:25 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Context**: `VTransform::optimize`. Works a bit like IGVN, it allows each node to perform optimizations. Recently introduced during JDK26.
>> 
>> **Problem**: I made the assumption that we don't need a worklist mechanism, we can just do multiple passes over all nodes. The assumption was that there would not be any "trickling" of updates over the graph. But that is wrong: for example we can have a long chain of dead nodes, and we need to progressively remove the last node and mark it as dead.
>> 
>> **Solution**: Implement proper worklist mechanism, so that updates can trickle over the graph.
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   limit steps of optimize, for Manuel

Looks good to me, too!

src/hotspot/share/opto/vtransform.cpp line 45:

> 43: void VTransformOptimize::worklist_push(VTransformNode* vtn) {
> 44:   if (_worklist_set.test_set(vtn->_idx)) { return; }
> 45:   _worklist.push(vtn);

I would flip this since it's only one line:

Suggestion:

  if (!_worklist_set.test_set(vtn->_idx)) {
    _worklist.push(vtn);
  }

test/hotspot/jtreg/compiler/loopopts/superword/TestLongReductionChain.java line 38:

> 36:  *                   -XX:CompileCommand=compileonly,${test.main.class}::test
> 37:  *                   ${test.main.class}
> 38:  * @run driver ${test.main.class}

Suggestion:

 * @run main ${test.main.class}

-------------

PR Review: https://git.openjdk.org/jdk/pull/28512#pullrequestreview-3528694733
PR Review Comment: https://git.openjdk.org/jdk/pull/28512#discussion_r2580027890
PR Review Comment: https://git.openjdk.org/jdk/pull/28512#discussion_r2580040060