RFR: 8372451: C2 SuperWord: "endless loop" assert. Need to implement proper worklist mechanism

[Manuel Hässig]

On Wed, 26 Nov 2025 16:02:20 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

> **Context**: `VTransform::optimize`. Works a bit like IGVN, it allows each node to perform optimizations. Recently introduced during JDK26.
> 
> **Problem**: I made the assumption that we don't need a worklist mechanism, we can just do multiple passes over all nodes. The assumption was that there would not be any "trickling" of updates over the graph. But that is wrong: for example we can have a long chain of dead nodes, and we need to progressively remove the last node and mark it as dead.
> 
> **Solution**: Implement proper worklist mechanism, so that updates can trickle over the graph.

Thank you for fixing this, @eme64!

The optimization mechanism you are replacing had a mechanism to detect when too many passes had been done. Now there is no such mechanism. How can you detect a condition where we take way too long to reach a fixpoint apart from a timeout?

What do implementations of `VTransform::optimize` have to guarantee to ensure that we reach a fixpoint?

-------------

PR Review: https://git.openjdk.org/jdk/pull/28512#pullrequestreview-3515641101