RFR: 8260650: test failed with "assert(false) failed: infinite loop in PhaseIterGVN::optimize" [v3]

[Igor Veresov]

> The test constructs a large tree of `AddINode`s that add constants to a variable. The tree is completely constant-foldable. Depending of the processing order the folding process may happen in the loop in `PhaseIterGVN::transform_old()` instead of going through the worklist. That blows through the loop limit, which triggers assert. The solution is to make the loop limit proportional to the number of live nodes. I chose it to be `K*live_nodes()`, which is already the limit in `PhaseIterGVN::optimize()`. I also noticed the same problem in `PhaseGVN::transform_no_reclaim()` and adjusted the code accordingly.

Igor Veresov has updated the pull request incrementally with one additional commit since the last revision:

  Address Vladimir's comments.

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3022/files
  - new: https://git.openjdk.java.net/jdk/pull/3022/files/ae9902c7..0efb4776

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3022&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3022&range=01-02

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3022.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3022/head:pull/3022

PR: https://git.openjdk.java.net/jdk/pull/3022