RFR: 8290711: assert(false) failed: infinite loop in PhaseIterGVN::optimize [v3]

[Roland Westrelin]

> In the test case: the loop is an infinite loop but loop optimizations
> don't add a NeverBranch node because the loop is reachable from Root
> (following Root's inputs) through a null check in the loop body. When
> CCP runs, it finds the null check never fails. PhaseCCP::transform()
> works through the graph starting from Root and following its inputs.
> But because the null check never fails, it doesn't follow the path to
> the loop body and it doesn't update the type of nodes in the loop body
> (as done in PhaseCCP::transform_once()). That's wrong as the loop body
> is not dead. As a consequence, for one CastPP node, its bottom type
> and the type recorded in PhaseCCP's type table differ. When igvn runs
> next, for some nodes, MemNode::Ideal_common() sees a difference
> between the address type recorded by igvn and the one reported by
> bottom_type(). That causes memory nodes to be indefinitely
> re-enqueued for igvn.
> 
> The fix I propose is similar to one Tobias implemented in the valhalla
> repo (JDK-8265973). With this fix, all loops are always considered
> reachable by PhaseCCP::transform() (which I think, worst case, is only
> a waste of compilation time but has no correctness issue). To achieve
> that safepoints are collected by CCP before PhaseCCP::transform()
> transform() follows inputs from Root and safepoints.

Roland Westrelin has updated the pull request incrementally with two additional commits since the last revision:

 - Update src/hotspot/share/opto/phaseX.cpp
   
   Co-authored-by: Christian Hagedorn <christian.hagedorn at oracle.com>
 - Update src/hotspot/share/opto/phaseX.cpp
   
   Co-authored-by: Christian Hagedorn <christian.hagedorn at oracle.com>

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/9961/files
  - new: https://git.openjdk.org/jdk/pull/9961/files/38fae0d6..b759594b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=9961&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=9961&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/9961.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/9961/head:pull/9961

PR: https://git.openjdk.org/jdk/pull/9961