RFR: 8290850: C2: create_new_if_for_predicate() does not clone pinned phi input nodes resulting in a broken graph [v2]

[Christian Hagedorn]

> The test cases of this bug reveal the same problem with `PhaseIdealLoop::create_new_if_for_predicate()` as in [JDK-8271954](https://bugs.openjdk.org/browse/JDK-8271954) (reusing pinned data nodes for different UCT paths into the UCT phi - see PR description of https://github.com/openjdk/jdk/pull/5185). While JDK-8271954 only fixed the usage of `PhaseIdealLoop::create_new_if_for_predicate()` for one specific case in loop unswitching, we now need this fix for other usages of `PhaseIdealLoop::create_new_if_for_predicate()` as well. I've found failing cases for most of the usages but I think we should always do a proper cloning as originally added with JDK-8271954. This is what a propose with this patch.
> 
> To always do this cloning, I've replaced the `UnswitchingAction` by a `rewire_uncommon_proj_phi_inputs` bool that is false by default but can be set if we should only do a rewiring (we can still do the rewiring for the slow loop as previously done with `UnswitchingAction::SlowLoopRewiring`, also see https://github.com/openjdk/jdk/pull/5185 for more details). However, the current fix does not update ctrl for the slow loop nodes - I've fixed that.
> 
> I've reused the implementation of `clone_data_nodes_for_fast_loop()` but refactored and split that method into multiple methods to reuse some of them for the ctrl update of the slow loop nodes.
> 
> Thanks,
> Christian

Christian Hagedorn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Add ResourceMark
 - Merge branch 'master' into JDK-8290850
 - Fix whitespaces
 - 8290850: C2: create_new_if_for_predicate() does not clone pinned phi input nodes resulting in a broken graph

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/11452/files
  - new: https://git.openjdk.org/jdk/pull/11452/files/42f98db5..dc1074d0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=11452&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=11452&range=00-01

  Stats: 103803 lines in 1610 files changed: 47481 ins; 38577 del; 17745 mod
  Patch: https://git.openjdk.org/jdk/pull/11452.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/11452/head:pull/11452

PR: https://git.openjdk.org/jdk/pull/11452