RFR: 8302459: Missing late inline cleanup causes compiler/vectorapi/VectorLogicalOpIdentityTest.java IR failure [v8]

[Damon Fenacci]

> # Issue
> 
> The `compiler/vectorapi/VectorLogicalOpIdentityTest.java` has been failing because C2 compiling the test `testAndMaskSameValue1` expects to have 1 `AndV` nodes but it has none.
> 
> # Cause
> 
> The issue has to do with the criteria that trigger a cleanup when performing late inlining. In the failing test, when the compiler tries to inline a `jdk.internal.vm.vector.VectorSupport::binaryOp` call, it fails because its argument is of the wrong type, mainly because some cast nodes “hide” the more “precise” type. 
> The graph that leads to the issue looks like this:
> ![1BCE8148-1E44-4CA1-AF8F-EFC6210FA740](https://github.com/user-attachments/assets/62dd917f-2dac-42a9-90cf-73eedcd3cf8a)
> The compiler tries to inline `jdk.internal.vm.vector.VectorSupport::load` and it succeeds:
> ![752E81C9-A37D-4626-81A9-E4A839FADD3D](https://github.com/user-attachments/assets/e61057b2-3093-4992-ba5a-b80e4000c0ec)
> The node `3027 VectorBox` has type `IntMaxVector`. `912 CastPP` and `934 CheckCastPP` have type `IntVector`instead.
> The compiler then tries to inline one of the 2 `bynaryOp` calls but it fails because it needs an argument of type `IntMaxVector` and the argument it is given, which is node `934 CheckCastPP` , has type `IntVector`.
> 
> This would not happen if between the 2 inlining attempts a _cleanup_ was triggered. IGVN would run and the 2 nodes  `912 CastPP` and `934 CheckCastPP` would be folded away. `binaryOp` could then be inlined since the types would match.
> 
> # Solution
> 
> Instead of fixing this specific case we try a more generic approach: when late inlining we keep track of failed intrinsics and re-examine them during IGVN. If the `Ideal` method for their call node is called, we reschedule the intrinsic attempt for that call.
> 
> # Testing
> 
> Additional test runs with `-XX:-TieredCompilation` are added to `VectorLogicalOpIdentityTest.java` and `VectorGatherMaskFoldingTest.java` as regression tests and `-XX:+IncrementalInlineForceCleanup` is removed from `VectorGatherMaskFoldingTest.java` (previously added as workaround for this issue)
> 
> Tests: Tier 1-4 (windows-x64, linux-x64/aarch64, and macosx-x64/aarch64; release and debug mode)

Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:

  JDK-8302459: add comments to asserts

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/21682/files
  - new: https://git.openjdk.org/jdk/pull/21682/files/cfa5252b..f84cafbe

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=21682&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=21682&range=06-07

  Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/21682.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21682/head:pull/21682

PR: https://git.openjdk.org/jdk/pull/21682