[11] RFR(M): 8205033: [REDO] Induction variable of over-unrolled loop conflicts with range checks

[Tobias Hartmann]

Hi,

my initial patch for JDK-8203915 [1] failed because the computation of the value of the loop
induction variable at the end of the first iteration of the unrolled loop was wrong. As a result,
the predicate was incorrectly found to always fail and we hit a HaltNode (SIGILL) in compiled code.

The problem is very easy to reproduce with the jdk/sun/security/tools/keytool/ tests, I therefore
did not add an additional regression test. The problematic loop looks like this:

for (int i = 20; i > 0; i -= 4) {
  array[i-1] = ...
  array[i-2] = ...
  array[i-3] = ...
  array[i-4] = ...
}

To make sure that the last array access is in bounds, we add a skeleton predicate of the form

max_i - 4 = (init + stride + 1) - 4 <u array.length

before the main loop and update the stride during unrolling. After unrolling 4 times, we know that
init <= 16 (because the pre-loop is executed at least once) and the stride is 4*-4 = -16. The
predicate is updated to:

( (int,16] - 16 + 1) - 4 = (int,-3] <u array.length

which is always false. The computation of max_i is incorrect because the (fully) unrolled loop looks
like this:

  // i <= 16
  array[i-1] = ...
  array[i-2] = ...
  array[i-3] = ...
  array[i-4] = ...
  i -= 4;
  array[i-1] = ...
  array[i-2] = ...
  array[i-3] = ...
  array[i-4] = ...
  i -= 4;
  array[i-1] = ...
  array[i-2] = ...
  array[i-3] = ...
  array[i-4] = ...
  i -= 4;
  array[i-1] = ...
  array[i-2] = ...
  array[i-3] = ...
  array[i-4] = ...
  // i <= 4

And therefore the value of i at the end of the iteration is max_i = init + stride - init_inc (where
'init_inc' is the initial loop increment). The correct predicate looks like this:

max_i - 4 = (init + stride - init_inc) - 4 = (16 - 16 + 4) -4  = 0 <u array.length

Incremental changes:
http://cr.openjdk.java.net/~thartmann/8205033/webrev.inc/

Full webrev:
http://cr.openjdk.java.net/~thartmann/8205033/webrev.00/

The failing tests pass. I'm now testing with hs-tier1-3, jdk-tier1-3 and hs-precheckin-comp. I will
also rerun the benchmarks.

Thanks,
Tobias

[1]
https://bugs.openjdk.java.net/browse/JDK-8203915
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2018-June/029250.html