RFR: 8319889: Vector API tests trigger VM crashes with -XX:+StressIncrementalInlining

[Jatin Bhateja]

This bug fix patch fixes following two issues:-

1) Removing memory operand based masked shift instruction selection patterns. As per Java specification section 15.19 shift count is rounded to fit within valid shift range by performing a bitwise AND with shift_mask this results into creation of an AndV IR after loading original mask into vector. Existing patterns will not be able to match this graph shape, extending the patten to cover AndV IR will associate memory operand with And operation and we will need to emit additional vectorAND instruction before shift instruction, existing memory operand patten for AndV already handle such a graph shape.

2) Crash occurs due to combined effect of bi-morphic inlining, exception handling, randomized incremental inlining. In this case top level slice API is invoked using concrete 256 bit vector, some of the intermediate APIs within sliceTemplate are marked for lazy inlining because due to randomized IncrementalInlining, these APIs returns an abstract vector which when used for virtual dispatch of subsequent APIs results into bi-morphic inlining on account of multiple profile based receiver types. Consider following code snippet.


    ByteVector sliceTemplate(int origin, Vector<Byte> v1) {
        ByteVector that = (ByteVector) v1;
        that.check(this);
        Objects.checkIndex(origin, length() + 1);
        VectorShuffle<Byte> iota = iotaShuffle();
        VectorMask<Byte> blendMask = iota.toVector().compare(VectorOperators.LT, (broadcast((byte)(length() - origin))));  [A]
        iota = iotaShuffle(origin, 1, true);     [B]    
        return that.rearrange(iota).blend(this.rearrange(iota), blendMask); [C]
    }



Receiver for sliceTemplate is a 256 bit vector, parser defers inlining of toVector() API (see code at line A) and generates a Call IR returning an abstract vector. This abstract vector then virtually dispatches compare API. Compiler observes multiple profile based receiver types (128 and 256 bit byte vectors) for compare API and parser generates a chain of PredictedCallGenerators for bi-morphically inlining it.
      
         PredictedCallGenerators (Vector.compare)
              PredictedCallGenerators (Byte256Vector.compare)
                 ParseGenerator (Byte256Vector.compare)               [D]
                 UncommonTrap (receiver other than Byte256Vector)
              PredictedCallGenerators (Byte128Vector.compare)
                 ParseGenerator (Byte128Vector.compare)               [E]
                 UncommonTrap (receiver other than Byte128Vector)     [F]
              PredictedCallGenerators (UncommonTrap)
         [converged state] = Merge JVM State orginating from C and E  [G]

   Since top level receiver of sliceTemplate is Byte256Vector hence while executing the call generator for Byte128Vector.compare (see code at line E) compiler observes a mismatch b/w incoming argument species i.e. one argument is a 256 bit vector while other is 128 bit vector and throws an exception.
    
At state convergence point (see code at line G), since one of the control path resulted into an exception, compiler propagates the JVM state of other control path comprising of Byte256Mask to downstream graph after  bookkeeping the pending exception state.

Similar to toVector API, iotaShuffle (see code at line B) is also lazily inlined and returns an abstract vector which results into bi-morphic inlining of rearrange. State convergence due to bi-morphic inlining of rearrange results into generation of an abstract ByteVector (Phi Byte128Vector Byte256Vector) which further causes bi-morphic inlining of blend API due to multiple profile based receiver types.
    
Byte128Vector.blend [Java implementation](https://github.com/openjdk/jdk/blob/master/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java#L412) explicitly cast incoming mask (Byte256Mask) by Byte128Mask type and this leads to creation of a[ null value](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/graphKit.cpp#L1417), this causes a crash while unboxing the mask during inline expansion of blend. To be safe here relaxing the null checking constraint during unboxing to disable intrinsification.

All existing Vector API JTREG tests are passing with -XX:+StressIncrementalInlining at various AVX levels.

Please review and share your feedback.

Best Regards,
Jatin

-------------

Commit messages:
 - 8319889: Vector API tests trigger VM crashes with -XX:+StressIncrementalInlining

Changes: https://git.openjdk.org/jdk/pull/18282/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18282&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8319889
  Stats: 51 lines in 2 files changed: 3 ins; 46 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/18282.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/18282/head:pull/18282

PR: https://git.openjdk.org/jdk/pull/18282