Vector API: stack overflow on Big Endian

[Paul Sandoz]

Hi Martin,

We definitely have far less exposure by default to BE platforms now that SPARC is not a thing, it's easy to unintentionally hardcode a bias to LE.

Would it be possible to try running the tests on your BE platforms with the following modification?

--- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/AbstractVector.java
+++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/AbstractVector.java
@@ -500,7 +500,7 @@ abstract class AbstractVector<E> extends Vector<E> {
     final <F>
     AbstractVector<F> defaultReinterpret(AbstractSpecies<F> rsp) {
         int blen = Math.max(this.bitSize(), rsp.vectorBitSize()) / Byte.SIZE;
-        ByteOrder bo = ByteOrder.LITTLE_ENDIAN;
+        ByteOrder bo = ByteOrder.nativeOrder();//LITTLE_ENDIAN;
         ByteBuffer bb = ByteBuffer.allocate(blen);
         this.intoByteBuffer(bb, 0, bo);
         VectorMask<F> m = rsp.maskAll(true);

Paul.


> On Oct 21, 2020, at 4:10 AM, Doerr, Martin <martin.doerr at sap.com> wrote:
> 
> Hi,
> 
> I noticed stack overflow issues because of endless recursion on Big Endian platforms (s390 and PPC64) in several tests:
> 
> E.g. Int64VectorLoadStoreTests:
> 
>        at jdk.incubator.vector/jdk.incubator.vector.IntVector.maybeSwap(IntVector.java:3330)
>        at jdk.incubator.vector/jdk.incubator.vector.IntVector.intoByteBuffer(IntVector.java:3151)
>        at jdk.incubator.vector/jdk.incubator.vector.AbstractVector.defaultReinterpret(AbstractVector.java:505)
>        at java.base/jdk.internal.vm.vector.VectorSupport.convert(VectorSupport.java:441)
>        at jdk.incubator.vector/jdk.incubator.vector.AbstractVector.convert0(AbstractVector.java:686)
>        at jdk.incubator.vector/jdk.incubator.vector.AbstractVector.asVectorRawTemplate(AbstractVector.java:173)
>        at jdk.incubator.vector/jdk.incubator.vector.AbstractVector.asByteVectorRawTemplate(AbstractVector.java:179)
>        at jdk.incubator.vector/jdk.incubator.vector.Int64Vector.asByteVectorRaw(Int64Vector.java:177)
>        at jdk.incubator.vector/jdk.incubator.vector.Int64Vector.asByteVectorRaw(Int64Vector.java:41)
>        at jdk.incubator.vector/jdk.incubator.vector.IntVector.reinterpretAsBytes(IntVector.java:3366)
>        at jdk.incubator.vector/jdk.incubator.vector.IntVector.maybeSwap(IntVector.java:3330)
> 
> Note that maybeSwap is endianness sensitive:
>    IntVector maybeSwap(ByteOrder bo) {
>        if (bo != NATIVE_ENDIAN) {
>            return this.reinterpretAsBytes()
>                .rearrange(swapBytesShuffle())
>                .reinterpretAsInts();
>        }
>        return this;
>    }
> 
> How is this supposed to work?
> Is anything platform specific missing?
> 
> Thanks and best regards,
> Martin
>