Vector API: stack overflow on Big Endian

[Doerr, Martin]

Hi Paul,

thanks for pointing me to the right place.

I've filed
https://bugs.openjdk.java.net/browse/JDK-8255349
and created
https://github.com/openjdk/jdk/pull/840
.

Feel free to modify/write comments.

Thanks for your help and best regards,
Martin


> -----Original Message-----
> From: Paul Sandoz <paul.sandoz at oracle.com>
> Sent: Freitag, 23. Oktober 2020 00:18
> To: Doerr, Martin <martin.doerr at sap.com>
> Cc: hotspot-compiler-dev at openjdk.java.net
> Subject: Re: Vector API: stack overflow on Big Endian
> 
> Thanks, that is helpful, so it's only re-bracket tests for lanewise conversion
> that fail.
> 
> This reinforces my suspicions that the actual output is fine, and the test
> assumes LE encoding in castByteArrayData, which I think takes a short cut for
> casting just for +ve integral types. Lanewise conversion skips floating point
> types and so is only partially tested.
> 
> The method castByteArrayData needs to be reimplemented loading into a
> ByteBuffer, reading values, casting them, and storing them into a resulting
> buffer. Not difficult to do, just a little laborious.
> 
> Paul.
> 
> > On Oct 22, 2020, at 11:36 AM, Doerr, Martin <martin.doerr at sap.com>
> wrote:
> >
> > Hi Paul,
> >
> > here's the snippet from the
> JTwork/jdk/incubator/vector/VectorReshapeTests.jtr from a run on an s390
> machine.
> > Let me know if you need anything else.
> >
> > Thanks and best regards,
> > Martin