RFR: 8315024: Vector API FP reduction tests should not test for exact equality [v2]

[Gergö Barany]

On Thu, 5 Oct 2023 14:00:54 GMT, Gergö Barany <gbarany at openjdk.org> wrote:

>> test/jdk/jdk/incubator/vector/Double128VectorTests.java line 1117:
>> 
>>> 1115:                 return fill(s * BUFFER_REPS,
>>> 1116:                             i -> (double)(i / (double) 10.0 + 0.1));
>>> 1117:             }),
>> 
>> Did this generate rounding issues for addition?
>> Another option would be random values that make sure to fill the whole mantissa with random information.
>> Of course the tricky part is to keep them within reasonable bounds so that on multiplication they do not degenerate to zero or infinity.
>> Also: it would be nice to have some cases with extreme values (infinity, NaN, etc).
>
> Yes, this generator generates a rounding issue for addition:
> 
> test FloatMaxVectorTests.ADDReduceFloatMaxVectorTests(float[i / 10.0 + 0.1]): failure
> java.lang.AssertionError: at index #16 expected [39.2] but found [39.199997]

I updated this generator to `0.01 + (i / (i + 1))`, plus a variant that replaces some of the elements of this sequence with values from the `cornerCaseValues` generator.
This should address all of your concerns. Almost all values in this sequence are very close to 1, so they can be added and multiplied without overflow, up to about 2000 elements for `float`s. The mantissas have bits all over the place. The exponents are very limited, but a wider range of exponents is exercised by the other tests.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16024#discussion_r1349188149