RFR: 8373722: [TESTBUG] compiler/vectorapi/TestVectorOperationsWithPartialSize.java fails intermittently

[Xiaohong Gong]

On Tue, 23 Dec 2025 12:16:01 GMT, Jie Fu <jiefu at openjdk.org> wrote:

> > For example, if the float array is `[1.0f, -1.0f, 2.0f, -2.0f]`, we genuinely expect a result very close to `0.0f`, not something near `2.0f`.
> > Using `max_abs` to set the tolerance risks inflating the admissible error range (since `max_abs` here would be 2.0f), which I'm afraid might make the test much less effective.
> 
> FYI: the current sum based tolerance may be also bigger than max_abs based. For example, if the float array is `[1.0f, 1.0f, 2.0f, 2.0f]`. The sum would be `6.0f`, the max_abs would be `2.0`. What do you think?

We used the `Math.ulp(sum) * 10` to calculate the tolerance, which means the difference of expected and actual value is inside of 10 ULP at the value of `sum`. Note that `Math.ulp(f)` is the positive distance between `f` and the next representable float value larger in magnitude (1 ulp around value `f`).  Hence, it is reasonable based on the reference value.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28960#issuecomment-3688389893