Difference in behaviour in native math library

[Ludovic Henry]

Hello,

I've noticed that some Math trigonometry tests are failing in the GNU Mauve
test suite. From digging into it, it's related to NaN values being passed
to java.lang.Math trigonometry functions, and how these values are handled
in the native libm library.

Given the following C test case compiled and run with `gcc acos.c -lm &&
./a.out`

```
#include <stdint.h>
#include <math.h>
#include <stdlib.h>
#include <stdio.h>

void main(int argc, char* argv[]) {
    int64_t bitsNaN = 0x7fff800000000000L;
    double valNaN = *((double*)&bitsNaN);

    double resD = acos(valNaN);
    int64_t res = *((int64_t*)&resD);
    if (!(res == bitsNaN)) {
        printf("expected 0x%lx but got 0x%lx\n", bitsNaN, res);
        exit(1);
    }
}
```

On a Linux-x64, the test succeeds, but on Linux-RISC-V, the test fails.

You've the same test failure in the equivalent Java code:

```
public class acos {
    public static void main (String[] args) {
        long bitsNaN = 0x7fff800000000000L;
        double valNaN = Double.longBitsToDouble(bitsNaN);

        long res = Double.doubleToRawLongBits(Math.acos(valNaN));
        if (!(res == bitsNaN)) {
            throw new RuntimeException(String.format("expected 0x%x but got
0x%x", bitsNaN, res));
        }
    }
}
```

What approach should we take in these cases? Is it that the test case is
wrong, and should assume that given a NaN, any value of NaN returned is
valid? Or should we make sure that the behavior is the same across
platforms and that we "fix" any difference in behavior of the native
library?

Cheers,
Ludovic
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/riscv-port-dev/attachments/20221208/bc744020/attachment.htm>