Difference in behaviour in native math library

[Palmer Dabbelt]

On Thu, 08 Dec 2022 08:26:30 PST (-0800), ludovic at rivosinc.com wrote:
> Adding the right address for core-libs-dev.
>
> On Thu, Dec 8, 2022 at 12:19 PM Ludovic Henry <ludovic at rivosinc.com> wrote:
>
>> Hello,
>>
>> I've noticed that some Math trigonometry tests are failing in the GNU
>> Mauve test suite. From digging into it, it's related to NaN values being
>> passed to java.lang.Math trigonometry functions, and how these values are
>> handled in the native libm library.
>>
>> Given the following C test case compiled and run with `gcc acos.c -lm &&
>> ./a.out`
>>
>> ```
>> #include <stdint.h>
>> #include <math.h>
>> #include <stdlib.h>
>> #include <stdio.h>
>>
>> void main(int argc, char* argv[]) {
>>     int64_t bitsNaN = 0x7fff800000000000L;
>>     double valNaN = *((double*)&bitsNaN);
>>
>>     double resD = acos(valNaN);
>>     int64_t res = *((int64_t*)&resD);
>>     if (!(res == bitsNaN)) {
>>         printf("expected 0x%lx but got 0x%lx\n", bitsNaN, res);
>>         exit(1);
>>     }
>> }
>> ```
>>
>> On a Linux-x64, the test succeeds, but on Linux-RISC-V, the test fails.
>>
>> You've the same test failure in the equivalent Java code:
>>
>> ```
>> public class acos {
>>     public static void main (String[] args) {
>>         long bitsNaN = 0x7fff800000000000L;
>>         double valNaN = Double.longBitsToDouble(bitsNaN);
>>
>>         long res = Double.doubleToRawLongBits(Math.acos(valNaN));
>>         if (!(res == bitsNaN)) {
>>             throw new RuntimeException(String.format("expected 0x%x but
>> got 0x%x", bitsNaN, res));
>>         }
>>     }
>> }
>> ```
>>
>> What approach should we take in these cases? Is it that the test case is
>> wrong, and should assume that given a NaN, any value of NaN returned is
>> valid? Or should we make sure that the behavior is the same across
>> platforms and that we "fix" any difference in behavior of the native
>> library?

It might just be a glibc bug, we're failing the acos() tests: 
https://sourceware.org/glibc/wiki/Release/2.35#RISC-V_.28rv64imafdc.2Flp64d.29 
.  I haven't looked at why, but they do have some nan-related bits in 
there.

>>
>> Cheers,
>> Ludovic
>>