RFR: 8346914: UB issue in scalbnA

[Kim Barrett]

On Wed, 18 Jun 2025 04:24:49 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> +  if ((u_k > 0x7fe) && (n > 0)) {
>> +    // Either (k+n > 0x7fe && k+n <= INT_MAX) or k+n would overflow.
>> +    return hugeX*copysignA(hugeX,x); /* overflow  */
>> +  }
>> 
>> But that is not correct - we should only take this "overflow" path for `0x7fe < k+n <= INT_MAX`. Your suggestion makes us take this path if `k+n` overflows to negative. ??
>
>> `+  // We know that k+n <= (int)0x7fe, and might be negative if n is negative.`
> 
> It can be negative if `n` is positive too.

> But that is not correct - we should only take this "overflow" path for
> `((k+n) > 0x7fe && (k+n) <= INT_MAX)`. Your suggestion makes us take this
> path if `(k+n)` overflows to negative. ??

It is intentional that the new test is true for the `(k+n)` => overflow case.
It fully handles the overflow case, eliminating the need for the later fixup
of the case where `((k <= -54) && (n > 5000))` (though `(n > 0)` would work
just as well; I don't know why that `5000` was inserted). That fixup returned
the same `hugeX`-based result as here.

> It can be negative if n is positive too.

`(k+n)` cannot be negative with `n` positive here, even under wrapping
semantics, because we can't get here in that case due to the prior overflow
detection.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25656#discussion_r2153961162