RFR: 8346914: UB issue in scalbnA

[Kim Barrett]

On Thu, 19 Jun 2025 01:20:42 GMT, David Holmes <dholmes at openjdk.org> wrote:

>>> But that is not correct - we should only take this "overflow" path for
>>> `((k+n) > 0x7fe && (k+n) <= INT_MAX)`. Your suggestion makes us take this
>>> path if `(k+n)` overflows to negative. ??
>> 
>> It is intentional that the new test is true for the `(k+n)` => overflow case.
>> It fully handles the overflow case, eliminating the need for the later fixup
>> of the case where `((k <= -54) && (n > 5000))` (though `(n > 0)` would work
>> just as well; I don't know why that `5000` was inserted). That fixup returned
>> the same `hugeX`-based result as here.
>> 
>>> It can be negative if n is positive too.
>> 
>> `(k+n)` cannot be negative with `n` positive here, even under wrapping
>> semantics, because we can't get here in that case due to the prior overflow
>> detection.
>
> Okay @kimbarrett  I suggest that you take over this issue and PR. I do not have the knowledge of the code that you have and I cannot affirm that your statements are correct.

For the record, my suggestion above isn't quite right. It doesn't properly handle subnormal x. I forgot that k (before adding n) can be negative in the subnormal case. So the correct test of u_k is not
((u_k > 0x7fe) && (n > 0))
but rather
((u_k > 0x7fe) && ((k|n) > 0))
I'll be putting more details in JBS.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25656#discussion_r2158422212