RFR: 8341470: BigDecimal.stripTrailingZeros() optimization [v33]

[fabioromano1]

On Fri, 11 Oct 2024 17:29:59 GMT, Raffaello Giulietti <rgiulietti at openjdk.org> wrote:

>> I'll have to check...
>> ... tomorrow ;-)
>
> IIUC, the code assumes that the floating-point computation `Math.ceil(intVal.bitLength() * LOG_5_OF_2)` has the same value as the mathematical ⌈ `intVal.bitLength()` × log5(2) ⌉.
> I don't think this is safe, as it might happen that the computed and mathematical values are off by ±1.
> To ensure 5^`maxPowsOf5` >= `intVal` (that is, maxPowsOf5 >= log5(intVal)) it would be more prudent to have
> 
> long maxPowsOf5 = (long) Math.ceil(intVal.bitLength() * LOG_5_OF_2) + 1;
> 
> 
> But I think what you really want is maybe to meet 5^`maxPowsOf5` <= `intVal` < 5^(`maxPowsOf5` + 1) instead?

Actually, if we reason in terms of "ulp vs precision", the computation should be safe: `Math.log()`'s results are within 1 ulp of the exact result, and the floating point operations are a multiplication and a division. The division to compute `LOG_5_OF_2` costs 1/2 ulp plus the errors of the operands, so 2.5 ulps. Same for multiplication, but `intVal.bitLength()` has an exact `double` value, so the total roundoff error of `intVal.bitLength() * LOG_5_OF_2` is 3 ulps. Since the integer part of `intVal.bitLength() * LOG_5_OF_2` is representable with 31 bits, and double has 53 bits of precision, we can reasonably say that `Math.ceil()` can always guarantee `maxPowsOf5 >= log5(intVal)`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21323#discussion_r1797292843