RFR: 8174722: Wrong behavior of DecimalFormat with RoundingMode.UP in special case

[Joe Darcy]

On Wed, 24 May 2023 18:36:39 GMT, Justin Lu <jlu at openjdk.org> wrote:

> > I just wonder if it is ok to reverse the rounding/truncation only in this special case. If the truncation always comes first, then 0.00 would be correct (then 1.01 may be something wrong). Looks to me that the original comment suggests it tries to truncate those cases. I suggest filing a CSR for further behavioral consideration.
> 
> The normal behavior of DecimalFormat (DigitList) "rounds" before truncation. For example, consider
> 
> `1.0000009` formatted to `0.0` with `RoundingMode.UP`
> 
> DigitList will search past the maximum digits allowed to decide if the formatted value should be rounded. '9' will be found, and as such the value will be "rounded", by incrementing the value by 1 at the max digit location -> `1.1000009`, then DigitList will truncate -> `1.1`
> 
> Thus I believe that this change is aligning this special case to be consistent with the existing behavior of rounding before truncation. I will follow-up in the CSR.

A quick comment without diving into the particular issues here, for each finite floating-point value, there is a range of the real number line that will round to that floating-point value. Under the default rounding policy, some of those real numbers will be less than the floating-point value and other of those real numbers will be greater than the floating-point value.

So there are decimal strings corresponding to those numbers greater than and less than the floating-point value, all of which will get rounded to the same double value on string conversion.

If the operation is phrased in terms of *the floating-point value* and not *the real number value of what happened to be the input string*, then the actual real value of the floating-point number should be the reference point for rounding up/down on double -> String conversion.

This may give "surprising" results, but is numerically better. Without finding the bug ids, some previous changes were made to decimal format when rounding to nearest along those lines. HTH

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14110#issuecomment-1561765426