<i18n dev> RFR: 8174722: Wrong behavior of DecimalFormat with RoundingMode.UP in special case

[Joe Darcy]

On Tue, 23 May 2023 23:16:01 GMT, Justin Lu <jlu at openjdk.org> wrote:

> Please review this PR, which addresses a case where Decimal Format would violate certain RoundingMode contracts given the right pattern and double.
> 
> For example,
> 
> 
> DecimalFormat df = new DecimalFormat();
> df.setRoundingMode(RoundingMode.UP);
> double small = 0.0001;
> double big = 1.0001;
> df.applyPattern("0.00");
> df.format(small); // returns 0.00, which violates UP
> df.format(big); // returns 1.01, which does not violate UP
> 
> 
> In this example `0.0001` becomes `0.00`, a decrease in magnitude. This violates the RoundingMode.UP contract as RoundingMode.UP states "Note that this rounding mode never decreases the magnitude of the calculated value."
> 
> This edge case is a result of when values smaller than the absolute value of `.1` have more leading zeros between the decimal and the first non-zero digit (_0.0001 -> **3**_) than maximum fractional digits in the pattern (_0.00 -> **2**_).
> 
> The change to Decimal Format ensures that during this case, the rounding mode is considered before truncation, which dictates if the formatted number should insert a 1 in the least significant digit location.
> 
> The test validates the change by using data from larger counterparts. For example, If we are testing `0.0009`, we can format `1.0009` with the same pattern and mode that we use on `0.0009`, then compare the fractional portions to each other and ensure they are equal.

Note there are some hazards here with respect to the numerical values of double and the decimal string used to represent the double; in exact terms:


jshell> new BigDecimal(1.0001)
$2 ==> 1.0000999999999999889865875957184471189975738525390625

jshell> new BigDecimal(0.0001)
$3 ==> 0.000100000000000000004792173602385929598312941379845142364501953125

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14110#issuecomment-1560254680