BigInteger.squareToLen() can be wrong (when called directly)

Jeff Hain jeffhainrocket at gmail.com
Wed May 20 20:55:07 UTC 2020


Hi.

>I filed https://bugs.openjdk.java.net/browse/JDK-8245038 to track this.

Can't seem to comment the bug so I post here.

>>The core issue is in implMulAdd(), where 'offset' is set to
>>"out.length-offset - 1"
>>instead of
>>"zlen - offset - 1"

There is another place that needs a fix for pure Java squareToLen()
to always work properly, it's in addOne(), where 'offset' is set to
"a.length-1-mlen-offset"
instead of
"zlen-1-mlen-offset"

I found a new test cases that allows to check both corrections:
x = [1, -2]
len = 2 (i.e. zlen = 4)
z.length = 5
z = [0, 1, -2, 0, 0] (no correction)
z = [0, 1, -6, 4, 0] (only correction 1)
z = [0, 3, -4, 0, 0] (only correction 2)
z = [0, 3, -8, 4, 0] (expected value)

I don't think there is a third place to correct:
this kind of thing is only done in these two places,
and I now checked this code through millions of random
magnitudes/values.

-Jeff


More information about the core-libs-dev mailing list