BigInteger.squareToLen() can be wrong (when called directly)

[Jeff Hain]

Hi.

While trying to use BigInteger.squareToLen(x, len, z) directly
(with reflection), I saw that its pure Java implementation
computed wrong results when a 'z' is specified with
(z.length > 2 * len).
Ex.:
x = {1, 2}
len = 2
with z.length = 4: {0, 1, 4, 4} (as expected)
with z.length = 5: {0, 1, 0, 8, 0} instead of {0, 1, 4, 4, 0}

The core issue is in implMulAdd(), where 'offset' is set to
"out.length-offset - 1"
instead of
"zlen - offset - 1"
(which requires to add zlen as argument).

In practice it doesn't hurt for current usages in BigInteger,
since it seems to always be called with either null or
a tightly sized array, and I'm not sure we can create a bug
only reproducible by calling a private method directly,
but I think it would be wise to defuse this behavior.

The intrinsic implementation doesn't have this problem
(once we can get it to kick in, with "-XX:+UseSquareToLenIntrinsic"
(for pre-JDK-8154945) and a few thousand calls).
No issue with multiplyToLen() either.


-Jeff

PS: I used to post here as jeffhain at rocketmail.com,
but now it's being rejected at some point,
hence the new address.