RFR: 8302017: Allocate BadPaddingException only if it will be thrown [v2]

[Valerie Peng]

On Thu, 13 Jul 2023 09:58:34 GMT, Ferenc Rakoczi <duke at openjdk.org> wrote:

>> src/java.base/share/classes/sun/security/rsa/RSASignature.java line 231:
>> 
>>> 229:                             RSAUtil.decodeSignature(digestOID, unpadded));
>>> 230:                 }
>>> 231:             }
>> 
>> I understand where the fallback code came from.  As the padding code is exactly the same as engineSign(), the risk may be minimal.  With the fallback code, the security concern (time-constant) we cared about will come back.  Did you run into testing failure without the fallback doe?
>
> Instead of falling back to unpad()/decodeSignature() I suggest to try a new version of encodeSignature() (in addition to trying the current version of it) in which you omit putting the null for params into the DER encoding and compare the decrypted message with that, too. Accept if any of the two encodings matches the decrypted one, reject otherwise. This can be done in constant time, although it is not necessary to be constant time as the time of doing it does not depend on any secret.

@ferakocz So, with this approach, we are paying the extra cost of encode signature + pad (for the omit null case) even for impls conforming to RFC 8017 spec. Based on the current interoperability testing, do you still feel that this is worthwhile to do?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/14839#discussion_r1267148387