RFR: 8271199: Mutual TLS handshake fails signing client certificate with custom sensitive PKCS11 key

[Xue-Lei Andrew Fan]

On Fri, 23 Jul 2021 10:33:14 GMT, Alexey Bakhtin <abakhtin at openjdk.org> wrote:

> Hello,
> 
> Could you please review the small patch for the issue described in JDK-8271199: Mutual TLS handshake fails signing client certificate with custom sensitive PKCS11 key
> 
> I suggest updating the RSAPSSSignature.isValid() method to verify if provided key components can be applied to SunRSASign implementation. 
> If not applied, implementation can try to select signer from other providers
> 
> Regards
> Alexey

It's a good catch.  Thank you for the fix.

src/java.base/share/classes/sun/security/rsa/RSAPSSSignature.java line 248:

> 246:                              crtKey.getCrtCoefficient().signum() == 0 ||
> 247:                              crtKey.getPublicExponent().signum() == 0 )))) ||
> 248:                 (rsaKey instanceof RSAPublicKey rsaPubKey &&

Mixing the public key and private key together in one method may be not straightforward enough to logics like this update.  What do you think it we have two isvalid() method, one for private key and one for public key?

src/java.base/share/classes/sun/security/rsa/RSAPSSSignature.java line 250:

> 248:                 (rsaKey instanceof RSAPublicKey rsaPubKey &&
> 249:                     rsaPubKey.getPublicExponent().signum() == 0)) {
> 250:                 throw new InvalidKeyException("Invalid key attributes");

The exception description may be confusing to users.  I'm not sure if the checking could be simplified and make this exception message better matching the problems.

src/java.base/share/classes/sun/security/rsa/RSAPSSSignature.java line 251:

> 249:                     rsaPubKey.getPublicExponent().signum() == 0)) {
> 250:                 throw new InvalidKeyException("Invalid key attributes");
> 251:             }

What's the motivation to check the signum?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4887