Upgrade to RSAKeyGenParameterSpec.F4 for RSA Keypair generation test?

[Liu, Xin]

Hi, Security developers,

We can’t pass the following test on our platform for OpenJDK8.

Test: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a782529d712/test/sun/security/pkcs11/rsa/TestKeyPairGenerator.java#l106
Error Message:
Generating 512 bit keypair...
STDERR:
java.security.ProviderException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ARGUMENTS_BAD
    at sun.security.pkcs11.P11KeyPairGenerator.generateKeyPair(P11KeyPairGenerator.java:424)
    at java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:697)
    at TestKeyPairGenerator.main(TestKeyPairGenerator.java:119)
    at PKCS11Test.premain(PKCS11Test.java:88)
    at PKCS11Test.testNSS(PKCS11Test.java:403)
    at PKCS11Test.main(PKCS11Test.java:98)
    at TestKeyPairGenerator.main(TestKeyPairGenerator.java:97)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
    at java.lang.Thread.run(Thread.java:748)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ARGUMENTS_BAD
    at sun.security.pkcs11.wrapper.PKCS11.C_GenerateKeyPair(Native Method)
    at sun.security.pkcs11.P11KeyPairGenerator.generateKeyPair(P11KeyPairGenerator.java:416)
    ... 12 more


We believe the problem is caused by the NSS standard[1].
There’s a bug related to Solaris[2]. Our platform also rejects this test for the same reason.   Is it okay backport this patch to jdk8u?   I think it’s backward-compatible.

Here is difference between FIPS 186-2 and FIPS 186-4

RSA: restrict n size to 1024 2048 3072, restrict e to 2^16+1 to 2^256−1, and specify RSA private key generation in detail with several options. This prohibits one traditionally popular e namely 3; F4 (65537) is allowed and IME more popular anyway.[1]

References:
1.      https://crypto.stackexchange.com/questions/35388/what-is-the-major-difference-between-fips-186-2-and-fips-186-4
2.    https://bugs.openjdk.java.net/browse/JDK-8129560


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/security-dev/attachments/20181018/f637943a/attachment.htm>