RFR 8238555: Allow Initialization of SunPKCS11 with NSS when there are external FIPS modules in the NSSDB

[Martin Balao]

Hi,

I'd like to propose a solution for 8238555 [1].

Webrev.00:

 * http://cr.openjdk.java.net/~mbalao/webrevs/8238555/8238555.webrev.00/

Reproducing this issue requires manual configuration steps and there is
not a single way of doing so. The ultimate goal for a reproduction is to
initialize a SunPKCS11 provider with an NSSDB that has at least 1
external module configured in FIPS mode, with at least 1 opened slot.

The 8238555_manual_reproducer_v0 code [2] provides a standalone
SunPKCS11 initialization with an NSSDB that has a single internal FIPS
module configured. That's not enough though because the external module
is still missing in the NSSDB. There are two paths from this point:

1) Manually add an external module ("modutil" command) in FIPS mode to
the NSSDB

2) Run the code in the latest Fedora/CentOS/RHEL Linux release -I'm not
sure if other distributions work- where p11-kit-proxy PKCS#11 module is
automatically added to every NSSDB. If you go this way, configure FIPS
policy globally (fips-mode-setup --enable) and recompile the NSS library
to artificially expose a slot for p11-kit-proxy module [3] (use
LD_PRELOAD when running the reproducer code). If you don't want to
recompile the NSS library, manually add a module to p11-kit (such as
softHSM) so a slot is opened.

In my own environment, I had the following output before the patch:

Beginning test run ExternalFipsModules...
Cannot resolve artifact, please check if JIB jar is present in classpath.
nssLibDir: /usr/lib64/
Exception in thread "main" java.lang.RuntimeException: FIPS flag set for
non-internal module: p11-kit-proxy.so, p11-kit-proxy
	at
jdk.crypto.cryptoki/sun.security.pkcs11.Secmod$Module.<init>(Secmod.java:418)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.Secmod.nssGetModuleList(Native
Method)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.Secmod.getModules(Secmod.java:258)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:219)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.SunPKCS11$1.run(SunPKCS11.java:112)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.SunPKCS11$1.run(SunPKCS11.java:109)
	at
java.base/java.security.AccessController.doPrivileged(AccessController.java:554)
	at
jdk.crypto.cryptoki/sun.security.pkcs11.SunPKCS11.configure(SunPKCS11.java:109)
	at PKCS11Test.getSunPKCS11(PKCS11Test.java:160)
	at PKCS11Test.testNSS(PKCS11Test.java:580)
	at PKCS11Test.main(PKCS11Test.java:220)
	at PKCS11Test.main(PKCS11Test.java:196)
	at ExternalFipsModules.main(ExternalFipsModules.java:31)

And after the patch:

Beginning test run ExternalFipsModules...
Cannot resolve artifact, please check if JIB jar is present in classpath.
nssLibDir: /usr/lib64/
Running test with provider SunPKCS11-NSS-FIPS (security manager
disabled) ...
Provider: SunPKCS11-NSS-FIPS version 15
TEST PASS - OK
Completed test with provider SunPKCS11-NSS-FIPS (2 ms).

Thanks,
Martin.-

--
[1] - https://bugs.openjdk.java.net/browse/JDK-8238555
[2] -
http://cr.openjdk.java.net/~mbalao/webrevs/8238555/8238555_manual_reproducer_v0.tar.gz
[3] -
http://cr.openjdk.java.net/~mbalao/webrevs/8238555/emulate_p11-kit-proxy_with_slots.nss.patch