SunPKCS11's Secmod and external modules in FIPS mode

[Martin Balao]

Hello,

SunPKCS11's Secmod in OpenJDK does not allow modules other than the NSS
Software Token to be configured in FIPS mode [1]. To give some context,
NSS represents modules internally with a structure called "struct
SECMODModuleStr" and the "fips" variable you see in [1] is the "isFIPS"
member of the module structure [2]. isFIPS is initialized by NSS to
false for all modules [3] but if the module spec string has a "FIPS"
flag, it may be turned to true [4]. Newer NSS versions (since bug
1531267 [5] [6]) may set isFIPS to true for all modules when
/proc/sys/crypto/fips_enabled is 1 in Linux systems. As a result, as
soon as the system is in FIPS mode and the NSSDB has more than the NSS
Software Token module in it, OpenJDK refuses to initialize the SunPKCS11
provider. You can see a real case with pk11-kit-trust as the external
module in RH1780335 [7].

This behavior has been the same since the very beginning of OpenJDK
(revision 2), and I couldn't find much information about it. There might
be a commit message previous to that.

I'm trying to understand the rationale behind it and see what would be
the implications of removing the check (note: couldn't notice anything
in my quick test by removing it).

Can someone give me a hint?

Thanks,
Martin.-

--
[1] -
https://hg.openjdk.java.net/jdk/jdk/file/59ddac265649/src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/Secmod.java#l417
[2] -
https://github.com/nss-dev/nss/blob/c1ff439ca931f53c318e7381636ed5889b3d66f1/lib/pk11wrap/secmodt.h#L49
[3] -
https://github.com/nss-dev/nss/blob/a141cd68ece76118aebf8033c06d46a3692b55fe/lib/pk11wrap/pk11pars.c#L49
[4] -
https://github.com/nss-dev/nss/blob/a141cd68ece76118aebf8033c06d46a3692b55fe/lib/pk11wrap/pk11pars.c#L819
[5] - https://bugzilla.mozilla.org/show_bug.cgi?id=1531267
[6] - https://hg.mozilla.org/projects/nss/rev/536fd7c9db5a
[7] - https://bugzilla.redhat.com/show_bug.cgi?id=1780335