SunPKCS11's Secmod and external modules in FIPS mode

[Sean Mullan]

Hi Martin,

I asked around but no-one can quite recall why the fips variable was set 
this way. Our best guess is that it was set this way as we did not have 
any tests for this use case.

I don't have any issue with changing this. However, is there a way you 
could provide some tests (ex: on linux) to make sure it is working as 
expected?

Thanks,
Sean


On 1/20/20 2:16 PM, Martin Balao wrote:
> Ping. Any hint about this?
> 
> Thanks,
> Martin.-
> 
> On 12/20/19 10:03 PM, Martin Balao wrote:
>> Hello,
>>
>> SunPKCS11's Secmod in OpenJDK does not allow modules other than the NSS
>> Software Token to be configured in FIPS mode [1]. To give some context,
>> NSS represents modules internally with a structure called "struct
>> SECMODModuleStr" and the "fips" variable you see in [1] is the "isFIPS"
>> member of the module structure [2]. isFIPS is initialized by NSS to
>> false for all modules [3] but if the module spec string has a "FIPS"
>> flag, it may be turned to true [4]. Newer NSS versions (since bug
>> 1531267 [5] [6]) may set isFIPS to true for all modules when
>> /proc/sys/crypto/fips_enabled is 1 in Linux systems. As a result, as
>> soon as the system is in FIPS mode and the NSSDB has more than the NSS
>> Software Token module in it, OpenJDK refuses to initialize the SunPKCS11
>> provider. You can see a real case with pk11-kit-trust as the external
>> module in RH1780335 [7].
>>
>> This behavior has been the same since the very beginning of OpenJDK
>> (revision 2), and I couldn't find much information about it. There might
>> be a commit message previous to that.
>>
>> I'm trying to understand the rationale behind it and see what would be
>> the implications of removing the check (note: couldn't notice anything
>> in my quick test by removing it).
>>
>> Can someone give me a hint?
>>
>> Thanks,
>> Martin.-
>>
>> --
>> [1] -
>> https://hg.openjdk.java.net/jdk/jdk/file/59ddac265649/src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/Secmod.java#l417
>> [2] -
>> https://github.com/nss-dev/nss/blob/c1ff439ca931f53c318e7381636ed5889b3d66f1/lib/pk11wrap/secmodt.h#L49
>> [3] -
>> https://github.com/nss-dev/nss/blob/a141cd68ece76118aebf8033c06d46a3692b55fe/lib/pk11wrap/pk11pars.c#L49
>> [4] -
>> https://github.com/nss-dev/nss/blob/a141cd68ece76118aebf8033c06d46a3692b55fe/lib/pk11wrap/pk11pars.c#L819
>> [5] - https://bugzilla.mozilla.org/show_bug.cgi?id=1531267
>> [6] - https://hg.mozilla.org/projects/nss/rev/536fd7c9db5a
>> [7] - https://bugzilla.redhat.com/show_bug.cgi?id=1780335
>>
>