[9] RFR: 8074580: sun/security/pkcs11/rsa/TestKeyPairGenerator.java fails due to PKCS11Exception: CKR_FUNCTION_FAILED

[Valerie Peng]

The webrev looks fine.
Thanks,
Valerie

On 6/21/2016 2:31 PM, Artem Smotrakov wrote:
> I forgot to include PKCS11.java to webrev, here is an updated webrev
>
> http://cr.openjdk.java.net/~asmotrak/8074580/webrev.01/
>
> Artem
>
> On 06/20/2016 11:12 AM, Artem Smotrakov wrote:
>> Hello,
>>
>> Please review the following patch below for 9.
>>
>> TestKeyPairGenerator.java test intermittently fails with 
>> CKR_FUNCTION_FAILED error when NSS crypto libs is used via SunPKCS11 
>> provider. Looks like the root cause is a bug 1012786 in NSS
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=1012786
>>
>> RSA_NewKey() funсtion may fail with SEC_ERROR_NEED_RANDOM in case of 
>> lack of entropy which results to CKR_FUNCTION_FAILED.
>>
>> PKCS11 spec says the following about CKR_FUNCTION_FAILED error:
>>
>> ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v2-30/pkcs-11v2-30b-d6.pdf
>>
>> ...
>> CKR_FUNCTION_FAILED: ... In any event, although the function call
>> failed, the situation is not necessarily totally hopeless, as it is 
>> likely to be when
>> CKR_GENERAL_ERROR is returned. Depending on what the root cause of the
>> error actually was, it is possible that an attempt to make the exact 
>> same function call
>> again would succeed.
>> ...
>>
>> In this particular case, CKR_FUNCTION_FAILED error doesn't seem to be 
>> hopeless, and next call may succeed. The patch below tries to 
>> workaround this problem by calling C_GenerateKeyPair() function again 
>> in case of CKR_GENERAL_ERROR.
>>
>> I was not able to reproduce this failure manually on several 
>> machines, so unfortunately I am not completely sure it solves the 
>> problem. I can back out the fix if we see those failures again.
>>
>> Webrev: http://cr.openjdk.java.net/~asmotrak/8074580/webrev.00/
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8074580
>>
>> Artem
>