RFR: 8231351: Add notes for PKCS11 tests in the test doc

[Jie Fu]

Hi John Jiang,

I don't care which NSS you have skipped.
I am just curious about the root cause for the failure on Ubuntu.

It seems more reasonable to close JDK-8231338 as won't fix.
But you just closed it as can't reproduce.

Thanks.
Best regards,
Jie

On 2019/9/24 下午4:01, sha.jiang at oracle.com wrote:
> Hi Jie,
> IIRC, this test passed on your Ubuntu 18.04 with a new built NSS 3.35 
> libs.
> So, I suspected your Linux or the system built-in NSS libs had 
> something wrong.
> Now that this test passed with NSS 3.35 on others' Linux, including 
> mine, it may not make sense that this test is skipped for NSS 3.35 on 
> all platforms (even Linux only).
> Certainly, you are always welcomed to re-open JDK-8231338.
>
> A bit furthermore...
> Some different PKCS11 test cases have been skipped due to the bugs on 
> different NSS releases or platforms.
> It would be better not to make the codes more complicated to deal with 
> this scenario.
> The alternative NSS libs could be specified (via system property 
> test.nss.lib.paths) for PKCS11 tests.
> It doesn't have to depend on the system built-in NSS libs.
> Different people can use different NSS versions based on their 
> requirements.
>
> Best regards,
> John Jiang
>
> On 2019/9/24 15:03, Jie Fu wrote:
>> I can't understand why JDK-8231338 was closed as "Cannot Reproduce" 
>> since it can always be reproduced on Ubuntu 18.04.2 LTS.
>>
>> Reproduce on Ubuntu 18.04 is very simple:
>> -------------------------------------
>> make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" 
>> CONF=re
>> -------------------------------------
>>
>> In the fix of JDK-8180837, the NSS-3.35 was assumed to work with 
>> AddTrustedCert.java, but it failed on Ubuntu 18.04.
>> I wonder whether there is some other reasons which may lead to the 
>> failure.
>> So I filed JDK-8231338 hoping to find the root cause of the failure.
>>
>> But I was disappointed since the author of JDK-8180837 just told me:
>>  "it's hard to say what's the problem, Linux, NSS build or others, on 
>> this test case."
>>
>> In fact, the root cause for the failure on Ubuntu 18.04 still remains 
>> unknown.
>>
>> Thanks a lot.
>> Best regards,
>> Jie
>>
>> On 2019/9/24 下午2:01, Xuelei Fan wrote:
>>> I may be a little bit hesitate to add such words, "highly 
>>> recommended to use the latest NSS version ...", in the general TOP 
>>> doc.  There are a few issues that I wary about:
>>>
>>> It is not always expected that all PKCS 11 test should be run on 
>>> latest NSS version.  Otherwise, there might be compatibility issues 
>>> that we did not handle properly.  The JDK is expected to work with 
>>> as much NSS versions as possible, not just the latest version.  It 
>>> is good to know which version does not really work because of a 
>>> specific bug.
>>>
>>> The note should be added as close as to the place where the issue 
>>> happens, for maintaining and searching.  I think the best place 
>>> could be the test code where the failure occurs 
>>> (test/jdk/sun/security/pkcs11/Secmod/AddTrustedCert.java?).
>>>
>>> However, I'm still not sure if we really want this note.
>>>
>>> JDK-8231338 is reported with NSS 3.35.  Per the comment, the test 
>>> could pass with NSS 3.35 on Debian and Ubuntu, and the bug submitter 
>>> could pass the test with a proper build of NSS 3.35. And then the 
>>> bug was closed as "Cannot Reproduce".  I think we are done with the 
>>> bug.  It might not be necessary to add a note any more.
>>>
>>> Just my $.02.
>>>
>>> Xuelei
>>>
>>> On 9/23/2019 9:06 PM, Jia Huang wrote:
>>>> Hi John Jiang,
>>>>
>>>> Thank you for your review.
>>>>
>>>> 在 2019年09月23日 20:54, sha.jiang at oracle.com 写道:
>>>>> In fact, PKCS11 tests have their own doc at: 
>>>>> test/jdk/sun/security/pkcs11/README 
>>>> I'm afraid most people wouldn't see 
>>>> test/jdk/sun/security/pkcs11/README at all.
>>>> So it makes very little sense to add the notes in it.
>>>> I still prefer doc/testing.md.
>>>>
>>>> A reference to test/jdk/sun/security/pkcs11/README had been added 
>>>> in [1].
>>>>
>>>> Thanks a lot.
>>>> Best regards,
>>>> Jia
>>>>
>>>> [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
>>>>
>>>>
>>>>
>>
>>