RFR JDK-8164639: Configure PKCS11 tests to use user-supplied NSS libraries

[sha.jiang at oracle.com]

Hi Max,

On 2018/8/13 13:58, Weijun Wang wrote:
>
>> On Aug 13, 2018, at 1:46 PM, sha.jiang at oracle.com wrote:
>>
>> On 2018/8/13 11:25, Weijun Wang wrote:
>>> Can test.nss.lib.path contain multiple paths? For example, some systems might have libsoftkn3.so and libnss3.so in different directories [1] and depending on whether secmod is used the test might load one or the other.
>> I assume the custom libs are in a single directory.
>> This property is used for manual test run. When run different tests, users can specify different target lib path.
> But as [1] shows they can be in different directories. Therefore either you have to move them into a single directory or you will not be able to run all PKCS11 tests with a single jtreg command.
For some platforms, including linux-x64 and linux-x86, the test may 
search multiple directories for a specific NSS lib, namely softokn3 or nss3.
Please see method getOsMap() in PKCS11Test.java.

But I assume this property is used when a user-built NSS libs, but not 
system NSS libs, are needed.
The user-built NSS libs could be in a single directory.

>>> I have the same question on the downloaded file from the artifact server. It seems for each platform it is a zip file. Will it extract all libraries into the same "nsslib" directory?
>> In my case, only one zip file is downloaded in a specific platform.
>>
>> And each archive is a separated artifact, which has different artifactId.
>> The local path contains the artifactId, so different artifacts cannot be in the same directory.
>>
>>> Also, this is the 1st time I hear about @Artifact in an openjdk test and know nothing about it. Is there a detailed description on this feature somewhere?
>> I also don't get the details.
> Is there an artifact server available on the open internet?
It's transparent to me. @Artifact tool delegates the downloading.

>>> As for this test, if customNssLib is the first element in nssLibDirs and not set by a user, does this mean the test will always download libraries from an artifact server?
>> If customNssLib is null, the test downloads the platform-specific artifact from Artifactory;
>> otherwise, the test just uses the specified local NSS libs and no downloading is triggered.
> But I think the most common case is no user-customized nss lib path.
Yes. So, generally the value of property test.nss.lib.path is null.
PKCS11 tests use system NSS libs on Linux and Solaris, but download 
windows and macosx NSS libs from Artifactory (previously, these libs are 
in repo).
My fix doesn't change this logic.

>>> Would it spend too much time on downloading?
>> In this case, with my testing, it didn't take much time.
>>
>>> Is there a cache mechanism so that after the 1st PKCS11 test downloads the libraries the other tests can reuse them?
>> With my testing, if the local path for a specific artifact exists, the same artifact should not be re-downloaded.
> What is the local path? I assume it is out of the scratch directory.
In my laptop, the path is:
/private/var/tmp/jib-sha.jiang/install/jpg/tests/jdk/nsslib/nsslib-macosx_x64/3.35/nsslib-macosx_x64-3.35.zip/nsslib

> If multiple tests are running in agentvm mode, is there a risk they share the same local path and see incomplete download?
They should share the same local path.
If no synchronization on actions of building local directory structure, 
downloading and/or unpacking, some issue may raise.
I will consult infra team on this issue.

Best regards,
John Jiang
>
> Thanks
> Max
>
>>> Are they cleaned up at some time?
>> The libs are NOT removed after my manual test execution.
>> But I don't know the details about this point in CI system.
>>
>> Best regards,
>> John Jiang
>>
>>> Thanks
>>> Max
>>>
>>> [1] https://packages.ubuntu.com/xenial-updates/arm64/libnss3/filelist
>>>
>>>> On Aug 13, 2018, at 10:44 AM, sha.jiang at oracle.com wrote:
>>>>
>>>> Hi,
>>>> This patch provides a system property, exactly test.nss.lib.path, for specifying the absolute path to the custom NSS lib.
>>>> And it also removes the NSS 3.16 binary libs on windows and macosx from repo.
>>>> On these two platforms, PKCS11 tests will download new built NSS 3.35 libs from Artifactory.
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~jjiang/8164639/webrev.00/
>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8164639
>>>>
>>>> Best regards,
>>>> John Jiang
>>>>
>