[9] RFR: JDK-8164322: sun/security/pkcs11/PKCS11Test.java shall be updated to run on ARM platforms

[Xuelei Fan]

On 9/29/2016 9:26 AM, Artem Smotrakov wrote:
> Hi Xuelei,
>
> This is not a problem with machine configuration. The actual test cases
> are not going to be run (even if there are NSS libs on a test machine)
> until "osMap" contains an entry for specific platform.
>
OK.  Please fix for the purpose accordingly.

Tim, the current fix has two parts.  The 1st part, which throw exception 
is a machine configuration problem, is not in the scope of Artem's 
consideration, please remove it.  The 2nd part, which add new items to 
the osMap, is fine to me.  I think we are on the same page now.  Please 
update the webrev accordingly.  I will approve the 2nd part fix.

Thanks,
Xuelei

> Artem
>
>
> On 09/28/2016 05:52 PM, Xuelei Fan wrote:
>> Hi Artem,
>>
>> What do you think fix the testing infrastructure (JPRT/Mach5) with
>> proper configuration?  I think it is a easier than update a large
>> bunch of test codes.
>>
>> I understand the concerns of yours, but I don't want add additional
>> effort for those who need to run the test on their own environment.
>> Some people run the test locally before submit JPRT jobs (or no access
>> to JPRT systems for external people).
>>
>> Update the testing infrastructure may solve both of your concerns and
>> my concerns.
>>
>> I'm not sure the @requires tag would work or not.  It would be great
>> if you can find a solution with the tag.
>>
>> Thanks,
>> Xuelei
>>
>> On 9/29/2016 4:32 AM, Artem Smotrakov wrote:
>>> Hi Xuelei,
>>>
>>> I understand your concerns. But I'd prefer to be aware of situations
>>> when a test reports that it passed when it actually did nothing.
>>>
>>> How about using @requires key? We can try to specify all expected
>>> platforms. If I understand correctly, jtreg won't run tests if specified
>>> requirements are not met. In this case, you need to look at the report
>>> to make sure all tests you are interested in were run.
>>>
>>> Artem
>>>
>>>
>>> On 09/28/2016 08:00 AM, Xuelei Fan wrote:
>>>> On 9/28/2016 9:20 PM, Denis Kononenko wrote:
>>>>> There're 60+ tests related to PKCS11. Two years they have been
>>>>> "passed" on 3 unsupported platforms on hosts where usually no NSS
>>>>> libraries were installed. How can we rely on these results?
>>>> ;-) The words of "unsupported platforms" are very confusing in this
>>>> mail thread.
>>>>
>>>> Let's think more about what if a test failed.  What one will do if a
>>>> test failed?
>>>> 1. Test fail means source code problems for developers.  One cannot
>>>> submit a change-set if a test failed.  He need to pay additional
>>>> effort and analysis the failure.  It take one developer a lot effort
>>>> to know the root cause.  I would never like this unnecessary cost.
>>>> 2. In order to get the test pass, he need to install the NSS libs
>>>> although NSS has nothing to do with his changeset.  It may be a very
>>>> very hard step or even impossible (for example licenses issues) step
>>>> for him.
>>>>
>>>> TBH, I did not see much benefits to fail on unsupported platforms.  I
>>>> agree that skip for pass is not a good idea, but fail to warn is worse.
>>>>
>>>> I think the root cause if "unsupported platforms" actually are
>>>> supported platforms, but by accident the NSS libraries are not
>>>> installed or not installed properly.
>>>>
>>>> If one is not interested in NSS, the test get ignored (passed). If one
>>>> is interested in NSS, he should install the NSS libs and the test get
>>>> checked.
>>>>
>>>> What do you think if fix the testing infrastructure with properly
>>>> installed NSS libs?
>>>>
>>>> > The problem is the tests report they passed but actually they were
>>>> > skipped. I have no objections against skipping tests but it would
>>>> > be better to give a hint somehow how many tests were skipped and why.
>>>> Agreed.  Unfortunately, there are only two options, "pass" or "fail",
>>>> at present.  It would be nice if there is a grey area. Any idea to
>>>> make summary of skipped tests and reasons?
>>>>
>>>> Xuelei
>>>
>