RFR: 8367994: test/jdk/sun/security/pkcs11/Signature/ tests pass when they should skip [v2]

[Matthew Donovan]

On Tue, 23 Sep 2025 15:23:23 GMT, Mikhail Yankelevich <myankelevich at openjdk.org> wrote:

>> test/jdk/sun/security/pkcs11/Signature/SignatureTestPSS.java line 79:
>> 
>>> 77:     private static final int UPDATE_TIMES_HUNDRED = 100;
>>> 78: 
>>> 79:     private static boolean skipTest = true;
>> 
>> It looks like this field is still being and we might test some algorithms but the still throw a SkippedException.  Can you get rid of this uncertainty somehow?
>
> I'm not sure what you mean here. I think the best way would be to split the `@test` and test each algorithm separately. However this approach will take forever to execute, which is a dealbreaker. 
> 
> I can add all the skips into one list and only then print out the skipped parameters, if this helps. However, given that the test is split into sha and sha3, all sha3 will be skipped for now. I'm not sure it would bring much more clarity in this case. What do you think?

In my opinion, tests shouldn't have logic that runs some test cases and skips others. It creates uncertainty when trying to analyze test coverage. In this case, if a SkippedException is thrown, we don't know what -- if any -- algorithms were successfully tested unless we review the logs and that's an unnecessary burden. If NSS doesn't support the SHA3 algorithms, then we shouldn't try to test them. When we update the NSS library, we need to look at what's newly supported (or removed) and update tests accordingly.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27367#discussion_r2375715510