Evaluating the TCK for JSR 376

[mark.reinhold at oracle.com]

2017/7/28 9:33:30 -0700, tim_ellison at uk.ibm.com:
> "Volker Simonis" <volker.simonis at gmail.com> wrote on 27/07/2017 10:50:38:
>> ...
>> 
>> That's a nice feature for playing around, but I think the TCK
>> conformance test should by default really use and test all the default
>> platform modules from the runtime image under test and automatically
>> tests all these modules (i.e. it should check all exports, all the
>> requires transitive relations and finally it should check if the
>> modules form a "closed" subset with regard to the specification (i.e.
>> 'requires transitive') and also a "closed" subset with regard to the
>> implementation (i.e. 'requires')).
>> 
>> I think this will make it possible for malicious people to certify
>> custom images which don't conform to the specification (because as far
>> as I see the TCK doesn't test a complete implementation but just the
>> part you ask it to test). But you probably already know that.
> 
> I think that is a usability request rather than a requirement on the TCK.
> 
> While it would be desirable for the TCK to check that the Implementation
> adheres to *all* the obligations of the Specification, that is unlikely
> to be achievable.
> 
> The TCK can't stop "malicious people" claiming compliance where they don't
> follow the conditions of the Specification, or run the TCK as intended,
> including testing all the SE modules in the Implementation.
> 
> However, I agree it is a desirable enhancement and will reduce honest
> mistakes.

Agreed.

- Mark