Open source TCK was: JDK 9: General Availability

[David Herron]

On Sat, Sep 23, 2017 at 4:25 PM, Volker Simonis <volker.simonis at gmail.com>
wrote:

> dalibor topic <dalibor.topic at oracle.com> schrieb am Fr. 22. Sep. 2017 um
> 22:30:
>
> > On 22.09.2017 18:21, Volker Simonis wrote:
> >
> > > improving the conformance tests and only then it will be possible to
> > > transparently certify one's own implementation as well as verifying
> > > the implementation of others being standards conforming.
> >
> > 'Verifying' wouldn't work, if you think a bit further about how that
> > would play out in practice. Failure to reproduce results of others could
> > have any number of benign reasons without cause for alarm.
>
>
> > For a related, timely discussion in the field of science, please see
> >
> > https://news.northeastern.edu/2015/09/failure-to-reproduce-
> results-is-a-normal-part-of-how-science-works/
> > .
> >
>
> I hope you don't want to propose that nobody should publish any scientific
> findings in the future just because their reproduction by others may fail.
> That sounds a little "Trumpish" to make obscurity great again :)
>
> For our concrete problem (i.e. a JCK certification run) publishing the
> complete certification data (especially all the .jtr files) would give a
> pretty good overview of what people really did. And if this data still
> leaves open questions which are debatable - that would actually be great!
> That would be fruitful for everybody: the Java community, the Java
> implementations, the Java specification and last but not least, for the TCK
> itself!
>
> I know that big companies love "security by obscurity" (I'm working for one
> myself ;) But that's not my personal opinion. Instead, I think an open TCK
> would help the Java ecosystem just as much as the OpenJDK did.
>
> Open sourcing the TCK for Java EE is a good step into the right direction.
> However if Oracle wants to be taken seriously with this step, open sourcing
> the Jave SE TCK must be the direct consequence. Otherwise there's a danger
> that the whole Java EE open sourcing story may look a little like riding a
> dead horse :)
>
> Regards,
> Volker
>
>
 You make a compelling case and I largely agree with the goal of "open
sourcing" the TCK's.  Doing so would go a long way towards truly opening
the OpenJDK project.

There is a practical question regarding the primary use for the TCK's -
which is to certify compliance with the specifications.

An open source software project is free to be downloaded, modified, and the
modified version distributed at will.

Hence, if someone tested with a modified TCK how can there be certainty
about the result?  Can the results of testing with a modified TCK be
trusted for certification of compliance?

For the purpose of sharing of test results, open sourcing the TCK's would
be great.  The TCK's have a different purpose than just verifying
functionality.

David Herron