test paths in repo

[Paul Sandoz]

On Sep 4, 2014, at 9:55 PM, John Rose <john.r.rose at oracle.com> wrote:
> David Chase and I just noticed files like this in the JDK:
>  http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/test/java/util/stream/test/org/openjdk/tests/java/util/stream/
> 
> The package for the test code is "org.openjdk.tests.java.util.stream".  Although that's long, it is completely unambiguous.
> 
> Generally, grammar of such paths appears to be:
>  $REPO / test / $TEST_PACKAGE_PREFIX / $API_PACKAGE
> where TEST_PACKAGE_PREFIX is fixed as "org/openjdk/tests", and API_PACKAGE is "java/util/stream" or some such.
> 

Further more, two other sets of classes are required to be on the bootclasspath and in the j.u.stream package: test framework classes; and white box tests.

For the tests you mention, that don't need to be on the bootclass path we chose something different. It's named as if, perhaps in a better world?, most pure JDK java-based tests could be compiled and packaged into one jar file. FWIW there also happen to be a few other tests in other packages (e.g. SplittableRandomTest, that tests streams leveraging the test framework).

These tests started out as pure testng tests and we used to run 'em via ant in the lambda repo, so i think that has influenced it's structure too (since IIRC it may have also contained more tests corresponding to other packages).

FWIW a similar type of naming structure is used for JCK tests e.g.:

  package javasoft.sqe.tests.api.java.util.stream.IntStream;


> Has this organization for tests worn well?  Would you do it again that way?
> 

I think it has worked out well so far, even if the naming is a little verbose (which is fine when using a compact package view in an IDE). I would do it again :-)


> Also, What is the reason for the close correspondence between the file system pathname and the Java package declaration?  Is it so that IDEs can easily find the test files?  (If so, which IDEs?)
> 

It definitely makes it easy for to tell an IDE, such as IntelliJ, "these are my test source roots". We refactored the tests as least as much as we refactored the API/impl. When i refactor the API by making a name change i want my tests (+ supporting framework) to be automatically updated too, when i refactor the test framework i want my tests to automatically update.

IMHO it is tricky to develop and maintain the JDK tests since it is hard to get a holistic view of all tests so one can crunch on 'em as a whole and doing useful analysis via IDEs that is possible on the JDK code itself (e.g. see recent cleanups for StringBuilder/Buffer).  In this respect i would argue the current structure is actively hostile for maintaining tests, which i think should be given the same care and attention as the code they are testing. This is perhaps one reason why many tests duplicate assertion logic, there is some supporting help, but it's kind of a minor pain to use it and find it (perhaps that is just me being lazy?).

Hth,
Paul.

> For David's Arrays 2.0 work, which in some ways resembles the streams work, we want a good organization for tests, so we are looking at the streams tests as a template.
> 
> Thanks for any insights...
> — John