Unit tests for OpenJDK?

[Alan Bateman]

On 27/04/2014 09:55, Harry Simons wrote:
> Hi,
>
> I downloaded OpenJDK 7u40 source bundle, and was expecting to find in it
> tons and tons of unit tests for, well, at least all public methods of all
> of its public classes.
>
> However, under the openjdk/jdk/test/ directory of the downloaded bundle,
> are sitting merely *regression* tests -- or, tests created for *specific*
> bugs.
>
> I'd like to know if OpenJDK comes with its unit tests or not? And, if yes,
> where can I find them? If not, why are they not being shared openly.
> Context: I'm writing a custom file system using the Java NIO2 API. To test
> it fully, I'd like to ensure that it at least passes the unit tests of the
> JDK itself. Right now, I don't know what all tests to create, even the
> so-called corner cases appear to be huge in number!
>
The test tree can be difficult to grok when you initially approach it 
but at a high level it has a mix of unit, regression, stress and other 
tests. For historical reasons that pre-date OpenJDK, the percentage, 
quality and coverage of each type depends wildly on the area. For older 
areas of the platform then you might not find much that resembles a 
traditional unit test. The reason for this is that development in those 
areas at the time mostly relied on other test suites (JCK for 
conformance tests, unit and functional test suites that resided 
elsewhere, these are test suites are not in OpenJDK). In more recent 
time it has become practice to push the tests for new features and new 
APIs with the feature/API itself. So you should see that newer areas do 
have a lot more unit tests. They might not be unit tests in the style of 
JUnit or TestNG tests that you might expect and it does takes a bit of 
getting used to the jtreg test runner. As to the number of regression 
tests for bugs then this is just the long standing rule that all bug 
fixes should come with test coverage unless there is a good reason not 
to. In some areas then bug fixes drive improvements and better test 
coverage to existing tests, in order areas then tests are just added to 
exercise the specific issue. So definitely a lot of inconsistency to 
baffle new people. Stuart Marks and I recorded a couple of sessions on 
the OpenJDK tests some time ago that I think on are YouTube if you can 
find them.

As regards the tests for the NIO file system area then then you are in 
jdk/test/java/nio/file. They are 12k+ lines of test code that provide 
reasonable coverage on all platforms. They pre-date the use of TestNG in 
OpenJDK so might not be initially obvious how you run, that just takes 
getting used to. That said, those tests are focused on the 
default/platform file system, they are not tests that will help much 
with a custom file system provider. You'll see the zip provider that we 
have in the JDK has its own tests. The reason for this is that you 
quickly get into specifies when testing a custom provider (the syntax of 
file paths is specific to the file system provider for example). This is 
a topic to bring up on nio-dev where there are others working on other 
file system providers. There might be opportunities to collaborate or 
contribute tests for example.

-Alan.