Tool suggestion: JTReg test module dependencies verifier.

[Jonathan Gibbons]

Shura,

I think the tool you are proposing could be the first in a series to 
help check for issues in jtreg test descriptions that for whatever 
reason (typically performance) we do not want to check every time we use 
jtreg to run tests.

Other (future) checks could be, using @compile where @build would be 
more appropriate, checking the set of @build directives, and so on.

-- Jon

On 3/23/18 5:54 PM, Alexandre (Shura) Iline wrote:
> Hi.
>
> This is an invitation to discuss requirements for a “JTReg test module dependencies verifier” (for not having a better name).
>
> Background: test module dependencies are declared with @modules JTReg tag and modules entry in test suite configuration files. Please consult JTReg documentation for more information.
>
> Problem statement: At the current state there is no effective way for a test developer to verify correctness of module declarations for a test or a group of tests. To verify that information, a test developer needs to manually limit module availability to those declared as module dependencies. This could be achieved either through VM options or by building a custom JDK image. That operation has to be repeated for every subset of tests having unique set of module dependencies. This is a tedious, error prone manual process.
>
> Intent of the tool is to assess correctness of declared module dependencies of JTReg tests.
>
> I. Functional requirements
> 1. The tool must be able to detect tests which, for execution, depend on presence of more modules than declared.
> 2. There must be a way to specify a subset of tests for analysis in a way that is compatible with JTReg.
> 3. Results of the module dependencies assessment should be stored in a machine parse-able format.
> 3. Results of the module dependencies assessment should be provided in a human readable format.
> 4. The tool must allow to specify additional test execution parameters which influence test behavior.
> 5. The tool may provide some analysis of test artifacts in order to identify missing module dependencies.
> 6. The tool may provide some analysis of test artifacts in order to identify extra, not needed module dependencies.
> 7. The tool must provide a command-line interface.
>
> II. Non-functional requirements
> 1. The tool must use JTReg tool or JTReg API to perform test selection and test execution.
>
> III. Non-requirement
> 1. The tool does not have to be a part of JTReg source repository.
> 2. The tool does not have to be a part of binary JTReg distribution.
> 3. It is not a requirement for the tool to verify other JTReg test tags or any other aspects of test suite consistency.
>
>
> Shura