Tool suggestion: JTReg test module dependencies verifier.

[Alexandre (Shura) Iline]

Jon,

I do not think I have ever replied to this in writing, even though we have discussed this in person.

There are indeed a few checks on test description which could be useful, in addition to what you mentioned.

I am specifically calling that out as non-requirement because my thinking was that a tool to verify JTResg test description could be a next step. Such tool would then be using the "JTReg test module dependencies verifier” as a module of a sort.

Shura

> On Mar 29, 2018, at 3:51 PM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> Shura,
> 
> I think the tool you are proposing could be the first in a series to help check for issues in jtreg test descriptions that for whatever reason (typically performance) we do not want to check every time we use jtreg to run tests.
> 
> Other (future) checks could be, using @compile where @build would be more appropriate, checking the set of @build directives, and so on.
> 
> -- Jon
> 
> On 3/23/18 5:54 PM, Alexandre (Shura) Iline wrote:
>> Hi.
>> 
>> This is an invitation to discuss requirements for a “JTReg test module dependencies verifier” (for not having a better name).
>> 
>> Background: test module dependencies are declared with @modules JTReg tag and modules entry in test suite configuration files. Please consult JTReg documentation for more information.
>> 
>> Problem statement: At the current state there is no effective way for a test developer to verify correctness of module declarations for a test or a group of tests. To verify that information, a test developer needs to manually limit module availability to those declared as module dependencies. This could be achieved either through VM options or by building a custom JDK image. That operation has to be repeated for every subset of tests having unique set of module dependencies. This is a tedious, error prone manual process.
>> 
>> Intent of the tool is to assess correctness of declared module dependencies of JTReg tests.
>> 
>> I. Functional requirements
>> 1. The tool must be able to detect tests which, for execution, depend on presence of more modules than declared.
>> 2. There must be a way to specify a subset of tests for analysis in a way that is compatible with JTReg.
>> 3. Results of the module dependencies assessment should be stored in a machine parse-able format.
>> 3. Results of the module dependencies assessment should be provided in a human readable format.
>> 4. The tool must allow to specify additional test execution parameters which influence test behavior.
>> 5. The tool may provide some analysis of test artifacts in order to identify missing module dependencies.
>> 6. The tool may provide some analysis of test artifacts in order to identify extra, not needed module dependencies.
>> 7. The tool must provide a command-line interface.
>> 
>> II. Non-functional requirements
>> 1. The tool must use JTReg tool or JTReg API to perform test selection and test execution.
>> 
>> III. Non-requirement
>> 1. The tool does not have to be a part of JTReg source repository.
>> 2. The tool does not have to be a part of binary JTReg distribution.
>> 3. It is not a requirement for the tool to verify other JTReg test tags or any other aspects of test suite consistency.
>> 
>> 
>> Shura
>