[rfc] make test ignored or valid only for specific JDK

[Jiri Vanek]

On 03/26/2015 07:18 PM, Jonathan Gibbons wrote:
> Jiri,
>
> I presume you are talking about a set of tests that is distinct from any of the jtreg test suites in

Sure :)

> an OpenJDK forest, such as langtools/test, jdk/test, etc.   Those test suites are tied to the code
> in the overall forest, and so are inherently versioned without any additional notation.
>
> There are a variety of existing mechanisms you could use to address this problem, without adding any
> new features.
>
> 1. Use test lists.   You can keep lists identifying which tests are valid in each version, and then
> specify files containing those lists on the command line with the "@file" mechanism
>
> 2. Use exclude lists.  You can keep lists identifying which tests are /not/ valid in each version,
> and then specify files containing those lists on the command line with the "-exclude file" mechanism
>

Both above have the same maintenance burden as various copies.

> 3. Keywords. You can tag individual tests with any keywords you choose to use, as long as you also
> declare the overall set of valid keywords in the TEST.ROOT file.   You can filter the set of tests
> to be executed with the "-k expr" command line option.

Yes keywords are pretty close, and were elaborated. However.. needs some settings on launcher side, 
and so didn't fit.

>
> 4. @requires. This is a relatively new feature. You can put a declarative tag in a test description

Ok I did not know this one.
> of the form "@requires expr" where there is a variety of terminals, of which jdk.version is but
> one.   So you could add a line like
>      @requires  jdk.version <=1.6 || jdk.version >1.7 || jdk.version = "1.4.2"
> Not as compact as your proposal, but it exists today.


Well, those looks awesome. I will definitely try. I guess they will work as desired.

Thank you very much for your time and complex reply.

Best regards from CZ
   J.

>
> -- Jon
>
> On 03/26/2015 12:46 AM, Jiri Vanek wrote:
>> Hello!
>>
>> I'm maintaining few thousand tests for various jdks (generally jdk 1.5-jdk 9) . However, current
>> workarounds about  which test is valid for which jdk are insufficient.
>>
>> I came to conclusion, that some kind of @validfor-jdk-version or @validfor or @ignoredfor or
>> enhancing of @ignore would solve my issues.
>>
>> Currentl I'm inklining to @validfor  or @validfor-version as (currently (with this 7years
>> history)) it is not necessary to distinguish anything else then java.version. My suggested format
>> will be white-chars separated list of conditions with control words of = < > <= >= followed  by
>> version string and with evaluation based on version sort and on or between predicates
>>
>> So eg:
>> @validfor  <=1.6 >1.7 =1.4.2
>>
>> If java.version  will not be  valid for any of predicate then the test will be handled in same way
>> as if @ignore is  specified. If it is valid, or annotation missing at all, test will be normally
>> proceed.
>>
>> Do you think enhancement like this is acceptable?
>>
>> If so ,I will prepare webrew soon, and will post it for review.
>>
>> Thank you!
>>
>>  J.
>