[rfc] make test ignored or valid only for specific JDK

[Jiri Vanek]

On 03/30/2015 01:25 PM, Jiri Vanek wrote:
> On 03/26/2015 07:18 PM, Jonathan Gibbons wrote:
>> Jiri,
>>
>> I presume you are talking about a set of tests that is distinct from any of the jtreg test suites in
>
> Sure :)
>
>> an OpenJDK forest, such as langtools/test, jdk/test, etc.   Those test suites are tied to the code
>> in the overall forest, and so are inherently versioned without any additional notation.
>>
>> There are a variety of existing mechanisms you could use to address this problem, without adding any
>> new features.
>>
>> 1. Use test lists.   You can keep lists identifying which tests are valid in each version, and then
>> specify files containing those lists on the command line with the "@file" mechanism
>>
>> 2. Use exclude lists.  You can keep lists identifying which tests are /not/ valid in each version,
>> and then specify files containing those lists on the command line with the "-exclude file" mechanism
>>
>
> Both above have the same maintenance burden as various copies.
>
>> 3. Keywords. You can tag individual tests with any keywords you choose to use, as long as you also
>> declare the overall set of valid keywords in the TEST.ROOT file.   You can filter the set of tests
>> to be executed with the "-k expr" command line option.
>
> Yes keywords are pretty close, and were elaborated. However.. needs some settings on launcher side,
> and so didn't fit.
>
>>
>> 4. @requires. This is a relatively new feature. You can put a declarative tag in a test description
>
> Ok I did not know this one.
>> of the form "@requires expr" where there is a variety of terminals, of which jdk.version is but
>> one.   So you could add a line like
>>      @requires  jdk.version <=1.6 || jdk.version >1.7 || jdk.version = "1.4.2"
>> Not as compact as your proposal, but it exists today.


hmhm. It does not seem to be working as "advertised":

  * @requires jdk.version <= 1.7

leads to :
test result: Error. Parse Exception: Syntax error in @requires expression: unrecognized character: `.'

  * @requires jdk.version <= 7
leads to
test result: Error. Error evaluating expression: invalid numeric value: 1.8

Is it expected bahviour? I tried also with various spaces/nospace....


J.
>
>
> Well, those looks awesome. I will definitely try. I guess they will work as desired.
>
> Thank you very much for your time and complex reply.
>
> Best regards from CZ
>    J.
>
>>
>> -- Jon
>>
>> On 03/26/2015 12:46 AM, Jiri Vanek wrote:
>>> Hello!
>>>
>>> I'm maintaining few thousand tests for various jdks (generally jdk 1.5-jdk 9) . However, current
>>> workarounds about  which test is valid for which jdk are insufficient.
>>>
>>> I came to conclusion, that some kind of @validfor-jdk-version or @validfor or @ignoredfor or
>>> enhancing of @ignore would solve my issues.
>>>
>>> Currentl I'm inklining to @validfor  or @validfor-version as (currently (with this 7years
>>> history)) it is not necessary to distinguish anything else then java.version. My suggested format
>>> will be white-chars separated list of conditions with control words of = < > <= >= followed  by
>>> version string and with evaluation based on version sort and on or between predicates
>>>
>>> So eg:
>>> @validfor  <=1.6 >1.7 =1.4.2
>>>
>>> If java.version  will not be  valid for any of predicate then the test will be handled in same way
>>> as if @ignore is  specified. If it is valid, or annotation missing at all, test will be normally
>>> proceed.
>>>
>>> Do you think enhancement like this is acceptable?
>>>
>>> If so ,I will prepare webrew soon, and will post it for review.
>>>
>>> Thank you!
>>>
>>>  J.
>>
>