RFR: 7903659 Verify Problemlist contents

[Ludvig Janiuk]

On Wed, 21 Feb 2024 15:29:40 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> This change introduces the flag `-verifyexclude` which will run additional checks on the problemlists passes through `-exclude`. If any of the checks fails, jtreg prints a helpful message and refuses to start running tests.
>
> It's good to have such a verification in `jtreg`/`jtharness` soon.
> 
> I wonder whether an existing option should be (re-) used to trigger the verification process. `jtreg.jar --help verify` prints:
> 
> General Options
> 
>     -c | -check     Verify correctness of test descriptions. Does NOT run tests.
>     -xml | --xml:verify
>                     Create ant/junit xml files into the workDir. Optionally
>                     verify if the file is well-formed.
> 
> 
> Perhaps `-c | -check` can be amended to also verify the correctness of _exlude files_ and maybe later _match files_, too?
> 
> Test Selection Options
>                 These options can be used to refine the set of tests to be
>                 executed.
> 
>     -match:<file>   Provide a file specifying tests that can be run (inverse of
>                     -exclude)

@sormuras So, having looked at -check: it "DOES NOT" run the tests. But I intend problemlist verification to be run as part of testing, e.g. in CI's. So I think this warrants existing outside of -check.

I agree -check should perhaps be extented to cover problemlists too, but that is not the scope of this PR.

Including -match seems like a very reasonable thing to add to this PR.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/181#issuecomment-1985675620