RFR: 7903930: Support running individual parameterized tests and @Nested test classes [v4]

[Jorn Vernee]

On Tue, 14 Jan 2025 18:03:32 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

>> See the JBS issue for an extended problem description.
>> 
>> This PR adds support for running individual parameterized JUnit test methods, as well as nested test classes, by extending the format that a query string can have. The new proposed format is as follows:
>> 
>> 
>> <class name>[::<method name>([<param type>[...,<param type>]])]
>> 
>> 
>> For example, running a method `foo` that takes no parameters would be done using `<test name>?TestClass::foo()`. If `foo` has parameters, then at least for JUnit tests they have to be specified as a comma separated list of binary names between the parentheses. For example:
>> 
>> 
>> TestDowncallScope::testDowncall(int,java.lang.String,CallGeneratorHelper$Ret,java.util.List,java.util.List)
>> 
>> 
>> The delimiting characters I choose here `::` and `(...)` have to be quoted when passed through most shells. If this is deemed to be an issue, we could also change the separators for class name/method name/parameters to simple hyphens. Although, in cases where a `$` sign appears in the binary name of a nested class, quoting would be needed either way. (I would personally choose to always quote the argument, just to be sure, so I don't mind using `::` and `(...)`).
>> 
>> This support is intended to be used by the intellij plugin to support running individual methods and nested classes (see https://github.com/openjdk/jtreg/pull/240).
>> 
>> Testing: I've added additional tests for the new cases (as far as I know, TestNG does not support nested test classes though). I've also been using a PoC of this feature as the basis for similar support in the intellij plugin that I'm working on.
>
> Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - update FAQ
>  - cleanup test

Some results of offline discussion:

- We'd like to rely on JUnit's builtin support for parsing these queries, so we don't have to implement our own parsing. This also means we can use any JUnit selector (such as uid, to run individual cases of a parameterized test)
- This means that we don't support the extended format for TestNG tests. But, this is okay, since TestNG doesn't support nested test classes any way, and we don't need to specify the parameter types to select a parameterized method
- To use the junit selector support, the query string is prefixed with `junit-select:`, and then an identifier such as the ones listed [here](https://junit.org/junit5/docs/current/user-guide/#running-tests-discovery-selectors) is used as a suffix.
- The current format, with just the method name, continues to work as well.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/241#issuecomment-2590752416