RFR: CODETOOLS-7903267: Support executing a single test method in a JUnit class

[Iris Clark]

On Fri, 9 Sep 2022 22:38:50 GMT, Jonathan Gibbons <jjg at openjdk.org> wrote:

> Please review a medium-sized patch to support new feature that provides a way to specify that only parts of a test should be executed.  For JUnit tests, this allows a single method in the test to be executed, but the underlying mechanism is more general than that and can be leveraged by any framework that supports running many test cases within an overall test.
> 
> The feature is enabled by using a new syntactic form on the command line, informally called _query syntax_: `path-to-test?string`. The syntax is based on URL syntax, but note that unlike URLs, the query component comes at the end, and not before any fragment component. In other words, the full syntax is `path#id?string`. Generally, this reflects the hierarchical structure:   a path to a file, a test description within the file, and a string to specify part of the test to be run.
> 
> In terms of implementation, the "heavy lifting" is done in `TestManager`, to track the _file, id, query_ tuple, as compared to the previous _file, id_.  New abstractions `TestSpec` and `GroupSpec` are introduced, to replace stylistic use of simple strings.
> 
> The underlying JT Harness does not know (or need to know) about query components and so is unchanged. But the back end `RegressionScript` used to execute each test _does_ need to know the query component, and so the information about which tests involve a query component is tunneled through the `RegressionParameters` in a side channel, embodied by a new resource in the object. In itself, the `RegressionScript` has little to do with the query component, it is simply passed down to the actions of the test in a new `test.query` property or `TESTQUERY` environment variable.
> 
> `JUnitRunner` is updated to check for the new `test.query` property: if set, it is used as the name of a test method to be run. Note: a JUnit exception will be thrown and the test will fail if the method is not found. The exception does contain enough useful information to diagnose the issue, but we may want to detect and report the issue in a more friendly form in a followup change.
> 
> Finally, new jtreg self-tests are added to exercise the new feature.

Fantastic feature to allow developers to conveniently run specific tests during development!

I appreciate the addition documentation (javadoc and comments).

Marked as reviewed by iris (Reviewer).

src/share/classes/com/sun/javatest/regtest/config/TestManager.java line 220:

> 218:          * It does not include the query.
> 219:          *
> 220:          * @return

Intentionally empty @return?

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 1104:

> 1102:             }
> 1103:         });
> 1104: //        testManager.addTestFiles(testFileArgs);

Did you intend to retain these commented lines?

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 2221:

> 2219:     /**
> 2220:      * Returns whether Windows Subsystem for Linux may be available,
> 2221:      * by examining to see whether wsl.exe can be found on the path.

I think that "to see" is redundant.

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 2277:

> 2275:     private String priorStatusValuesArg;
> 2276:     private Path reportDirArg;
> 2277: //    public List<String> testGroupArgs = new ArrayList<>();

Intentionally left?

src/share/doc/javatest/regtest/faq.md line 446:

> 444: If you specify `?`_string_ after the name of a test, the _string_ will be
> 445: passed down to the test, for the test to filter the parts of the test to be
> 446: executed. For any tests executed by JUnit Platform, the string is interpreted

Last one used?  (similar to com/sun/javatest/regest/tool/i18n.prpoerties line 380-381)

src/share/doc/javatest/regtest/faq.md line 492:

> 490: a test should be run on any particular system.
> 491: 
> 492: ### How do I specify to run a single test method in a JUnit test?

Is "specify to" required for parallelism between this and other questions?  If not, I'd drop those two words.

test/junitQueryTest/JUnitQueryTest.gmk line 8:

> 6: # under the terms of the GNU General Public License version 2 only, as
> 7: # published by the Free Software Foundation.  Oracle designates this
> 8: # particular file as subject to the "Classpath" exception as provided

The use of GPL+CPE seems unusual for a test.

test/junitQueryTest/JUnitQueryTest.gmk line 233:

> 231: 
> 232: TESTS.jtreg += $(BUILDTESTDIR)/JUnitQueryTest.invalidQuery.ok
> 233: 

I'm sure there are others, but behavior when the query is associated with a valid test springs to mind, i.e.

Negative test: a/b/c/Test1.java?m14 

Also test: a/b/c/Test1.java?m14 and a/b/c/Test.java

-------------

PR: https://git.openjdk.org/jtreg/pull/115