RFC: jtreg IntelliJ plugin: run individual test methods

[Jorn Vernee]

Hello,

Jtreg has for a while now had the ability to attach a query string to 
the name of a test, which then gets used by the JUnit or TestNG runner 
to select a particular test method to run.

The intellij plugin currently doesn't have support for specifying a 
query string, or running individual test methods, so I've put together a 
patch that adds that support [1]. This support has two parts: add a 
'Query String' field to the jtreg test configuration window in intellij, 
and automatically populate the query string when creating a test 
configuration from a test method in a source file.

Most of this is pretty straightforward. I did run into a small issue: 
when a method takes parameters, junit requires the parameter types the 
be specified as binary names. It takes some work to derive the binary 
name of the parameter types from the source AST representation exposed 
by intellij to the plugin.

Additionally, I had to amend the format of the query strings accepted by 
the TestNG and JUnit runners to accept a trailing list of parameter type 
names. The new format of the query string is:

<methodName>[-<parameterTypeName>[...,<parameterTypeName>]]

Does this seem like a good approach?

Future enhancements could include:

1.) displaying individual test methods as results in the intellij UI. 
The problem with doing this is currently that jtreg does not forward the 
results from JUnit/TestNG to the observer the plugin attaches to jtreg. 
Adding support for this seems straightforward enough. Potentially it 
would also be useful to 'itemize' other steps that a jtreg test takes, 
such as compiling, or individual @run tags.
2.) for parameterized tests, the TestNG runner that comes with intellij 
shows each case (i.e. combination of paramters) individually as a result 
in the UI, and allows re-running just a single case through a unique ID. 
I think JUnit has similar support for this. We could add a similar 
feature to the jtreg plugin.

However, I thought what I have currently is useful enough in isolation 
to start the discussion on this.

Jorn

[1]: 
https://github.com/openjdk/jtreg/compare/master...JornVernee:jtreg:RunMethods