RFC: jtreg IntelliJ plugin: run individual test methods

[Maurizio Cimadamore]

Hi Jorn,
your extension to the plugin seems a fine idea - it would be nice to be 
able to run specific methods through the plugin.

IMHO, adding parameter types might be a step too far - the general 
problem of "predicting" which binary name a source class name might have 
is rather hard, and it amounts at heuristics (e.g. there's more to it 
than just replacing dots with dollars, as you also take into account as 
to whether other class declared with a dollar in their name (!!) are 
present).

I wonder if a sensible first step would be to just support the same 
logic that jtreg runners also support -- e.g. just a method name. After 
all, it is extremely unlikely to have overloaded test methods. E.g. it 
seems to me that we're in a case where we can get almost all the benefit 
at a fraction of the complexity.

What do you think?

Maurizio

On 07/01/2025 17:05, Jorn Vernee wrote:
> Hello,
>
> Jtreg has for a while now had the ability to attach a query string to 
> the name of a test, which then gets used by the JUnit or TestNG runner 
> to select a particular test method to run.
>
> The intellij plugin currently doesn't have support for specifying a 
> query string, or running individual test methods, so I've put together 
> a patch that adds that support [1]. This support has two parts: add a 
> 'Query String' field to the jtreg test configuration window in 
> intellij, and automatically populate the query string when creating a 
> test configuration from a test method in a source file.
>
> Most of this is pretty straightforward. I did run into a small issue: 
> when a method takes parameters, junit requires the parameter types the 
> be specified as binary names. It takes some work to derive the binary 
> name of the parameter types from the source AST representation exposed 
> by intellij to the plugin.
>
> Additionally, I had to amend the format of the query strings accepted 
> by the TestNG and JUnit runners to accept a trailing list of parameter 
> type names. The new format of the query string is:
>
> <methodName>[-<parameterTypeName>[...,<parameterTypeName>]]
>
> Does this seem like a good approach?
>
> Future enhancements could include:
>
> 1.) displaying individual test methods as results in the intellij UI. 
> The problem with doing this is currently that jtreg does not forward 
> the results from JUnit/TestNG to the observer the plugin attaches to 
> jtreg. Adding support for this seems straightforward enough. 
> Potentially it would also be useful to 'itemize' other steps that a 
> jtreg test takes, such as compiling, or individual @run tags.
> 2.) for parameterized tests, the TestNG runner that comes with 
> intellij shows each case (i.e. combination of paramters) individually 
> as a result in the UI, and allows re-running just a single case 
> through a unique ID. I think JUnit has similar support for this. We 
> could add a similar feature to the jtreg plugin.
>
> However, I thought what I have currently is useful enough in isolation 
> to start the discussion on this.
>
> Jorn
>
> [1]: 
> https://github.com/openjdk/jtreg/compare/master...JornVernee:jtreg:RunMethods
>