RFC: jtreg IntelliJ plugin: run individual test methods

[Jorn Vernee]

Unfortunately without support for parameter type names, JUnit test 
methods that accept parameters will not work at all. i.e. it is not just 
about overload selection. If the parameter types are not specified, 
JUnit will look for a method taking no parameters.

Note, though, that I'm not just using a string-based heuristic to derive 
the binary name. Intellij actually has facilities to resolve a source 
name + resolution scope into a 'PsiClass'. The issue is just that there 
is no straightforward way to get a binary name from a PsiClass. So, for 
nested classes I am reconstructing the binary name manually from the 
fully qualified name, and replacing dots with dollars in the name based 
on the actual enclosing classes of the type.

Jorn

On 7-1-2025 18:24, Maurizio Cimadamore wrote:
> Hi Jorn,
> your extension to the plugin seems a fine idea - it would be nice to 
> be able to run specific methods through the plugin.
>
> IMHO, adding parameter types might be a step too far - the general 
> problem of "predicting" which binary name a source class name might 
> have is rather hard, and it amounts at heuristics (e.g. there's more 
> to it than just replacing dots with dollars, as you also take into 
> account as to whether other class declared with a dollar in their name 
> (!!) are present).
>
> I wonder if a sensible first step would be to just support the same 
> logic that jtreg runners also support -- e.g. just a method name. 
> After all, it is extremely unlikely to have overloaded test methods. 
> E.g. it seems to me that we're in a case where we can get almost all 
> the benefit at a fraction of the complexity.
>
> What do you think?
>
> Maurizio
>
> On 07/01/2025 17:05, Jorn Vernee wrote:
>> Hello,
>>
>> Jtreg has for a while now had the ability to attach a query string to 
>> the name of a test, which then gets used by the JUnit or TestNG 
>> runner to select a particular test method to run.
>>
>> The intellij plugin currently doesn't have support for specifying a 
>> query string, or running individual test methods, so I've put 
>> together a patch that adds that support [1]. This support has two 
>> parts: add a 'Query String' field to the jtreg test configuration 
>> window in intellij, and automatically populate the query string when 
>> creating a test configuration from a test method in a source file.
>>
>> Most of this is pretty straightforward. I did run into a small issue: 
>> when a method takes parameters, junit requires the parameter types 
>> the be specified as binary names. It takes some work to derive the 
>> binary name of the parameter types from the source AST representation 
>> exposed by intellij to the plugin.
>>
>> Additionally, I had to amend the format of the query strings accepted 
>> by the TestNG and JUnit runners to accept a trailing list of 
>> parameter type names. The new format of the query string is:
>>
>> <methodName>[-<parameterTypeName>[...,<parameterTypeName>]]
>>
>> Does this seem like a good approach?
>>
>> Future enhancements could include:
>>
>> 1.) displaying individual test methods as results in the intellij UI. 
>> The problem with doing this is currently that jtreg does not forward 
>> the results from JUnit/TestNG to the observer the plugin attaches to 
>> jtreg. Adding support for this seems straightforward enough. 
>> Potentially it would also be useful to 'itemize' other steps that a 
>> jtreg test takes, such as compiling, or individual @run tags.
>> 2.) for parameterized tests, the TestNG runner that comes with 
>> intellij shows each case (i.e. combination of paramters) individually 
>> as a result in the UI, and allows re-running just a single case 
>> through a unique ID. I think JUnit has similar support for this. We 
>> could add a similar feature to the jtreg plugin.
>>
>> However, I thought what I have currently is useful enough in 
>> isolation to start the discussion on this.
>>
>> Jorn
>>
>> [1]: 
>> https://github.com/openjdk/jtreg/compare/master...JornVernee:jtreg:RunMethods
>>