RFR: 8344942: Template-Based Testing Framework [v2]

[Galder Zamarreño]

On Thu, 27 Mar 2025 05:19:43 GMT, Galder Zamarreño <galder at openjdk.org> wrote:

>> Emanuel Peter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Merge branch 'master' into JDK-8344942-TemplateFramework-v3
>>  - fix tests
>>  - whitespace
>>  - whitespace
>>  - fix whitespace
>>  - JDK-8344942
>
> Looks great though I'm not too familiar with the code to be able to do a reasonable review, but I had a question:
> 
> Have you got any practical use case that can show where you've used this and show what it takes to build such a use case? `VectorReduction2` or similar type of microbenchmarks would be great to see auto generated using this? 
> 
> The reason I ask this is because I feel that something that is missing in this PR is a small practical use case where this framework is put into action to actually generate some jtreg/IR/microbenchmark test and see how it runs as part of the CI in the PR. WDYT?

> @galderz Thanks for your questions!
> 
> > Looks great though I'm not too familiar with the code to be able to do a reasonable review
> 
> Well the code is all brand new, so really anybody could review ;)

Right, what I meant is that developers that have past history with this work will be able to provide a more thorough review :)

> > Have you got any practical use case that can show where you've used this and show what it takes to build such a use case?
> 
> I actually have a list of experiments in this branch (it is linked in the PR description): #23418 Some of them use the IR framework, though for now just as a testing harness, not for IR rules. Generating IR rules automatically requires quite a bit of logic... I hope that is satisfactory for now?

Yes it is. Sorry I missed the linked PR when I read the description. The examples there look great, it's what I was looking for.

> Ah, but there was this test: `test/hotspot/jtreg/compiler/loopopts/superword/TestDependencyOffsets.java` I did now not refactor it, but it would not be too hard to see how to use the Templates for it. And I do generate IR rules in that one. I don't super like just refactoring old tests... there is always a risk of breaking it and then coverage is worse than before...
> 
> > The reason I ask this is because I feel that something that is missing in this PR is a small practical use case where this framework is put into action to actually generate some jtreg/IR/microbenchmark test and see how it runs as part of the CI in the PR. WDYT?
> 
> I also have a few tests in this PR that just generate regular JTREG tests, without the IR framework, did you see those?

Yeah I've seen them now.

> 
> > VectorReduction2 or similar type of microbenchmarks would be great to see auto generated using this?
> 
> I don't yet have a solution for microbenchmarks. It's mostly an issue of including the `test/hotspot/jtreg/compiler/lib` path... And I fear that JMH requires all benchmark code to be compiled beforehand, and not dynamically as I do with the class loader. But maybe there is a solution for that.
> 
> The patch is already quite large, and so I wanted to just publish the basic framework. Do you think that is ok?

Yeah sure.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24217#issuecomment-2765342389