RFR: 8344942: Template-Based Testing Framework [v6]

[Emanuel Peter]

On Mon, 31 Mar 2025 07:21:15 GMT, Galder Zamarreño <galder at openjdk.org> wrote:

>> Looks great though I'm not too familiar with the code to be able to do a reasonable review, but I had a question:
>> 
>> Have you got any practical use case that can show where you've used this and show what it takes to build such a use case? `VectorReduction2` or similar type of microbenchmarks would be great to see auto generated using this? 
>> 
>> The reason I ask this is because I feel that something that is missing in this PR is a small practical use case where this framework is put into action to actually generate some jtreg/IR/microbenchmark test and see how it runs as part of the CI in the PR. WDYT?
>
>> @galderz Thanks for your questions!
>> 
>> > Looks great though I'm not too familiar with the code to be able to do a reasonable review
>> 
>> Well the code is all brand new, so really anybody could review ;)
> 
> Right, what I meant is that developers that have past history with this work will be able to provide a more thorough review :)
> 
>> > Have you got any practical use case that can show where you've used this and show what it takes to build such a use case?
>> 
>> I actually have a list of experiments in this branch (it is linked in the PR description): #23418 Some of them use the IR framework, though for now just as a testing harness, not for IR rules. Generating IR rules automatically requires quite a bit of logic... I hope that is satisfactory for now?
> 
> Yes it is. Sorry I missed the linked PR when I read the description. The examples there look great, it's what I was looking for.
> 
>> Ah, but there was this test: `test/hotspot/jtreg/compiler/loopopts/superword/TestDependencyOffsets.java` I did now not refactor it, but it would not be too hard to see how to use the Templates for it. And I do generate IR rules in that one. I don't super like just refactoring old tests... there is always a risk of breaking it and then coverage is worse than before...
>> 
>> > The reason I ask this is because I feel that something that is missing in this PR is a small practical use case where this framework is put into action to actually generate some jtreg/IR/microbenchmark test and see how it runs as part of the CI in the PR. WDYT?
>> 
>> I also have a few tests in this PR that just generate regular JTREG tests, without the IR framework, did you see those?
> 
> Yeah I've seen them now.
> 
>> 
>> > VectorReduction2 or similar type of microbenchmarks would be great to see auto generated using this?
>> 
>> I don't yet have a solution for microbenchmarks. It's mostly an issue of including the `test/hotspot/jtreg/compiler/lib` path... And I fear that JMH requires all benchmark code to be compiled beforehand, and not dynamically as I do with the class loader. But maybe there is a solution for that.
>> 
>> The patch is already quite large, and so I wanted to just publish the basic framework. Do you think that is ok?
> 
> Yeah sure.

@galderz @chhagedorn I added an additional test now:
https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119

@chhagedorn I think I addressed all your suggestions :)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24217#issuecomment-2766280378