RFR: 8344942: Template-Based Testing Framework

[Galder Zamarreño]

On Tue, 25 Mar 2025 08:31:36 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

> **Goal**
> We want to generate Java source code:
> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
> 
> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
> 
> **How to get started**
> When reviewing, please start by looking at:
> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
> 
> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
> 
> And then for a "tutorial", look at:
> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
> 
> It shows these features:
> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
> - The use of recursive templates, and `fuel` to limit the recursion.
> - `Name`s: useful to register field and variable names in code scopes.
> 
> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
> 
> For a better experience, you may want to generate the `javadocs`:
> `javadoc -sourcepath test/hotspot/jtreg:./test/lib compiler.lib.template_framework`
> 
> **History**
> @TobiHartmann and I have played with code generators for a while, and have had the dream of doing that in a more principled way. And to hopefully...

Looks great though I'm not too familiar with the code to be able to do a reasonable review, but I had a question:

Have you got any practical use case that can show where you've used this and show what it takes to build such a use case? `VectorReduction2` or similar type of microbenchmarks would be great to see auto generated using this? 

The reason I ask this is because I feel that something that is missing in this PR is a small practical use case where this framework is put into action to actually generate some jtreg/IR/microbenchmark test and see how it runs as part of the CI in the PR. WDYT?

-------------

PR Review: https://git.openjdk.org/jdk/pull/24217#pullrequestreview-2719765351