RFR: 8344942: Template-Based Testing Framework [v29]
Christian Hagedorn
chagedorn at openjdk.org
Fri May 16 09:13:13 UTC 2025
On Fri, 16 May 2025 07:02:50 GMT, Emanuel Peter <epeter at openjdk.org> wrote:
>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>>
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>>
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>>
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>>
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>>
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>>
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>>
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>>
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
>
> Elaborate why there is only one Renderer
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 53:
> 51: *
> 52: * <p>
> 53: * Once we rendered the source code to a {@link String}, we can compile it with the {@code CompileFramework}.
You could also link the `CompileFramework`:
Suggestion:
* Once we rendered the source code to a {@link String}, we can compile it with the {@link CompileFramework}.
This would require an additional:
import compiler.lib.compile_framework.CompileFramework;
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 86:
> 84: *
> 85: * <p>
> 86: * To get an executable test, we define a Template that produces a class body with a main method. The Template
Suggestion:
* To get an executable test, we define a {@link Template} that produces a class body with a main method. The Template
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 88:
> 86: * To get an executable test, we define a Template that produces a class body with a main method. The Template
> 87: * takes a list of types, and calls the {@code testTemplate} defined above for each type and operator. We use
> 88: * the {@code TestFramework} to call our {@code @Test} methods.
For the first mention, you can also add a link:
Suggestion:
* the {@link TestFramework} to call our {@code @Test} methods.
Also requires:
import compiler.lib.ir_framework.TestFramework;
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 111:
> 109: * """,
> 110: * // Call the testTemplate for each type and operator, generating a
> 111: * // list of lists of (Template) Tokens:
Since we eventually settled on exposing `TemplateToken`, I guess you can name it as such here:
Suggestion:
* // list of lists of TemplateToken:
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 124:
> 122: * <p>
> 123: * Finally, we generate the list of types, and pass it to the class template:
> 124: *
Suggestion:
*
* <p>
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 143:
> 141: * <strong>Details:</strong>
> 142: * <p>
> 143: * A Template can have zero or more arguments. A template can be created with {@code make} methods like
Suggestion:
* A {@link Template} can have zero or more arguments. A template can be created with {@code make} methods like
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 145:
> 143: * A Template can have zero or more arguments. A template can be created with {@code make} methods like
> 144: * {@link Template#make(String, Function)}. For each number of arguments there is an implementation
> 145: * (e.g. {@code Template.TwoArgs} for two arguments). This allows the use of Generics for the
Suggestion:
* (e.g. {@link Template.TwoArgs} for two arguments). This allows the use of Generics for the
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 146:
> 144: * {@link Template#make(String, Function)}. For each number of arguments there is an implementation
> 145: * (e.g. {@code Template.TwoArgs} for two arguments). This allows the use of Generics for the
> 146: * Template argument types, i.e. the Template arguments can be type checked.
Suggestion:
* Template argument types which enables type checking of the Template arguments.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 151:
> 149: * A {@link Template} can be rendered to a {@link String} (e.g. {@link Template.ZeroArgs#render()}).
> 150: * Alternatively, we can generate a {@link Token} (e.g. {@link Template.ZeroArgs#asToken()}),
> 151: * and use the {@link Token} inside another {@link Template#body}.
Maybe we can mention here that it's actually a `TemplateToken`:
Suggestion:
* Alternatively, we can generate a {@link Token} (more specifically, a {@link TemplateToken}) with {@code asToken()}
* (e.g. {@link Template.ZeroArgs#asToken()}), and use the {@link Token} inside another {@link Template#body}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 168:
> 166: *
> 167: * <p>
> 168: * A {@link TemplateToken} can not just be used in {@link Template#body}, but but it can also be
Suggestion:
* A {@link TemplateToken} can not just be used in {@link Template#body}, but it can also be
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 178:
> 176: * with a certain amount of {@link #fuel}, which is decreased at each Template nesting by a certain amount
> 177: * (can be changed with {@link #setFuelCost}). Recursive templates are supposed to terminate once the {@link #fuel}
> 178: * is depleted (i.e. reaches zero).
I suggest to also mention the default fuel and how to change that once you start rendering. Maybe something like this?
Suggestion:
* with a certain amount of {@link #fuel} (default: 100, see {@link #DEFAULT_FUEL}), which is decreased at each
* Template nesting by a certain amount (default: 10, see {@link #DEFAULT_FUEL_COST}). The default fuel for a
* template can be changed when we {@code render()} it (e.g. {@link ZeroArgs#render(float)}) and the default
* fuel cost with {@link #setFuelCost}) when defining the {@link #body(Object...)}. Recursive templates are
* supposed to terminate once the {@link #fuel} is depleted (i.e. reaches zero).
```
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092455409
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092459709
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092462635
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092470827
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092472111
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092474294
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092476166
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092477901
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092494798
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092498957
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092516132
More information about the hotspot-compiler-dev
mailing list