RFR: 8344942: Template-Based Testing Framework [v7]
Christian Hagedorn
chagedorn at openjdk.org
Wed May 7 09:13:34 UTC 2025
On Tue, 1 Apr 2025 07:18:45 GMT, Emanuel Peter <epeter at openjdk.org> wrote:
>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>>
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>>
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>>
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>>
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>>
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>>
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>>
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>>
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
>
> typo
Thanks for the updates in the meantime!
Just some random comments here and there when browsing through the code and trying to grasp what is included everywhere. I will make more passes later :-)
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 59:
> 57: * string with hashtag {@code #} "holes" that are then replaced by the template arguments and the
> 58: * {@link #let} definitions.
> 59: *
Should we also insert `<p>` between the snippets? Same further down at method comments.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 129:
> 127: * );
> 128: *
> 129: * // Use the template with one arguments, and render it to a String.
Suggestion:
* // Use the template with one argument, and render it to a String.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 422:
> 420: * Multi-line string
> 421: * """,
> 422: * "normal string ", Integer.valueOf(3), Float.valueOf(1.5f),
Is `valueOf()` required or would it auto-box?
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 428:
> 426: * }
> 427: *
> 428: * @param tokens A list of tokens, which can be {@link String}s,boxed primitive types
Suggestion:
* @param tokens A list of tokens, which can be {@link String}s, boxed primitive types
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 492:
> 490: * var template = Template.make("a", (Integer a) -> let("b", a * 2, (Integer b) -> body(
> 491: * """
> 492: * System.out.prinln("Use a and b with hashtag replacement: #a and #b");
Suggestion:
* System.out.println("Use a and b with hashtag replacement: #a and #b");
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 494:
> 492: * System.out.prinln("Use a and b with hashtag replacement: #a and #b");
> 493: * """,
> 494: * "System.out.println(\"Use a and b as capture variables:\" + a + " and " + b + ");\n"
Suggestion:
* "System.out.println("Use a and b as capture variables:"" + a + " and " + b + ");\n"
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 539:
> 537: * """
> 538: * System.out.println("Currently at depth #depth with fuel #fuel");
> 539: * """
Suggestion:
* """,
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java line 141:
> 139: """,
> 140: // Call the testTemplate for each type and operator, generating a
> 141: // list of list of TemplateWithArgs:
lists?
Suggestion:
// list of lists of TemplateWithArgs:
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java line 151:
> 149: ));
> 150:
> 151: // For each type, we chose a list of operators that do not throw exceptions.
Suggestion:
// For each type, we choose a list of operators that do not throw exceptions.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java line 159:
> 157: new Type("long", () -> GEN_LONG.next(), List.of("+", "-", "*", "&", "|", "^")),
> 158: new Type("float", () -> GEN_FLOAT.next(), List.of("+", "-", "*", "/")),
> 159: new Type("double", () -> GEN_DOUBLE.next(), List.of("+", "-", "*", "/"))
You can directly use `GEN_X::next` instead of `() -> GEN_X.next()`.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java line 162:
> 160: );
> 161:
> 162: // Use the template with one arguments, and render it to a String.
Suggestion:
// Use the template with one argument and render it to a String.
-------------
PR Review: https://git.openjdk.org/jdk/pull/24217#pullrequestreview-2820964558
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077181100
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077179114
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077177786
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077168801
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077184218
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077184500
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077174853
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077153706
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077154003
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077158958
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2077159333
More information about the hotspot-compiler-dev
mailing list