RFR: 8344942: Template-Based Testing Framework [v80]

Wed Jun 4 12:28:50 UTC 2025

On Tue, 3 Jun 2025 15:57:32 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>> 
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>> 
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>> 
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>> 
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>> 
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>> 
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>> 
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>> 
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix whitespaces from applied suggestion

Looks good, I just have a few language suggestions. Thanks for driving this Emanuel!

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 248:

> 246:  *         // The count is still not different to "c1".
> 247:  *         let("c3", dataNames(MUTABLE).exactOf(someType).count()),
> 248:  *         // We nest a Template. This creats a TemplateToken, which is later evaluated.

Suggestion:

 *         // We nest a Template. This creates a TemplateToken, which is later evaluated.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 552:

> 550:      *
> 551:      * <p>
> 552:      * Here an example with template arguments {@code 'a'} and {@code 'b'}, captured once as string names

Suggestion:

     * Here is an example with template arguments {@code 'a'} and {@code 'b'}, captured once as string names

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 596:

> 594:     /**
> 595:      * Creates a {@link TemplateBody} from a list of tokens, which can be {@link String}s,
> 596:      * boxed primitive types (e.g. {@link Integer} or auto-boxed {@code int}), any {@link Token},

For correct javadoc description generation in `Template.html`:

Suggestion:

     * boxed primitive types (for example {@link Integer} or auto-boxed {@code int}), any {@link Token},

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 627:

> 625:      *
> 626:      * <p>
> 627:      * Here an example where a Template creates a local variable {@code 'var'},

Suggestion:

     * Here is an example where a Template creates a local variable {@code 'var'},

test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 277:

> 275:                     // Fourth Template use with template2, no use of dollar, so
> 276:                     // no "_4" shows up in the generated code. Internally, it
> 277:                     // calls template1, shich is the fifth Template use, with

Suggestion:

                    // calls template1, which is the fifth Template use, with

test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 832:

> 830: 
> 831:     // Having defined these helper methods, let us start with the first example.
> 832:     // You should start reading this example bottum-up, starting at

Suggestion:

    // You should start reading this example bottom-up, starting at

-------------

Marked as reviewed by rcastanedalo (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/24217#pullrequestreview-2896428767
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126413316
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126436853
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126427705
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126435300
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126407483
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2126410019