RFR: 8344942: Template-Based Testing Framework [v25]

Thu May 15 14:51:03 UTC 2025

On Thu, 15 May 2025 13:37:40 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>> 
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>> 
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>> 
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>> 
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>> 
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>> 
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>> 
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>> 
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

Thank you for the refactoring and your patience. I like the result and its simplicity a lot.

I found a few typos, but otherwise it looks excellent.

test/hotspot/jtreg/compiler/lib/template_framework/Renderer.java line 47:

> 45:     /**
> 46:      * There can be at most one Renderer instance at any time. This is to avoid that users accidentally
> 47:      * render templates to strings, rather than letting them all render together.

Suggestion:

     * There can be at most one Renderer instance at any time. This is to avoid users accidentally
     * rendering templates to separate strings, rather than letting them all render together.

I do not understand the original sentence. My suggestion reflects what I understood.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 255:

> 253:          *
> 254:          * @param a The value for the (first) argument.
> 255:          * @return The template its argument applied.

Suggestion:

         * @return The template with its argument applied.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 303:

> 301:          * @param a The value for the first argument.
> 302:          * @param b The value for the second argument.
> 303:          * @return The template all (two) arguments applied.

Suggestion:

         * @return The template with all (two) arguments applied.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 378:

> 376:          * @param b The value for the second argument.
> 377:          * @param c The value for the third argument.
> 378:          * @return The template all (three) arguments applied.

Suggestion:

         * @return The template with all (three) arguments applied.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 453:

> 451:      * @param <A> Type of the (first) argument.
> 452:      * @param arg0Name The name of the (first) argument for hashtag replacement.
> 453:      * @return An {@link Template} with one argument.

Suggestion:

     * @return A {@link Template} with one argument.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 672:

> 670: 
> 671:     /**
> 672:      * Weight the {@link Name}s for the specified {@link Name.Type}.

Suggestion:

     * Weigh the {@link Name}s for the specified {@link Name.Type}.

I think here you want the verb?

test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 142:

> 140:             // Optimal would have been Java String Templates, but since those do not
> 141:             // currently exist, we use hashtag replacements. These can be either
> 142:             // defined by capturing arguments as string names, or by a "let" definition,

Suggestion:

            // defined by capturing arguments as string names, or by using a "let" definition,

There is a verb missing here.

-------------

Marked as reviewed by mhaessig (Author).

PR Review: https://git.openjdk.org/jdk/pull/24217#pullrequestreview-2843790505
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091243038
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091261403
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091260895
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091259872
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091264291
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091233289
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2091295497