RFR: 8344942: Template-Based Testing Framework [v30]

Christian Hagedorn chagedorn at openjdk.org
Fri May 16 09:13:06 UTC 2025 

On Fri, 16 May 2025 08:03:55 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>> 
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>> 
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>> 
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>> 
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>> 
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>> 
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>> 
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>> 
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   apply offline suggestions by Christian

Some first comments for `Template`, will continue with the other files :-)

And thanks for bearing with us! I'm now also very happy about the naming and design. It was well worth to discuss the names and the design more in-depth. The result is very good now I think! :-)

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 229:

> 227: 
> 228:         /**
> 229:          * Renders the {@link Template} to {@link String}.

Suggestion:

         * Renders the {@link Template} to a {@link String}.

Could also be changed at the other `render()` methods.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 410:

> 408:          */
> 409:         public String render(float fuel, A a, B b, C c) {
> 410:             return new TemplateToken.ThreeArgs(this, a, b, c).render(fuel);

Suggestion:

            return new TemplateToken.ThreeArgs<>(this, a, b, c).render(fuel);

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 413:

> 411:         }
> 412:     }
> 413: 

Suggestion:

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 612:

> 610: 
> 611:     /**
> 612:      * Default amount of fuel for Template rendering. It guides the nesting depth of Templates.

Maybe add here how to change:
Suggestion:

     * Default amount of fuel for Template rendering. It guides the nesting depth of Templates. Can be changed when
     * rendering a template with {@code render(fuel)} (e.g. {@link ZeroArgs#render(float)}).

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 618:

> 616:     /**
> 617:      * The default amount of fuel spent per Template. It is subtracted from the current {@link #fuel} at every
> 618:      * nesting level, and once the {@link #fuel} reaches zero, the nesting is supposed to terminate.

Same here:
Suggestion:

     * nesting level, and once the {@link #fuel} reaches zero, the nesting is supposed to terminate. Can be changed
     * with {@link #setFuelCost(float)} inside {@link #body(Object...)}.

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 620:

> 618:      * nesting level, and once the {@link #fuel} reaches zero, the nesting is supposed to terminate.
> 619:      */
> 620:     public final static float DEFAULT_FUEL_COST = 10.0f;

Suggestion:

    float DEFAULT_FUEL = 100.0f;

    /**
     * The default amount of fuel spent per Template. It is subtracted from the current {@link #fuel} at every
     * nesting level, and once the {@link #fuel} reaches zero, the nesting is supposed to terminate.
     */
    float DEFAULT_FUEL_COST = 10.0f;

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 631:

> 629:      * Example of a recursive Template, which checks the remaining {@link #fuel} at every level,
> 630:      * and terminates if it reaches zero. It also demonstrates the use of {@link TemplateBinding} for
> 631:      * the recursive use of Templates. We {@link Template.OneArgs#render} with {@code 30} total fuel, and spend {@code 5} fuel at each recursion level.

Long line can be split:
Suggestion:

     * the recursive use of Templates. We {@link Template.OneArgs#render} with {@code 30} total fuel, 
     * and spend {@code 5} fuel at each recursion level.
     * 
     * <p>

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 640:

> 638:      *     System.out.println("Currently at depth #depth with fuel #fuel");
> 639:      *     """,
> 640:      *     (fuel() > 0) ? binding.get().asToken(depth + 1)

Missing `:`:
Suggestion:

     *     (fuel() > 0) ? binding.get().asToken(depth + 1) :

test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 675:

> 673:      * @return The token that performs the defining action.
> 674:      */
> 675:     static Token addName(Name name) {

I have not fully grasped the concept of these `Name`s, yet. The `generateWithNames()` example in `TestTutorial` is quite big and complex. Could you add there a simpler example to better understand this idea of names?

-------------

PR Review: https://git.openjdk.org/jdk/pull/24217#pullrequestreview-2845694798
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092554087
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092561787
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092562738
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092621482
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092625480
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092618666
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092637966
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092629499
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2092658089

More information about the hotspot-compiler-dev mailing list