RFR: 8344942: Template-Based Testing Framework [v55]
Roberto Castañeda Lozano
rcastanedalo at openjdk.org
Thu May 22 07:20:10 UTC 2025
On Wed, 21 May 2025 15:24:18 GMT, Emanuel Peter <epeter at openjdk.org> wrote:
>> **Goal**
>> We want to generate Java source code:
>> - Make it easy to generate variants of tests. E.g. for each offset, for each operator, for each type, etc.
>> - Enable the generation of domain specific fuzzers (e.g. random expressions and statements).
>>
>> Note: with the Template Library draft I was already able to find a [list of bugs](https://bugs.openjdk.org/issues/?jql=labels%20%3D%20template-framework%20ORDER%20BY%20created%20DESC%2C%20summary%20DESC).
>>
>> **How to get started**
>> When reviewing, please start by looking at:
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestSimple.java#L60-L76
>>
>> We have a Template with two arguments. They are typed (Integer and String). We then apply the arguments `template.withArgs(42, "7")`, producing a `TemplateWithArgs`. This can then be `render`ed to a String. And then that can be compiled and executed with the CompileFramework.
>>
>> Second, look at this advanced test:
>> https://github.com/openjdk/jdk/blob/77079807042fc5a3af04e0ccccad4ecd89e21cdb/test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestAdvanced.java#L102-L119
>>
>> And then for a "tutorial", look at:
>> `test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java`
>>
>> It shows these features:
>> - The `body` of a Template is essentially a list of `Token`s that are concatenated.
>> - Templates can be nested: a `TemplateWithArgs` is also a `Token`.
>> - We can use `#name` replacements to directly format values into the String. If we had proper String Templates in Java, we would not need this feature.
>> - We can use `$var` to make variable names unique: if we applied the same template twice, we would get variable collisions. `$var` is then replaced with e.g. `var_7` in one template use and `var_42` in the other template use.
>> - The use of `Hook`s to insert code into outer (earlier) code locations. This is useful, for example, to insert fields on demand.
>> - The use of recursive templates, and `fuel` to limit the recursion.
>> - `Name`s: useful to register field and variable names in code scopes.
>>
>> Next, look at the documentation in. This file is the heart of the Template Framework, and describes all the important features.
>> https://github.com/openjdk/jdk/blob/d21a8aabaf3b191e851b6997c11bb30fcd0f942f/test/hotspot/jtreg/compiler/lib/template_framework/Template.java#L31-L76
>>
>> For a better experience, you may want...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
>
> move order in tutorial
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 485:
> 483: // can access these fields and registers again with "dataNames()".
> 484: //
> 485: // Here a few use-cases:
Suggestion:
// Here are a few use-cases:
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 493:
> 491: // an inner Template could read from or even write to. You can "addDataName" the
> 492: // variable, and the inner Template can then find that variable in "dataNames()".
> 493: // If the inner Template wants to find a random field or varialbe, it may sample
Suggestion:
// If the inner Template wants to find a random field or variable, it may sample
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 494:
> 492: // variable, and the inner Template can then find that variable in "dataNames()".
> 493: // If the inner Template wants to find a random field or varialbe, it may sample
> 494: // from "dataNodes()", and with some probability, it would sample the your variable.
Suggestion:
// from "dataNodes()", and with some probability, it would sample your variable.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 551:
> 549:
> 550: public class InnerTest7 {
> 551: // Let us define a some fields.
Suggestion:
// Let us define some fields.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 764:
> 762: }
> 763:
> 764: // Even simpler: count the available variable and return the count immediately.
Suggestion:
// Even simpler: count the available variables and return the count immediately.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 955:
> 953: // we want to store to a field or variable. We have to make sure that we
> 954: // do not generate code that tries to store to a final field or variable.
> 955: // In other cases, we are only want to load, and we do not care if the
Suggestion:
// In other cases, we only want to load, and we do not care if the
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 961:
> 959: // is irrelevant, but with instances of Objects, this becomes relevant.
> 960: // We may want to load an object of any field or variable of a certain
> 961: // class, or any subclass. When a value of a given class, we can only
Suggestion:
// class, or any subclass. When a value is of a given class, we can only
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 997:
> 995: // We only load from the field, so we do not need a mutable one,
> 996: // we can load from final and non-final fields.
> 997: // We want to find any field of which we can read the value and store
Suggestion:
// We want to find any field from which we can read the value and store
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1049:
> 1047: myClassList.stream().map(c -> templateLoad.asToken(c)).toList(),
> 1048: """
> 1049: // Now lets mutate some fields.
Suggestion:
// Now let us mutate some fields.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1074:
> 1072: // We now introduce another set of "Names", the "StructuralNames". They are
> 1073: // useful for modeling method names an class names, and possibly more. Anything
> 1074: // that has a fixed name in the Java code, for which mutability is inapplicalbe.
Suggestion:
// that has a fixed name in the Java code, for which mutability is inapplicable.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1086:
> 1084: // caught.
> 1085: //
> 1086: // Let us show an examples with Method names. But for simplicity, we assume they
Suggestion:
// Let us show an example with Method names. But for simplicity, we assume they
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1136:
> 1134: // If we directly nest the templateMethod, then the addStructuralName goes to the nested
> 1135: // scope, and is not available at the class scope, i.e. it is not visible
> 1136: // for sampleStructuralName in outside of the templateMethod.
Suggestion:
// for sampleStructuralName outside of the templateMethod.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1141:
> 1139: templateMethod.asToken("+"),
> 1140: templateMethod.asToken("-"),
> 1141: // However, if we insert to the CLASS_HOOK, then the Rendere makes the
Suggestion:
// However, if we insert to the CLASS_HOOK, then the Renderer makes the
test/hotspot/jtreg/testlibrary_tests/template_framework/tests/TestTemplate.java line 121:
> 119:
> 120: public static void main(String[] args) {
> 121: // The follwing tests all pass, i.e. have no errors during rendering.
Suggestion:
// The following tests all pass, i.e. have no errors during rendering.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101812122
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101813916
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101813623
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101814762
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101815298
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101815758
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101816278
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101816774
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101817110
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101817702
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101818057
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101818686
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101819135
PR Review Comment: https://git.openjdk.org/jdk/pull/24217#discussion_r2101807152
More information about the hotspot-compiler-dev
mailing list