RFR: 8367531: Template Framework: use scopes and tokens instead of misbehaving immediate-return-queries [v24]
Christian Hagedorn
chagedorn at openjdk.org
Fri Nov 7 12:33:22 UTC 2025
On Thu, 6 Nov 2025 15:50:50 GMT, Emanuel Peter <epeter at openjdk.org> wrote:
>> I got some feedback from users of the Template Framework, especially @galderz . And personally, I already was slightly unsatisfied by some of the issues described below, but did not expect it to be as bad as it is.
>>
>> So I'm sorry, but I think we need to do a significant re-design. It is now still early enough, and only trivial changes are required for the "real" uses of the framework. Only the framework internal tests require significant changes.
>>
>> Many thanks to @galderz for trying out the framework, and reporting the issues. And thanks to @chhagedorn for spending a few hours in an offline meeting discussing the issue.
>>
>> **Major issue with Template Framework: lambda vs token order**
>>
>> The template rendering involves some state, such as keeping track of hashtag replacements, names and fuel cost.
>> Some methods have side-effects (`addDataName`, `let`, ...) and others are simple queries (`sample`, ...).
>> Sadly, the first version of the template framework was not very consistent, and created tokens (deferred evaluation, during token evaluation) for some, and for others it queried the state and returned the result immediately (during lambda execution). One nasty consequence is that an immediately returning query can "float" above a state affecting token. For example, `addDataName` generated a token (so that we know if it is to be added for the template frame or a hook anchoring frame), but answered sampling queries immediately (because that means we can use the returned value immediately and make decisions based on it immediately, which is nice). Looking at the example below, this had the confusing result that `addDataName` only generates a token at first, then `sample` does not have that name available yet, and only later during token evaluation is the name actually added.
>>
>> var testTemplate = Template.make(() -> body(
>> ...
>> addDataName("name", someType, MUTABLE),
>> let("name", dataNames(MUTABLE).exactOf(someType).sample().name()),
>> ...
>> ));
>>
>>
>> **Two possible solutions: all-in on lambda execution or all-in on tokens**
>>
>> First, I thought I want to go all-in on lambda execution, and have everything have immediate effect and return results immediately. This would have the nice effect that the user feels like they are directly in control of the execution order. But I did not find a good way without exposing too many internals to the user, or getting rid of the nice "token lists" we currently have inside Templates (the...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
>
> fix whitespace
Thanks a lot for the attribution! 😊 And also for all the updates and additional discussions! I appreciate that a lot.
Oof, it became quite challenging with multiple review and update rounds interleaved - might be something to watch out more next time :-)
Update to `Template.java` review: Looks good!
Update to `TestTutorial.java` view: Some more small suggestions but otherwise looks good!
Update to `TemplateFrame` and `CodeFrame`: Some more minor comments. I have not looked at the new visual example - I thought I'll do that when diving further into the code.
test/hotspot/jtreg/compiler/lib/template_framework/TemplateFrame.java line 42:
> 40: * <p>
> 41: * A {@link Template} can have multiple {@link TemplateFrame}s, if there are nested
> 42: * scopes. The outermost {@link TemplateFrame} determines the id of the {@link Tmeplate}
Suggestion:
* scopes. The outermost {@link TemplateFrame} determines the id of the {@link Template}
test/hotspot/jtreg/compiler/lib/template_framework/TemplateFrame.java line 52:
> 50: * and queries such as {@link Template#let} definitions. Each {@link TemplateFrame}
> 51: * has such a set of hashtag replacements, and implicitly provides access to the
> 52: * hashtag replacmeents of the outer {@link TemplateFrame}s, up to the outermost
Suggestion:
* hashtag replacements of the outer {@link TemplateFrame}s, up to the outermost
test/hotspot/jtreg/compiler/lib/template_framework/TemplateFrame.java line 62:
> 60: * on how much time is spent on the code from the template corresponding to the frame,
> 61: * and to give a termination criterion to avoid nesting templates too deeply.
> 62: *
It now more sounds like a "TemplateScope" since we have a "TemplateFrame" per scope and not per template which the latter name somehow suggests. But just wanted to share that thought here.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1166:
> 1164:
> 1165: // In this section, we will look at some subtle facts about the behavior of
> 1166: // transparent scopes around hook insertion. This inteded for expert users
Suggestion:
// transparent scopes around hook insertion. This is intended for expert users
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1168:
> 1166: // transparent scopes around hook insertion. This inteded for expert users
> 1167: // so feel free to skip it until you extensively use hook insertion.
> 1168: // More info can also be found in the javadocs of the Hook class.
Suggestion:
// More info can also be found in the Javadocs of the Hook class.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1171:
> 1169:
> 1170: // Helper method to check that the expected DataNames are available.
> 1171: var templateVerify = Template.make("toList", (String toList) -> scope(
Awesome, thanks for adding the expert example - that is really helpful to better grasp the interaction of hooks and scopes.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1220:
> 1218: // x4 escapes to the caller out here, and not to the anchor scope.
> 1219: "// x4: #x4\n",
> 1220: // And v4 escapes to the anchor scope, which is available from hee too.
Suggestion:
// And v4 escapes to the anchor scope, which is available from here too.
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1241:
> 1239: templateVerify.asToken("v1, v4, v5, v2")
> 1240: )),
> 1241: templateVerify.asToken("v1"),
For completeness:
Suggestion:
// We left the non-transparent anchoring scope which does not let anything escape
templateVerify.asToken("v1"),
test/hotspot/jtreg/testlibrary_tests/template_framework/examples/TestTutorial.java line 1261:
> 1259: templateVerify.asToken("v1, v6, v7")
> 1260: )),
> 1261: templateVerify.asToken("v1, v6, v7")
We could probably also add another `let("x6", 5)` inside the `anchor` and access it here to show that everything escapes the anchor. What do you think?
For completeness:
Suggestion:
let("x6", 42) // escapes the anchor scope
)),
// We left the transparent anchoring scope which lets the DataNames and
// hashtags escape.
"// x6: #x6\n",
templateVerify.asToken("v1, v6, v7")
-------------
PR Review: https://git.openjdk.org/jdk/pull/27255#pullrequestreview-3432859378
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2503092934
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2503094493
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2503115113
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502620560
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502621715
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502637540
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502641117
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502654333
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2502684113
More information about the hotspot-compiler-dev
mailing list