RFR: 8367531: Template Framework: use scopes and tokens instead of misbehaving immediate-return-queries [v12]
Christian Hagedorn
chagedorn at openjdk.org
Wed Nov 5 13:19:35 UTC 2025
On Wed, 5 Nov 2025 11:40:39 GMT, Emanuel Peter <epeter at openjdk.org> wrote:
>> I got some feedback from users of the Template Framework, especially @galderz . And personally, I already was slightly unsatisfied by some of the issues described below, but did not expect it to be as bad as it is.
>>
>> So I'm sorry, but I think we need to do a significant re-design. It is now still early enough, and only trivial changes are required for the "real" uses of the framework. Only the framework internal tests require significant changes.
>>
>> Many thanks to @galderz for trying out the framework, and reporting the issues. And thanks to @chhagedorn for spending a few hours in an offline meeting discussing the issue.
>>
>> **Major issue with Template Framework: lambda vs token order**
>>
>> The template rendering involves some state, such as keeping track of hashtag replacements, names and fuel cost.
>> Some methods have side-effects (`addDataName`, `let`, ...) and others are simple queries (`sample`, ...).
>> Sadly, the first version of the template framework was not very consistent, and created tokens (deferred evaluation, during token evaluation) for some, and for others it queried the state and returned the result immediately (during lambda execution). One nasty consequence is that an immediately returning query can "float" above a state affecting token. For example, `addDataName` generated a token (so that we know if it is to be added for the template frame or a hook anchoring frame), but answered sampling queries immediately (because that means we can use the returned value immediately and make decisions based on it immediately, which is nice). Looking at the example below, this had the confusing result that `addDataName` only generates a token at first, then `sample` does not have that name available yet, and only later during token evaluation is the name actually added.
>>
>> var testTemplate = Template.make(() -> body(
>> ...
>> addDataName("name", someType, MUTABLE),
>> let("name", dataNames(MUTABLE).exactOf(someType).sample().name()),
>> ...
>> ));
>>
>>
>> **Two possible solutions: all-in on lambda execution or all-in on tokens**
>>
>> First, I thought I want to go all-in on lambda execution, and have everything have immediate effect and return results immediately. This would have the nice effect that the user feels like they are directly in control of the execution order. But I did not find a good way without exposing too many internals to the user, or getting rid of the nice "token lists" we currently have inside Templates (the...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
>
> Apply Christian's suggestions directly
>
> Co-authored-by: Christian Hagedorn <christian.hagedorn at oracle.com>
test/hotspot/jtreg/compiler/lib/template_framework/ScopeTokenImpl.java line 30:
> 28: /**
> 29: * Represents a scope with its tokens. Boolean flags indicate if names,
> 30: * hashtag replacements and {@link setFuelCost} are local, or escape to
The IDE cannot resolve it:
Suggestion:
* hashtag replacements and {@link Template#setFuelCost} are local, or escape to
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 215:
> 213: * </tr>
> 214: * <tr>
> 215: * <th> {@link setFuelCostScope} </th><th> transparent </th><th> transparent </th><th> non-transparent </th>
Missing `#`:
Suggestion:
* <th> </th><th> hashtag </th><th> {@link DataName} and {@link StructuralName} </th><th> {@link #setFuelCost} </th>
* </tr>
* <tr>
* <th> {@link #scope} </th><th> non-transparent </th><th> non-transparent </th><th> non-transparent </th>
* </tr>
* <tr>
* <th> {@link #hashtagScope} </th><th> non-transparent </th><th> transparent </th><th> transparent </th>
* </tr>
* <tr>
* <th> {@link #nameScope} </th><th> transparent </th><th> non-transparent </th><th> transparent </th>
* </tr>
* <tr>
* <th> {@link #setFuelCostScope} </th><th> transparent </th><th> transparent </th><th> non-transparent </th>
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 586:
> 584: * {@link setFuelCost}. This means that any such name, hashtag-replacement or
> 585: * {@link setFuelCost} declared inside the scope is local and does not escape to outer
> 586: * scopes.
Maybe put some more emphasis that it's non-transparent for anything. Otherwise it reads like non-transparent for a number of things but you're left wondering if it's a complete list. Suggestion:
Suggestion:
* Creates a {@link ScopeToken} that represents a scope that is completely
* non-transparent, <strong>not</strong> allowing anything to escape. This
* means that no {@link DataName}, {@link StructuralName}s, hashtag-replacement
* or {@link #setFuelCost} defined inside the scope is available outside. All
* these usages are only local to the defining scope here.
*
* <p>
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 593:
> 591: * <p>
> 592: * If you require a scope that is transparent for some or all of the above, consider
> 593: * using {@link transparentScope}, {@link nameScope}, {@link hashtagScope}, or {@link setFuelCostScope}.
Maybe you could be more specific and cross reference the matrix in the interface comment further up:
Suggestion:
* If you require a scope that is either fully transparent (i.e. everything escapes)
* or only restricts a specific kind to not escape, consider using one of the other
* provided scopes: {@link #transparentScope}, {@link #nameScope}, {@link #hashtagScope},
* or {@link #setFuelCostScope}. A "scope-transparency-matrix" can also be found in
* the interface comment for {@link Template}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 596:
> 594: *
> 595: * <p>
> 596: * The most common use of {@link scope} is in the construction of templates:
You miss some `#` prefixes for methods for Javadocs. Same for methods below.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 618:
> 616: * variables, etc. In rare cases where the scope of the template needs to be
> 617: * transparent (e.g. because we need to insert a variable or field into an
> 618: * outer scope), it is recommended to use {@link transparentScope}.
Maybe we can focus more on the fact that it really does not matter whether you use `scope()` or `transparentScope()` - hashtag replacements and fuel cost will not cross the template boundary.
Feel free to take over this suggestion to better highlight that:
Suggestion:
* Note that regardless of the chosen scope for {@code Template.make},
* hashtag-replacements and {@link #setFuelCost} are always implicitly
* non-transparent (i.e. non-escaping) for hashtag-replacements and
* {@link #setFuelCost} (e.g. a {@link #let} will not escape the template
* scope even when using {@link #transparentScope}. As a default, it is
* recommended to use {@link #scope} for {@code Template.make} since in
* most cases template scopes align with code scopes that are
* non-transparent for fields, variables, etc. In rare cases, where the
* scope of the template needs to be transparent (e.g. because we need
* to insert a variable or field into an outer scope), it is recommended
* to use {@link #transparentScope}. This allows to make {@link DataName}s
* and {@link StructuralName}s available outside this template crossing
* the template boundary.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 632:
> 630: * ),
> 631: * // CODE3: more code in the outer scope, names and hashtags from CODE2 are
> 632: * // not available any more because of the non-transparent "scope".
Suggestion:
* // not available anymore because of the non-transparent "scope".
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 657:
> 655: * {@link setFuelCost}. This means that any such name, hashtag-replacement or
> 656: * {@link setFuelCost} declared inside the scope escapes that scope and is still
> 657: * available in the outer scope.
Analogously to above:
Suggestion:
* Creates a {@link ScopeToken} that represents a completely transparent scope, allowing
* anything to escape anything. This means that {@link DataName}s, {@link StructuralName}s,
* hashtag-replacements and {@link #setFuelCost} declared inside the scope will be available
* in the outer scope.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 664:
> 662: * <p>
> 663: * If you require a scope that is non-transparent for some or all of the above, consider
> 664: * using {@link scope}, {@link nameScope}, {@link hashtagScope}, or {@link setFuelCostScope}.
Analogously to above:
Suggestion:
* If you require a scope that is non-transparent (i.e. nothing escapes) or only restricts
* a specific kind to not escape, consider using one of the other provided scopes:
* {@link #scope}, {@link #nameScope}, {@link #hashtagScope}, or {@link #setFuelCostScope}.
* A "scope-transparency-matrix" can also be found in the interface comment for {@link Template}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 679:
> 677: * Creates a {@link ScopeToken} which represents a scope that is non-transparent for
> 678: * {@link DataName}s and {@link StructuralName}s, but transparent for hashtag-replacements
> 679: * and {@link setFuelCost}.
Adapting from above suggestions:
Suggestion:
* Creates a {@link ScopeToken} that represents a scope that is non-transparent for
* {@link DataName}s and {@link StructuralName}s (i.e. cannot escape), but
* transparent for hashtag-replacements and {@link #setFuelCost} (i.e. available
* in outer scope).
*
* <p>
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 686:
> 684: * <p>
> 685: * If you require a scope that is non-transparent for some or all of the above, consider
> 686: * using {@link scope}, {@link transparentScope}, {@link hashtagScope}, or {@link setFuelCostScope}.
Suggestion:
* If you require a scope that is transparent or uses a different restriction, consider
* using one of the other provided scopes: {@link #scope}, {@link #transparentScope},
* {@link #hashtagScope}, or {@link #setFuelCostScope}. A "scope-transparency-matrix" can
* also be found in the interface comment for {@link Template}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 701:
> 699: * Creates a {@link ScopeToken} which represents a scope that is non-transparent for
> 700: * hashtag-replacements, but transparent for {@link DataName}s and {@link StructuralName}s
> 701: * and {@link setFuelCost}.
Suggestion:
* Creates a {@link ScopeToken} that represents a scope that is non-transparent for
* hashtag-replacements (i.e. cannot escape), but transparent for {@link DataName}s
* and {@link StructuralName}s and {@link #setFuelCost} (i.e. available in outer scope).
*
* <p>
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 708:
> 706: * <p>
> 707: * If you require a scope that is non-transparent for some or all of the above, consider
> 708: * using {@link scope}, {@link transparentScope}, {@link nameScope}, {@link setFuelCostScope}.
Suggestion:
* If you require a scope that is transparent or uses a different restriction, consider
* using one of the other provided scopes: {@link #scope}, {@link #transparentScope},
* {@link #nameScope}, or {@link #setFuelCostScope}. A "scope-transparency-matrix" can
* also be found in the interface comment for {@link Template}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 742:
> 740: * Creates a {@link ScopeToken} which represents a scope that is non-transparent for
> 741: * {@link setFuelCost}. but transparent for {@link DataName}s and {@link StructuralName}s
> 742: * and hashtag-replacements.
Last but not least also adapted from above:
Suggestion:
* Creates a {@link ScopeToken} that represents a scope that is non-transparent for
* {@link #setFuelCost} (i.e. cannot escape), but transparent for hashtag-replacements,
* {@link DataName}s and {@link StructuralName}s (i.e. available in outer scope).
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 749:
> 747: * <p>
> 748: * If you require a scope that is non-transparent for some or all of the above, consider
> 749: * using {@link scope}, {@link transparentScope}, {@link nameScope}, {@link setFuelCostScope}.
Suggestion:
* If you require a scope that is transparent or uses a different restriction, consider
* using one of the other provided scopes: {@link #scope}, {@link #transparentScope},
* {@link #hashtagScope}, or {@link #nameScope}. A "scope-transparency-matrix" can
* also be found in the interface comment for {@link Template}.
test/hotspot/jtreg/compiler/lib/template_framework/Template.java line 812:
> 810: */
> 811: static String $(String name) {
> 812: // Note, since the dollar replacements do not change during a template
within?
Suggestion:
// Note, since the dollar replacements do not change within a template
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494138179
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494153205
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494220738
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494257064
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494196276
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494282318
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494284210
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494309710
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494321316
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494328403
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494338507
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494348553
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494347425
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494370502
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494373147
PR Review Comment: https://git.openjdk.org/jdk/pull/27255#discussion_r2494382680
More information about the hotspot-compiler-dev
mailing list