RFR: 8367531: Template Framework: use scopes and tokens instead of misbehaving immediate-return-queries [v29]

[Emanuel Peter]

On Fri, 14 Nov 2025 13:17:11 GMT, Roberto Castañeda Lozano <rcastanedalo at openjdk.org> wrote:

>>> Hi Emanuel, thanks for improving the design of the template framework, the enforcement of "everything is a token" and the introduction of explicit scope constraints seem like a step in the right direction. Before I go on with the review, I would like to ask two high-level questions (apologies if these are already discussed, it is hard to browse through a PR history):
>>> 
>>> * The tutorial and the Template documentation remark that we would ideally have used string templates rather than hashtag replacements. Is this still true after the introduction of explicit scoping constraints, i.e. could we still simply use string templates and still enforce the user-provided scoping rules if the feature was available?
>> 
>> Yes, I think so. We would probably get rid of `let` and just use local variables in lambdas, and then format them directly into strings. Scopes would be colocated with lambdas, so that local variables could be local to the scopes. I'm less sure about letting local variables (instead of hashtags) escape lambdas .. that's not really possible. But maybe there would be work-arounds.
>> 
>>> * If I got the comments in the tutorial right, it seems that the user has good control over the "transparency level" of scopes, while the transparency rules for templates are hardcoded (hashtag replacements never escape, DataNames always escape, etc.). This felt a bit surprising, would it be feasible to just let the outermost scope in a template determine the template's transparency level?
>> 
>> The understanding seems to maybe be incomplete:
>> 
>>> transparency rules for templates are hardcoded 
>> 
>> It is only "hardcoded" to never let hashtags and setFuelCost escape, it just implicitly downgrades a scope on those two "dimensions".
>> 
>> But the user still has control over the name "dimension":
>> It still needs to be possible to decide if `DataName`s and `StructuralName`s escape the Template scope, otherwise they cannot escape from a `Hook.insert`.
>> 
>> Maybe it would be nice to force the user to use only scopes that exactly match the semantics of the implementation (only allow control if names are transparent or not). So maybe one could only use `scope` (completely non-transparent) or `scopeWithNameTransparency` (only transparent to names), and that would somehow be enforced by the Java types/interfaces of the framework. Or maybe we just throw an Exception if the wrong one is used. But it would require us to define this extra `scopeWithNameTransparency`. Do you think...
>
>> We would probably get rid of `let` and just use local variables in lambdas, and then format them directly into strings.
> 
> But wouldn't this take us out of the "everything is a token" design again? What I am getting at is that the comments in the tutorial and the documentation suggesting that string templates could be used to provide the same functionality could confuse the user (they at least did challenge my mental model of the framework), and maybe it would be worth removing them, or at least adding some nuance to them.

@robcasloz I suppose we could just remove them, they are just a bit of background.

> But wouldn't this take us out of the "everything is a token" design again?

I don't think so. It just avoids having to do the detour via `let` and hashtags, and would allow the values to go directly into the string. The formatted strings would then be the tokens. So you would end up with fewer tokens: instead of a `let` and the string with the hashtag, you would just have a templated string that injects the variable into it.

But of course, the Template Framework provides more than just string formatting: it also has the `fuel` concept, and the `DataName/StructuralName` concept. And for those, we would still need the explicit scoping. Especially for names - those are fundamentally related to scopes, just because Java has "names in scopes".

> at least adding some nuance to them.

Ok, well let me try that...

So I looked for mentions of "string template", and I only found the passage below from `Template.java`. Let me know if there are more that you were thinking about.


   159  * Ideally, we would have used <a href="https://openjdk.org/jeps/430">string templates</a> to inject these Template                                                                                           
   160  * arguments into the strings. But since string templates are not (yet) available, the Templates provide
   161  * <strong>hashtag replacements</strong> in the {@link String}s: the Template argument names are captured, and   
   162  * the argument values automatically replace any {@code "#name"} in the {@link String}s. See the different overloads
   163  * of {@link #make} for examples. Additional hashtag replacements can be defined with {@link #let}.
   164  *


I think that's still largely accurate now. We need hashtag replacements because we don't have string templating. But that does not mean we would get rid of the Template Framework once we have string templating. Though I suppose we might be able to get rid of hashtag replacements at that point.

@robcasloz What do you think: does it still need more nuance? Should I just delete it? Or keep as is?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/27255#issuecomment-3532872057