String template interpolation as a two steps process

[forax at univ-mlv.fr]

> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>, "amber-spec-experts"
> <amber-spec-experts at openjdk.java.net>
> Sent: Wednesday, April 3, 2024 8:07:42 PM
> Subject: Re: String template interpolation as a two steps process

> We've had this discussion before.
Not exactly, i will try to emphasis the differences. 

> In the old model, it *is* a two-step process; processors could cache type and
> analysis information in the indy call site at capture time through the Linkage
> mechanism, and then could use that information at application time. It's just
> that in the old model, this was not exposed to processors outside the JDK, at
> least not initially. We know that you didn't like that, but we felt that the
> Linkage API needed a lot more work before we were willing to expose it to
> arbitrary code, and didn't want to delay the feature for that. This is old
> news.

> For the JDK processors, the "shortcomings" you list (e.g., caching of types,
> duplicated validation) are not present.

> In the new model, it's the same story. There's a currently-privileged API that
> "processors" can use to cache analysis information in the call site, which is
> computed at first application rather than at capture time, and reused in
> subsequent applications. Again, it is not yet available to processors outside
> the JDK. It's the exact same story.

> I don't see anything in the mail that isn't already handled for the JDK
> processors, but if I missed something, let me know?
The String Template API servent two masters, the end users and we have made some progress on that front and the library developers, the one that will provide libraries with methods that take StringTemplate as parameter and here we do not have really help them, as you said the "shortcomings" for them are still present . 

So as a end user, if i want to use a library that uses a StringTemplate, i know it will come with tradeoff due to those "shortcomings", did the library bypass the security so it's faster, did the library uses a cache with no eviction policy, did the library uses internal implementation details that will make my code not working with the future version of Java, etc. 
In a nutshell, we do not provide the tools for library developers to write a good library using string templates. 

In the past, due to the fact that an interface was used for the processor, it was hard, not impossible, but hard, to provide a good api without adding security issues to the platform (the Lookup object being transfered to the string template processor). With the new design, I think we can provide a better API to the library implementors, not a priviledged API like only the JDK can use but a better API that the one currently proposed. 

Rémi 

> On 3/28/2024 5:05 AM, Remi Forax wrote:

>> Hello,
>> over last week-end, i've implemented an XML template processor using the Java 22
>> state of the spec (using old template processor syntax) and i would like to
>> propose to see the processing of a string template as a two steps process.

>> I will use the XML template processor i've developed as an example, [
>> https://github.com/forax/html-component/blob/master/src/test/java/Demo.java |
>> https://github.com/forax/html-component/blob/master/src/test/java/Demo.java ]
>> Here is how it works, the idea is that if i want to generate the XML of a
>> product, i will write something like this.

>> record Product(String name, int price) implements Component {
>>   public Renderer render() {
>>     return $."""
>>           <tr class=".product">
>>             <td>\{name}</td><td>\{price * 1.20}</td>
>>           </tr>
>>           """;
>>   }
>> }

>> Component is an interface with only one method render() that returns a Renderer
>> and a Renderer is also an interface that is able to send XML events.
>> And "$" is the name of the template processor defined in Component as a static
>> field.

>> The code of the template processor is here [
>> https://github.com/forax/html-component/blob/master/src/main/java/com/github/forax/htmlcomponent/ComponentTemplateProcessor.java#L193
>> |
>> https://github.com/forax/html-component/blob/master/src/main/java/com/github/forax/htmlcomponent/ComponentTemplateProcessor.java#L193
>> ] Conceptually, what a template processor should do is a two step process,
>> first validate the template, in my case validate that the template is a valid
>> XML fragment and then interpolate the result of the validation using the
>> arguments of the template.

>> So processing a sting template is currently
>>  process(StringTemplate) <=> { validate(StringTemplate);
>>   interpolate(StringTemplate); }

>> There are two main shortcomings of the idea that processing a string template is
>> equivalent to calling a method that takes a StringTemplate.
>> - (notypes) the types of the holes are no propagated to the StringTemplate, so
>> the validation part can not verify that the template is correctly typed.
>> - (cache) the validation part has to be re-executed each time.

>> To illustrate the issue (notype), I can have a XML fragment that depends on
>> another class, but i've no way to test if the referenced Product is a record
>> that takes a name of type String and a price of type int because while those
>> types are known by the compiler, they are not available into the String
>> Template.

>> record Cart() implements Component {
>>   public Renderer render() {
>>     return $."""
>>           <table>
>>             <Product name="wood" price="\{10}"/>
>>             <Product name="cristal" price="\{300}"/>
>>           </table>
>>           """;
>>   }
>> }

>> To illustrate the issue (cache), in the code above, i've two calls to rend a
>> Product with different attributes, but for each call to Product::render(), the
>> validation step will be re-executed. As an implementer, I can try to cache the
>> result of the validation but that's far from easy, very bug prone and
>> ultimately not very efficient.

>> Given that a string template literal is a literal, i propose that the Java
>> runtime helps by doing the caching of the validation step.

>> The simplest way I see for that is to separate string template in two, a
>> constant template part composed of the fragments (List<String>) and the types
>> (List<Class<?>>) from the non constant part, the arguments of the template
>> (List<Object>).

>> For that, we need a user-defined intermediary object that correspond to the
>> result of the validation, the creation of this object is the proof that the
>> string template is validated and this object can be cached by the JDK runtime.

>> In that case, processing a string template is equivalent to
>>  var cached userDefinedValidatedTemplate = validateAndCreate(List<String>
>>   fragment, List<Class<?>> types);
>>   process(userDefinedValidatedTemplate, arguments); }

>> So
>> - I propose that StringTemplate is the tuple List<String> fragment,
>> List<Class<?>> types.

>> - Users can create a special template validated class, with a factory method
>> that takes a StringTemplate and is tagged a being a template validator

>>   for example
>>     __template_validated__ class ValidatedXMLDOM {
>>         ...
>>        public static __template__validator__ ValidatedXMLDOM of(StringTemplate
>>         stringTemplate) { ... }
>>     }

>> - a processor method is a method that takes a __template_validated__ object
>> followed by parameters storing the template string arguments
>>   By example
>>     processXML(ValidatedXMLDOM dom, Object... arguments)

>> At compile time, either processXML is called using an invokedynamic or the
>> __template_validated__ instance is computed with a constant dynamic or both.
>> But the idea is that the generated bytecode ensure that the
>> __template_validated__ instance is created once and cached.

>> This is a rough sketch, a lot of details are up to debate but i think we should
>> start to think that the template processing is a two steps process.

>> Rémi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20240404/ad3525a0/attachment.htm>