Feedback for StringTemplate; indify StringTemplate Processor invocation to cache compiled Processor at runtime

[ShinyaYoshida]

Hi,
(Please tell me if there's another proper ML than amber-dev)

I recently played around with StringTemplate.

I developed a StringTemplate Processor for JSON recently:
https://github.com/bitterfox/json-string-template
This parses JSON format strings in fragments with placeholders with
referring values array element.

In the first naive implementation, it tokenizes fragments and parses the
tokens creating Json Objects taking elements from values, and putting the
value as the Json Object key or value.
So this is the one-pass implementation, but it parses fragments every time
the JSON String Template processing is called.
https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonParserV1.java

In the second implementation, it parses fragments and creates AST for JSON.
After that, it reads the AST again and creates a Lambda expression for
`(Object[] value) -> JSON` that puts an element of values to a part of
JsonObject.
https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonCompiler.java

Thanks to returning (Object[] value) -> JSON and immutability of fragments
of the string template expression (as long as we use through the string
template expression, it will be the same fragments for the same invocation),
We can now cache the lambda expression and skip parsing fragments to JSON
AST.

This provides us a space for optimization of the performance of JSON String
Template by caching, however, StringTemplate.Processor is called for the
same receiver usually because we usually define StringTemplate.Processor
instance as static final field and use string template expression through
it.
So we need to implement a caching layer by ourselves.
https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonStringTemplateProcessorV2CachedImpl.java

The downside of such caching implementation is
- The caching layer could be the bottleneck and cause another performance
issue
- We need to depend on the implementation detail of Javac, and JDK for
better performance of caching. For example, I assumed runtime always
creates the same fragments reference
  -
https://github.com/bitterfox/json-string-template/blob/b53898da2d2c70aeee9a36cdffc34ba31460ba28/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonStringTemplateProcessorV2CachedImpl.java#L63
  - This might not work on someday

Now I'm working on a third implementation with Java22 using ClassFileAPI.
Using ClassFileAPI, we can generate a Java class after parsing fragments
that is
```
static JsonObject createJson(Object[] values) {
    return Json.createObjectBuilder().add("name",
Json.createValue(value[0].toString()));
}
```
for JSON"{'name': \{name}}".
Once it's compiled to the Java class and loaded, calling it will have
almost zero overhead even though we run it through string template
expression.

However, as the same reason for the 2nd implementation, there's no caching
mechanism in StringTemplate, still, we need to implement the caching layer
as well as the 2nd approach with the same problems.

Suppose we can couple the compiled class (or (values: Object[]) -> T) to
the invocation of string template processor at string template expression.
In that case, we can solve this issue and provide the best performance for
every StringTemplate Processor in the Java world.

Now we already have a tool for it in JVM, invokedynamic.

What do you think about using invokedynamic not only for creating
StringTemplate objects but also calling StringTemplate Processor invocation
and providing a caching layer at Java spec?

I made a bytecode level PoC for StringTemplate Processor invocation using
invokedynamic with caching for each StringTemplate expression.

https://github.com/bitterfox/indy-string-template-processing/

In the PoC, I changed the interface of StringTemplate Processor.
I added StringTemplateProcessorFactory: (String[] fragments) ->
StringTemplateProcessor.
StringTemplateProcessor is (Object[] values) -> Object (i.e. T).
And it caches StringTemplateProcessor.
StringTemplateProcessorFactory has a method to indicate to cache the
processor or create the processor every time to runtime.

StringTemplateRuntime runs the processing for
StringTemplateProcessorFactory and StringTemplate(fragments and values).
If cache is required, it creates a processor only once.

StringTemplateRuntime is used for the MethodHandle coupled to the
invokedynamic.
StringTemplateBSM#createStringTemplateRuntimeCallSite is a BSM for the
invokedynamic.

If we compile
doStringTemplate() {
    Sting name = "duke";
    System.out.println(MY_STP."Hello \{name}");
}

It will generate classfile like

  public static void doStringTemplate();
    descriptor: ()V
    flags: (0x0009) ACC_PUBLIC, ACC_STATIC
    Code:
      stack=6, locals=2, args_size=0
         0: getstatic     #10                 // Field
StringTemplateSTRDebug.STR:LStringTemplateProcessorFactory;
         3: iconst_2
         4: anewarray     #12                 // class java/lang/String
         7: dup
         8: iconst_0
         9: ldc           #14                 // String Hello
        11: aastore
        12: dup
        13: iconst_1
        14: ldc           #16                 // String
        16: aastore
        17: iconst_1
        18: anewarray     #18                 // class java/lang/Object
        21: dup
        22: iconst_0
        23: ldc           #20                 // String duke
        25: aastore
        26: invokedynamic #31,  0             // InvokeDynamic
#0:process:(LStringTemplateProcessorFactory;[Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/Object;
        31: astore_1
        32: getstatic     #37                 // Field
java/lang/System.out:Ljava/io/PrintStream;
        35: aload_1
        36: invokevirtual #43                 // Method
java/io/PrintStream.println:(Ljava/lang/Object;)V
        39: return

BootstrapMethods:
  0: #27 REF_invokeStatic
StringTemplateBSM.createStringTemplateRuntimeCallSite:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
    Method arguments:

This creates string template processor only once for this method
```
// call doStringTemplate
Create new processor, some processor may parse fragments, so slow
Process string template
Hello duke
// call doStringTemplate
Process string template
Hello duke
// call doStringTemplate
Process string template
Hello duke
```


This kind of mechanism will be useful for most of string template
processors like
- FMT: This parses formatter string like %02d
- SQL: Same overhead with queryBuilder.select("*").from(...).where(...) is
preferred
- LocalizationProcessor: should avoid loading resource bundle every time
- ...

Let me know your thoughts about this approach for StringTemplate Processor
invocation

Best regards,
Shinya Yoshida (@bitter_fox, @shinyafox)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-dev/attachments/20240328/7905b65b/attachment-0001.htm>