String Tapas Redux: Beyond mere string interpolation

[Remi Forax]

----- Original Message -----
> From: "John Rose" <john.r.rose at oracle.com>
> To: "Jim Laskey" <james.laskey at oracle.com>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Vendredi 17 Septembre 2021 08:43:29
> Subject: Re: String Tapas Redux: Beyond mere string interpolation

> Yay!
> 
> I agree with Brian’s response to Remi:  Nothing new here
> regarding eval or ASTs.

Ok !

> 
> My favorite wished-for-use-case for templated strings is
> a grammar where the “holes” are grammar actions or
> other configuration points for rules.  This paper made
> me envious for the sake of Java, and also made me think,
> “when we get string templates we can try our own
> shenanigans”:
> 
> http://www.inf.puc-rio.br/~roberto/lpeg/#grammar
> http://www.inf.puc-rio.br/~roberto/docs/peg.pdf
> 
>> We can meet our diverse goals by separating mechanism from
>> policy. How we introduce parameters into string expressions is
>> mechanism; how we combine the parameters and the string into a final
>> result (e.g., concatenation) is policy.
> 
> One might also say we separate wiring from function.  How we introduce
> a segmented string with holes, and also (typed) expressions to fill
> those holes, is the wiring.  (No function yet: We just observe all
> those values waiting for us to do something with them.)  How we
> arrange the static structure of those values verges on function, but
> it’s really just setting up for the moment when we have all the values
> and can run our function.  The function comes in when we have all the
> values (crucially, the dynamic and typed hole-filling values).  At
> that point it’s really “only” a method call, but that method contains
> all the function (the “policy”).  The intermediate step where we
> derive a recipient for the function call, from the static parts
> (strings, and also hole types), is a factory method or constructor
> call, one which creates the receiver object (which is constant for
> that expression, just like a no-capture lambda).

yes, it's a two steps process, first return a method handle (as you say below) from the formatted String, then call it with the arguments.

Hum, it's really, really like a bootstrap method, no ?

And we already know that because this is how the StringConcatFactory works, the only difference is that the format used by the StringConcatFactory define a hole as a special unicode character where here we want to have named holes (for the ResourceBundle example).

> 
> It’s important to separate wiring from function in part because
> (a) wiring can be fully understood while (b) function is inherently
> undecidable.  So it’s good (for extensibility, universality) when
> the wiring is really simple and clear, and the transitions into
> the mysterious function-boxes are also really clear.

yes,

> 
> Also, if we focus on wiring that is as universal as possible,
> we can do fancy stuff like grammars with functional actions.
> Otherwise, it’s harder.
> 
> Also, making the function part “only a method call” means
> that whatever resources the language has to make method calls
> be a good notation apply to templated strings.
> 
> Also, since the “wiring part” includes the round-up of the
> static parts of the template expression, it follows that we can
> do lots of interesting “compile time” work in the indy or condy
> instruction that implements the expression as a whole.

amen

> 
> I am gently insisting that the types of the “holes” are part
> of the template setup code because, after all, that’s what the
> indy needs anyway, and it seems a shame to make them all
> be erased to Object and Object[].

or String[] like in Scala.

> 
> One reason “just use Object” is a missed opportunity:  You
> get lots of boxing.  Another, deeper one:  Without FI-based
> target typing that would be provided by a general Java method
> call, you can’t put poly-expressions (like lambdas) into the holes.

The other issue is checked exception, we need a way to allow the possible checked exception to be thrown (i have no idea how ?).

> 
> For example, a PEG template might take a varargs argument of
> type (PEGRuleAction… actions) where PEGRuleAction is a FI.
> (Mixing FIs and other data is a challenge I will delay for now,
> but it’s under the rubric of “Varargs 2.0”, allowing methods
> to capture variable-length yet type-heterogeneous arguments.
> Think also for Map<K,V>.of(…) which wants to take pairs
> of arguments of alternating types.  But that’s for later.)

In my fantasy world, Record** is the type you are looking for, it's how you can declare a keyword arguments that that should be generated as a synthetic record by the compiler.

void foo(Record** r) ask for a record generated by the compiler so the MethodType passed to the bootstrap method will be something like ($SyntheticRecord$)V

> 
> I’m fully aware that I will be asking for stuff that is beyond
> a 1.0-level feature set, but in discussing stuff like this I’m
> hoping to stake out a path for growth to wider set of use
> cases that inevitably comes if the meaning of the expression
> is “call this method on this compute-once receiver”, as opposed
> to something more constrained (even if the 1.0 level is constrained).
> 
> Or, to go back to the “mechanism vs. policy” formulation,
> the (very interesting) policy is embodied in the template
> expression receiver object itself that is built on first use
> of the expression, and it is also in the (very interesting)
> method that is called on the object with the typed hole
> values.  The mechanism consists of the rules by which
> the expression receiver object (TemplatePolicy) is created,
> and what values are passed to its constructor or factory
> (just once, lazy), and then (each time the expression is
> evaluated) how the holes are passed to the object, and
> via which method.
> 
> The wiring I’m suggesting starts with a one-time setup
> operation, which needs to be statically defined (no dynamic
> argument dependencies).  I’m suggesting that the policy
> provides a factory method which is run once (probably
> via a condy or indy) per expression.

[...]

Let me summarize what i'm thinking of:

3) at the end, calling a String Template is like calling a MethodHandle
2) we need a one time wiring in a very similar way to how a boostrap method is called by invokedynamic
   one of the boostrap arguments is the String representing the Template String
1) unlike a bootstrap method, we want to define all possibles signatures for the compiler
  so the compiler will not accept all String template but the one
  - with a peculiar receiver (an instance or a class ?)
  - with some constraints on the types of the parameters like PEGRuleAction...
  - we should be able to declare the possible checked exceptions
 
We already know how to do (2) and (3) because it's very similar to the StringConcatFactory.

(1) is more complex because it's how we should interact with the compiler so we reject some Template String at compile time,
and i believe that here we don't want the compiler to execute any code so it has to done in a declarative way
(in a similar way the meta-annotation @Target restricts where you can put an annotation in a declarative way so can be interpreted by the compiler without running any user code).


Here is my proposal.
(1) there is a new keyword __template-string__ that can be used on instance or static method, this method is like an abstract method because it has no code,
    but is used by the compiler to typecheck the Template String
(2) if there is a method declared __template-string__ in the class, the class must also declare a boostrap method that works the same way as a classic bootstrap method
    and takes the template as the last parameter (as a String to keep things simple for now)
(3) any occurrences of a Template String that reference a class containing a method __template-string__ is desugared as an invokedynamic that calls the bootstrap method
    and takes the parameter of the Template String as regular arguments.


Some examples below:

---

With the Template String
  var name = ...
  var age = ...
  var s = String."Hello, \(name), I am \(age) years old";   // "\(name)" is a shortcut for String."\(name)"

The class j.l.String declares:

package java.lang.String;

public class String {
  public static __template-string__ String interpolate(Object... args);

  public static CallSite bootstrap(Lookup lookup, String name, MethodType desc, String template) {
    String stringConcatTemplate = escapeHoleAsStringConcatFactoryHoles(template);
    return StringConcatFactory.boostrap(lookup, name, desc, stringConcatTemplate);
  }
}


---

With the Template String (with no parameters)
  var matcher = Pattern."[A-Za-z]+".matcher(text);

The class j.u.regex.Pattern declares:

package java.util.regex;

public class Pattern {
  public static Pattern compile(String regex) { ... }

  public static __template-string__ Pattern interpolate();

  public static CallSite bootstrap(Lookup lookup, String name, MethodType desc, String template) {
    return new ConstantCallSite(MethodHandles.constant(Pattern.class, Pattern.compile(template)));
  }
}


---
With the Template String,
  var connection = ...
  var table = ...
  try(var resultSet = connection."SELECT * FROM \(table)") {
    ...
  }


package java.sql;

public class Connection {
  public __template-string__ ResultSet interpolate(Object... args) throws IOEception;

  public static CallSite bootstrap(Lookup lookup, String name, MethodType desc, String template) {
    String queryText = convertTemplateToSQLQuery(template);
    return new ConstantCallSite(...);
  }
}

> 
> — John

Rémi