[External] : TemplatedString feedback - logging use-case
Jim Laskey
james.laskey at oracle.com
Fri May 13 13:39:48 UTC 2022
As a follow up for the examples I gave yesterday, I forgot I mentioned 'chaining' in my first reply.
static final SimplePolicy<TemplatedString> EVAL = ts -> {
List<Object> values = ts.values()
.stream()
.map(v -> v instanceof Supplier<?> s ? s.get() : v)
.toList();
return TemplatedString.of(ts.stencil(), values);
};
...
static final StringPolicy STR_EVAL = StringPolicy.chain(STR, EVAL);
...
String deferredResult = STR_EVAL."\{supplier}";
In this example, the EVAL policy simply 'gets' the value for each Supplier value. The end result is a new TemplatedString with all of its values fully resolved.
By using the chain method, we can create a new policy that combines resolution of the EVAL policy with concatenation done by the STR policy. The above deferredResult expression is equivalent to;
String deferredResult = STR.apply(EVAL.apply("\{supplier}"));
Cheers,
-- Jim
On May 13, 2022, at 6:16 AM, Remi Forax <forax at univ-mlv.fr<mailto:forax at univ-mlv.fr>> wrote:
I think the logger is a great example of codes that should work.
I think we are well aware of Log4Shell and the danger of dynamic interpolation, by that i mean interpolation that depends on the dynamic class of the value instead of the static type of the value. It's not always possible to only use the static type, if we do not allow java.lang.Object people will not use our new API but at least we can easily distinguish what should be escaped (objects) from what should not.
To Adam, the policy is a not the only way to use a templated string, you can write your methods log/warning/info/error etc to take a TemplatedString as parameter.
For me, at the moment, the whole API is not in a great shape.
TemplatePolicy is more of less restricted to literals because apply() does not take parameters.
TemplatedString forget the type of every values and box them, so it can not be really used as parameter of a Logger method because any other existing methods are faster.
Also PolicyLinkage should not be public because only the classes of the JDK can use it, the interface is sealed. It's like @SignaturePolymorphic, the compiler needs to know the interface but it's restricted to JDK only usage.
I suppose the next iteration will use the carrier API to store the values of the templated string ?
So the TemplatedString will store the stencil, a carrier object and the MethodType describing the carrier (it can still expose the values as a List<Object> for convenience).
With that i suppose we can get good enough performance if the logger methods takes a Supplier<TemplatedString> as parameter (the lambda will have to capture the values of the templated string so by introspection at runtime you can observe the declared type of those values and you only need to do that once per call).
But it does not solve the issue of the missing types at compile time, i don't think people will find nice to have to declare the lambda in a static field each time they want to pass one as value of a templated string like below.
regards,
Rémi
----- Original Message -----
From: "Jim Laskey" <james.laskey at oracle.com<mailto:james.laskey at oracle.com>>
To: "amber-dev" <amber-dev at openjdk.java.net<mailto:amber-dev at openjdk.java.net>>, "Adam Juraszek" <juriad at gmail.com<mailto:juriad at gmail.com>>
Cc: "attila kelemen85" <attila.kelemen85 at gmail.com<mailto:attila.kelemen85 at gmail.com>>
Sent: Thursday, May 12, 2022 7:25:02 PM
Subject: Re: TemplatedString feedback and extension - logging use-case, lazy values, blocks
[Note that all code samples below are speculative and subject to change. There
is also an assumption reader has read and has an understanding of the JEP
draft. https://openjdk.java.net/jeps/8273943]
I've had a chance to isolate some examples to demonstrate an approach for
deferred evaluation using suppliers.
Starting with the general pattern for a (string result) policy as follows;
static final StringPolicy MY_SP = ts -> {
StringBuilder sb = new StringBuilder();
Iterator<String> fragmentsIter = ts.fragments().iterator();
for (Object value : ts.values()) {
sb.append(fragmentsIter.next());
sb.append(value);
}
sb.append(fragmentsIter.next());
return sb.toString();
};
...
int x = 10, y = 20;
String result = MY_SP."\{x} + \{y} = \{x + y}"; // The result will be "10 + 20 =
30".
To introduce deferred evaluation we can add a test for
java.util.function.Supplier and
use the supplier's value instead;
static final StringPolicy MY_SP = ts -> {
StringBuilder sb = new StringBuilder();
Iterator<String> fragmentsIter = ts.fragments().iterator();
for (Object value : ts.values()) {
sb.append(fragmentsIter.next());
//------------------------------------------
if (value instanceof Supplier<?> supplier) {
sb.append(supplier.get());
} else {
sb.append(value);
}
//------------------------------------------
}
sb.append(fragmentsIter.next());
return sb.toString();
};
...
static final DateTimeFormatter HH_MM_SS =
DateTimeFormatter.ofPattern("HH:mm:ss");
static final Supplier<String> TIME_STAMP = () ->
LocalDateTime.now().toLocalTime().format(HH_MM_SS);
...
String message = MY_SP."\{TIME_STAMP}: Received response"; // The result will be
"13:56:48: Received response".
In this case, the formatting of the time stamp is deferred until the policy is
applied. This example can be expanded to test for
java.util.concurrent.Future and java.util.concurrent.FutureTask. The advantage
of futures
over Supplier is that they are only evaluated once;
static final StringPolicy MY_SP = ts -> {
StringBuilder sb = new StringBuilder();
Iterator<String> fragmentsIter = ts.fragments().iterator();
for (Object value : ts.values()) {
sb.append(fragmentsIter.next());
//------------------------------------------
if (value instanceof Future future) {
if (future instanceof FutureTask task) {
task.run();
}
try {
sb.append(future.get());
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
} else if (value instanceof Supplier<?> supplier) {
sb.append(supplier.get());
} else {
sb.append(value);
}
//------------------------------------------
}
sb.append(fragmentsIter.next());
return sb.toString();
};
To answer Attila's comments. 1) Nothing is computed unless the policy performs
an action, so a logging policy
that has an if(ENABLED) around it's main code and static final ENABLED = false;
will have little or no overhead.
2) One of the primary goals of template policies is to screen inputs to prevent
injection attacks. So a well written policy should not be
any more susceptible that a println statement. There is no interpretation or a
need for interpretation in string templates. Policies may
add interpretation, but then the onus is on the policy
Cheers,
-- Jim
On May 7, 2022, at 11:52 AM, Jim Laskey
<james.laskey at oracle.com<mailto:james.laskey at oracle.com><mailto:james.laskey at oracle.com>> wrote:
[OOTO]
I’ve already done examples of all of the items in your list (all good), but i’m
glad you brought these topics up for open discussion. I’ll sit back a see what
others have to say.
However, I would like to point out that deferred evaluation requires an
agreement from the policy to special case suppliers (or futures). This is
something we don’t really want to bake in. Every policy should be free to pick
and choose what it does.
Another approach is to use policy chaining where one policy in the chain
evaluates suppliers and futures to produce a new TemplatedString to be
processed by the next policy in the chain.
The ugliness of lambdas in an embedded expression is made worse by the fact it
has to be cast to Supplier<T> or Future<T> since the default type for embedded
expressions is Object. In all my examples, I’ve out of lined these types of
expressions and used a local or better yet a global constant variable (ex.,
TIME_STAMP) to use in the embedded expression. I think this technique makes the
string template read better.
When I’m back the end of next week I can post some examples.
Cheers,
—Jim
After this response we should move the discussion to the amber-dev mailing list.
��
On May 7, 2022, at 10:58 AM, Adam Juraszek
<juriad at gmail.com<mailto:juriad at gmail.com><mailto:juriad at gmail.com>> wrote:
The TemplatedString proposal is amazing; I can see using it in many places.
I have a few comments that I have not seen addressed.
Logging
=======
One place where some kind of string templating is often used is logging.
Most logging frameworks use a pair of braces {}. TemplatedStrings have the
potential to revolutionize it.
Log record usually consists of the following parts that the programmer
needs to provide:
* log level (such as error, info)
* message (such as "Oh no, the value {} is negative")
* parameters (such as -42)
* exception/throwable (only sometimes)
Let's assume that TemplatedString is performant and virtually free, then
these options would be possible. I cannot say which syntax the logging
frameworks would choose:
logger.error."Oh no, the value \{value} is negative\{ex}";
logger.error("Oh no, the value \{value} is negative", ex);
logger."Oh no, the value \{value} is negative".error(ex);
Versus the existing:
logger.error("Oh no, the value {} is negative", value, ex);
Lazy Logging
============
Some logging frameworks such as Log4j2 allow lazy evaluation for example:
logger.trace("Number is {}", () -> getRandomNumber());
I must admit that doing the same using TemplatedStrings is ugly and the 6
distinct characters introducing a lazily evaluated value are impractical:
logger.trace("Number is \{() -> getRandomNumber()}");
Can we offer something better using TemplatedStrings? The JEP already talks
about evaluation. For example, this is valid:
int index;
String data = STR."\{index=0}, \{index++}";
and mentions that the TemplatedString provides a list of values:
TemplatedString ts = "\{x} plus \{y} equals \{x + y}";
List<Object> values = ts.values();
I want to propose an additional syntax. The escape sequence \( would denote
a lazily computed value - probably stored as a Supplier. Because the value
may not be evaluated, it must only refer to effectively final variables
(the same rules as for lambda expressions apply).
The following should be illegal:
int index;
String data = STR."\(index=0), \(index++)";
The logging use-case could be expressed more naturally:
logger.trace("Number is \(getRandomNumber())");
I personally can see a parallel between \(...) and ()-> as both use
parentheses. This unfortunately is the exact opposite of the syntax of
closures in Groovy, where {123} is "lazy" and (123) produces the value
immediately. Different languages, different choices.
When should this lazy expression be evaluated, and who is responsible for
it? It can either be:
* when ts.values() is called;
* when the value is accessed (via ts.get(N) or .next() on the List's
Iterator);
* manually when it is consumed by the author of the policy using an
instanceof check for Supplier.
I keep this as an open question (first we need to find out if this is even
a useful feature). It also remains open what to do when a user provides a
Supplier manually:
Supplier s = () -> 123
logger.trace("Number is \{s} \(s)");
Another way to express a Supplier is via a method reference, which should
also be supported and would suggest the equivalence of the following:
logger.trace("Number is \{this::getRandomNumber}");
logger.trace("Number is \(this.getRandomNumber())");
Blocks
======
What if the computation of the value requires multiple statements?
STR."The intersection of \{a} and \{b} is \{((Supplier) ()-> {var c = new
HashSet<>(a); c.retainAll(b); return c;}).get()}"
STR."The intersection of \{a} and \{b} is \{switch(1){default:var c = new
HashSet<>(a); c.retainAll(b); yield c;}}"
These are (as far as my imagination goes) the easiest ways to turn a block
into an expression.
What if we introduced a special syntax for blocks? One option would be
simply doubling the braces:
STR."The intersection of \{a} and \{b} is \{{var c = new HashSet<>(a);
c.retainAll(b); yield c;}}"
The last closing brace is probably not needed but keeping them balanced
will help dumb editors with highlighting.
This could be a more general syntax useful even outside TemplatedStrings
(especially in final field initialization):
var intersection = {var c = new HashSet<>(a); c.retainAll(b); yield c;};
Lazy Blocks
===========
We may want to combine the two proposed features and compute the
intersection lazily when logging:
logger.trace("The intersection of \{a} and \{b} is \({var c = new
HashSet<>(a); c.retainAll(b); return c;})")
which would be equivalent to this already supported syntax provided that
the policy understands Supplier values:
logger.trace("The intersection of \{a} and \{b} is \{() -> {var c = new
HashSet<>(a); c.retainAll(b); return c;}}")
If we introduce \( as described in Lazy Logging above and say that
\(expression) is equivalent to \{() -> expression}, then \({statements})
would naturally be equivalent to \{() -> {statements}}.
Literals
========
It was already mentioned in the JEP that it is possible to create a policy
returning a JSON object. We can also think of them as a way to express
literals. Some Java types already have a way to parse a String. Some
examples are:
DURATION."PT20.345S"
Duration.from("PT20.345S")
COMPLEX."1+3i"
Complex.from(1, 3)
US_DATE."7/4/2022"
LocalDate.of(2022, 7, 4)
This is almost as powerful as what C++ offers (
https://urldefense.com/v3/__https://en.cppreference.com/w/cpp/language/user_literal__;!!ACWV5N9M2RV99hQ!M4Ng7vA4iXTBrb9WEbUiISXGems-vPKZjn_H7xYL5Bbhry1gPTadmf4z-QnZ-RhLk6gq_ZmX8GzTDU3t_j0L$ ).
Conclusion
==========
I am very happy with the current proposal and tried to explore some
possible extensions. I proposed a syntax for lazy evaluation of values,
blocks as values, and a combination thereof. I would like to hear from you,
whether this is something worth considering, and whether there is demand
for it.
Regards
Adam
More information about the amber-dev
mailing list