Project Leyden: Beginnings

[Remi Forax]

----- Original Message -----
> From: "John Rose" <john.r.rose at oracle.com>
> To: "Ioi Lam" <ioi.lam at oracle.com>
> Cc: "leyden-dev" <leyden-dev at openjdk.java.net>
> Sent: Tuesday, May 24, 2022 8:30:32 PM
> Subject: Re: Project Leyden: Beginnings

> On 23 May 2022, at 9:10, Ioi Lam wrote:
> 
> (more)
>> …
>> One parallel to draw from is the "constexpr" keyword in C++.
> 
> Please take a look also at the ideas in D around pure and immutable
> computations.  They allow time-shifting of very complicated D programs
> to compile time.  I saw a demo (long ago) of compile-time data weaving
> in D which took an immutable bundle of strings, transformed them and ran
> the result as D expressions through the D compiler itself (as a subtask,
> at compile-time), and took the resulting output as further input to
> incorporate into the D program. Basically, it was as if the D
> preprocessor suddenly was a metaprogramming framework.  And it worked,
> not as a special hack, but as a corollary of very cleanly worked out
> D-language rules for purity and immutability, plus the fact that much of
> the D standard libraries (including the D compiler) were pure enough to
> play these games with.
> 
> Reference: https://dlang.org/spec/function.html#pure-functions
> 
> (I see from dlang.org they have fancy templates now.  They probably
> interoperate well with ad hoc compile-time computations.  The C++
> constexpr stuff is moving that way too, I guess.)
> 
> Also, unlike a compiled language, we have a virtual machine that can (in
> principle) be asked to verify purity of methods on the fly.  This means
> we can (in principle) have pure functions which are separately linked,
> and do not need to be dumped into the current compilation.  Of course
> native methods and Panama downcalls would have to either be rejected or
> manually certified, but that’s all part of the game.

yes, Zig does somethoing similar too.
https://kristoff.it/blog/what-is-zig-comptime/

> 
>> However, "constexpr" only deals with language-level constructs. For
>> Java, perhaps we need something that includes a wider set of
>> environmental dependencies.
> 
> Yes.  (I have thought for some years that our keyword `const` has been
> waiting to be used to annotate time-shifted computations.  That’s just
> bikeshedding of course.)  I think, given Java’s embrace of dynamic
> linking, it might make sense to define an idea of a time-shifted *value*
> as well as a time-shifted expression or statement.  By that I mean a
> normal method could (perhaps) declare some but not all of its
> *parameters* as `static` (or `constexpr` or whatever) with the meaning
> that it is requesting that the corresponding actual arguments be
> time-shifted (if possible) at every invocation point of that method.
> Then a dynamically linked method could still partially play the
> time-shifting game, in some of its parameters.  (And similarly for local
> variables.  Sort of a “better static” with a dependency-driven
> initialization order.)  If a method’s formal parameter is marked
> `static`, then expressions using that parameter inside the method are
> also candidates for early evaluation.  (This means the JVM or somebody
> has to keep track of separate derived values for each method call site.
> Doable but tricky.)  This might give a framework to thread through all
> the pre-evaluated values, through an application workload, without
> disrupting the logic by partitioning it into disjoint “before” and
> “after” phases.

The problem of any keywords like constexpr is that it does not work well
when you have libraries in the middle (like an XML parser) and you
requires everything to be transitively a constexpr.

> 
> And I think all of the above works about as well, not only for
> time-shifting back in time to pre-evaluation in jlink, but also for
> time-shifting forward in time to lazy evaluation.  Java already has lots
> of lazy evaluation in it, notably on-demand class initialization and
> (under the covers) condy/indy.  If we had a way to mark program portions
> as time-shiftable, that would naturally parley out into a way to work
> with lazy computations, as well as pre-evaluated ones.

I believe we can conflate the two by saying that an expression can be evaluated whenever it suits the jlink/runtime because in both cases you want something that relax the initialization order.

> 
> This is the way I would prefer to handle recurrent requests (from me and
> others) for APIs which help string templates to support syntax-specific
> validation (SQL, XML, etc.).  Such validation, for a constant template,
> should happen as early as possible, ideally in the IDE, and certainly at
> jlink time.  Time-shifting can be a foundation for static validation.

yes !
with less syntactic sugar please (i wonder if there is a rehab for syntactic sugar ?).

Rémi