Project Leyden: Beginnings

[John Rose]

On 23 May 2022, at 9:10, Ioi Lam wrote:

(more)
> …
> One parallel to draw from is the "constexpr" keyword in C++.

Please take a look also at the ideas in D around pure and immutable 
computations.  They allow time-shifting of very complicated D programs 
to compile time.  I saw a demo (long ago) of compile-time data weaving 
in D which took an immutable bundle of strings, transformed them and ran 
the result as D expressions through the D compiler itself (as a subtask, 
at compile-time), and took the resulting output as further input to 
incorporate into the D program. Basically, it was as if the D 
preprocessor suddenly was a metaprogramming framework.  And it worked, 
not as a special hack, but as a corollary of very cleanly worked out 
D-language rules for purity and immutability, plus the fact that much of 
the D standard libraries (including the D compiler) were pure enough to 
play these games with.

Reference: https://dlang.org/spec/function.html#pure-functions

(I see from dlang.org they have fancy templates now.  They probably 
interoperate well with ad hoc compile-time computations.  The C++ 
constexpr stuff is moving that way too, I guess.)

Also, unlike a compiled language, we have a virtual machine that can (in 
principle) be asked to verify purity of methods on the fly.  This means 
we can (in principle) have pure functions which are separately linked, 
and do not need to be dumped into the current compilation.  Of course 
native methods and Panama downcalls would have to either be rejected or 
manually certified, but that’s all part of the game.

> However, "constexpr" only deals with language-level constructs. For 
> Java, perhaps we need something that includes a wider set of 
> environmental dependencies.

Yes.  (I have thought for some years that our keyword `const` has been 
waiting to be used to annotate time-shifted computations.  That’s just 
bikeshedding of course.)  I think, given Java’s embrace of dynamic 
linking, it might make sense to define an idea of a time-shifted *value* 
as well as a time-shifted expression or statement.  By that I mean a 
normal method could (perhaps) declare some but not all of its 
*parameters* as `static` (or `constexpr` or whatever) with the meaning 
that it is requesting that the corresponding actual arguments be 
time-shifted (if possible) at every invocation point of that method.  
Then a dynamically linked method could still partially play the 
time-shifting game, in some of its parameters.  (And similarly for local 
variables.  Sort of a “better static” with a dependency-driven 
initialization order.)  If a method’s formal parameter is marked 
`static`, then expressions using that parameter inside the method are 
also candidates for early evaluation.  (This means the JVM or somebody 
has to keep track of separate derived values for each method call site.  
Doable but tricky.)  This might give a framework to thread through all 
the pre-evaluated values, through an application workload, without 
disrupting the logic by partitioning it into disjoint “before” and 
“after” phases.

And I think all of the above works about as well, not only for 
time-shifting back in time to pre-evaluation in jlink, but also for 
time-shifting forward in time to lazy evaluation.  Java already has lots 
of lazy evaluation in it, notably on-demand class initialization and 
(under the covers) condy/indy.  If we had a way to mark program portions 
as time-shiftable, that would naturally parley out into a way to work 
with lazy computations, as well as pre-evaluated ones.

This is the way I would prefer to handle recurrent requests (from me and 
others) for APIs which help string templates to support syntax-specific 
validation (SQL, XML, etc.).  Such validation, for a constant template, 
should happen as early as possible, ideally in the IDE, and certainly at 
jlink time.  Time-shifting can be a foundation for static validation.