Expression Language Prototyping

[Radosław Smogura]

Dear John and all,

I’m sorry for late answer but I moved a little bit far, and had to spend time on organising personal stuff.

Let me firstly correct one thing. I have not exposed JavaC AST in java.base. The “javax.expression” is intended to be public, because 
(1) It represents the expression which should be readable by any executor;
(2) Can be used to programatically build expression - dynamic queries, dynamic CUDA kernels, an so on...

For the rest I think we are on same page. 

Actually when I was looking for some good representation of IR, I’ve found one together with parser, both created by people 100x smarter than me those is JVM specification and JVM itself. My blocker was how to deal with constants. Using BC, I can get classes and methods (handles) from pool, with LDC opcode which may compensate cost of executable part.

John, I’m really glad that you want to engage and I hope that IR will be improved.

Meantime I would like to move forward, not with IR, but other stuff, as I would like to “close” flow from compilation to translating expression and see how it works real life. Maybe today I would be able to submit something around retrieving expression in more handy way.

Thanks and have a nice day,
Radek

P.S. Writing this I’ve got one idea, which maybe could be useful if we would like to avoid changing class file. I’m thinking about creating a “builder” method, but returning a double-dereferenced constant pool, and “token codes”, I.e:
return new javax.expression.ExpressionDescriptor(
	“Token codes kept as string as better handled by CLASS file”
	// Double-dereferenced constant pool
        ldc some class, 
        ldc some handle, ldc something else
)


> On 28 Jun 2017, at 15:08, John Rose <john.r.rose at oracle.com> wrote:
> 
> On Jun 27, 2017, at 5:25 PM, John Rose <john.r.rose at oracle.com> wrote:
>> 
>> Thank you!  Decoding lambda body bytecodes is the best we can do now,
>> and it might be good enough for prototyping work.
> 
> I looked at your test case (BasicMatadataTest.java) and see that you
> tap into the AST before it gets to bytecodes, so I retract my question
> about your experience with unwinding bytecodes.
> 
> I see you move the javac AST APIs into java.base to do this.  I would
> rather keep the APIs separate and import a lower-level representation
> into java.base.  So instead of tapping into javac and getting its more
> or less raw tree representation, I think we need to define an IR that
> is suited to java.base.  I have one in mind, called "token codes".
> 
> The idea is to represent each node in the AST as a constant in the constant
> pool, and compose nodes concatenatively by positing a stack machine
> that they execute on.  So "() -> 2 + 2" would be "push 2; push 2; int/plus"
> where the last thing is a (standard) MH constant defined in java.base.
> And "(x) -> x + x" would be "dup; int/plus", and so on.
> 
> Such a representation is abstract, native to the JVM (via BSM argument
> sequences), untied to javac internals, easy to lower to bytecodes, easy
> (I hope) to raise to semantically equivalent ASTs.  Other details:  Integer
> constants are ad hoc opcodes; MHs self-invoke; everything else self-pushes.
> (Think Forth.)  Local side effects and temporary management via stack
> dup/swap/pop.  Control flow using MH combinators with explicit effects
> transfer (equals SSA).
> 
> I have a much more concrete proposal for this in the works; please
> stay tuned.  Disclaimer:  This stuff is really early and raw, for experimentation
> only.  But it may provide a way to experiment with "wiring together" the
> kind of AST capture stuff like your prototype, with low-level code spinner
> running out of java.base.
> 
> The same points could be made for your stuff; it's partly a matter of taste
> what IR to "push" through the class-file.  And that IR can be adjusted later,
> so I'm actually pretty impressed by your approach also, of code-generating
> tree-builders, which gives a fast route to prototyping with lambda cracking.
> But I don't think it scales; IMO the IR in the class file has to be more compact,
> and that means more natural.
> 
> — John