Call for Dicussion: JEP: Java Expression Trees API
-
liangchenblue at gmail.com
Mon Apr 22 02:11:49 UTC 2024
Hi Konstantin,
What you propose has a large overlap with Project Babylon (
https://openjdk.org/projects/babylon/), which accomplishes "code
reflection" from the Java compiler. The project itself also ships a code
model that's suitable for representing ASTs from different programming
languages, including ones from Java bytecode. Then your proposal would be
much simpler - to generate an expression tree from class files.
Since the discuss list isn't really for development questions, I am
replying to babylon-dev mailing list instead and we will sail from there.
Regards
On Sun, Apr 21, 2024 at 8:32 PM Konstantin Triger <kostat at gmail.com> wrote:
> Author: Konstantin Triger
> Type: Feature
> Scope: SE
> Relates to: JEP 466: Class-File API (Second Preview)
> <https://openjdk.org/jeps/466>
> Template: 2.0
>
> Summary
> -------
>
> Provide a standard API for parsing and semantic analysis of Java methods.
>
> Goals
> -----
>
> * Provide an API to convert the bytecode of a Java method into an
> expression tree (AST) suitable for semantic analysis in runtime.
>
> Non-Goals
> ---------
>
> * It is not a goal to obsolete existing libraries or Class-File API.
> Rather, the intention is to provide a higher-level abstraction.
> * It is not a goal to support all the possible language constructs. Non
> logical instructions, such as try/catch, etc are out of scope.
>
> Motivation
> ----------
>
> Programs usually go beyond the boundaries of a single execution
> environment and delegate part of their logic to external servers such as
> databases.
> Even if executed within the same environment (process), there might be
> engines that do not understand Java bytecode and require input in another
> language, DSL or a configuration file.
> In a common Java program, a lot of configuration is supplied via external
> files or spread in annotations, making them hard to find and maintain.
>
> While Java and other ecosystem languages excel in expressing logic or
> instructions in general, this *_information_* is extremely hard to get
> and process in runtime. This makes it practically impossible to use Java to
> _*express the logic_*, but _*delegate_* its execution to an external
> engine, i.e. convert Java bytecode to SQL, another DSL or provide a "fluent
> configuration".
>
> Consider the following use cases:
> 1. SQL execution
> ```java
> JPA.SQL((Person p) -> {
> // intended for execution inside an SQL database
> SELECT(p);
> FROM(p);
> WHERE(p.getName() == name);
> });
> ```
>
> 2. DSL creation
> ```java
> QueryBuilder<Restaurant> builder = Mongo.queryBuilder(Restaurant.class);
>
> // the following lambda is converted to Bson
> Bson filter = builder.filter(r -> r.getStars() >= 2 && r.getStars() < 5
> &&
> r.getCategories().contains("Bakery"));
> ```
>
> 2.1 What are the options today?
> Let's take an example of the new Jakarta API for NoSQL databases:
> https://www.jnosql.org/javadoc/jakarta.nosql.core/jakarta/nosql/Template.html
>
> An example of a select method looks like this:
> ```java
> List<Book> books = template.select(Book.class)
> .where("author")
> .eq("Joshua Bloch")
> .and("edition")
> .gt(3)
> .results();
> ```
>
> But could be this:
> ```java
> var author = "Joshua Bloch";
> var edition = 3;
>
> // Under the hood use the Expression Trees API to parse the lambda
> List<Book> books = template.select((Book b) -> b.getAuthor() == author &&
> b.getEdition() == edition).results();
> ```
>
> 3. Fluent Configuration
> ```java
> // Configure DateOfBirth column
> modelBuilder.entity<Student>()
> .property(p::getDateOfBirth) // use Expression Trees API to
> get the property name
> .hasColumnName("DOB")
> .hasColumnOrder(2)
> .hasColumnType("datetime2");
> ```
>
> Description
> ------------
>
> ### The following design goals and principles used for the Expression
> Trees API:
> * The expression tree can be created in runtime by passing a lambda
> instance or java.lang.reflection.Method instance with the corresponding
> class instance, if the passed method is an instance method.
> * The expression tree is a logical/semantic representation of a method
> body, including the method parameters. Possible node types are not tied to
> Java language and able to express a construct in any imperative language
> compiled to Java bytecode.
> * To be successfully parsed, the method must not contain any non-supported
> constructs; otherwise a runtime exception is thrown.
> * Methods accepting Lambdas intended for parsing can be marked with an
> annotation, thus tools and IDEs can have a fair ability to identify lambdas
> with non-supported constructs.
> * The expression tree is immutable. This facilitates reliable sharing when
> a tree is being analyzed or transformed.
>
> ### Nodes and Expression Types
>
> Nodes are Java classes. Each node may have a different Expression Type.
> For example, `BinaryExpression` node might have type `Add` or `Divide`.
>
> #### Nodes
> ```
> - abstract Expression - provides the base class from which the classes
> that describe expression tree
> nodes are derived. It also contains static
> factory methods to create the
> various node types.
> - ConstantExpression extends Expression - Describes an expression that
> has a constant value.
> - UnaryExpression extends Expression - Describes an expression that has
> a unary operator.
> - BinaryExpression extends UnaryExpression - Describes an expression
> that has a binary operator.
> - ParameterExpression extends Expression - Describes an indexed
> argument or parameter expression.
> - abstract InvocableExpression extends Expression - Describes an
> expression that can be invoked by applying
> 0 or more arguments and might return a result.
> - MemberExpression extends InvocableExpression - Describes accessing a
> field or method.
> - LambdaExpression<F> extends InvocableExpression - Describes a lambda
> expression.
> Captures a block of code that is equivalent to a
> method body.
> - DelegateExpression extends InvocableExpression - Describes a higher
> order construct, where InvocableExpression
> is chained.
> - InvocationExpression extends Expression - Describes an expression
> that applies a list of argument expressions
> to an InvocableExpression
> - BlockExpression extends Expression - Describes a sequence of
> expressions
> - NewArrayInitExpression extends Expression - Describes a
> one-dimensional array and initialising it from a list of elements.
> ```
>
> #### Expression Types
> ```
> - Add // A node that represents arithmetic addition.
> - BitwiseAnd // A node that represents a bitwise AND operation.
> - LogicalAnd // A node that represents a logical AND operation.
> - ArrayIndex // A node that represents indexing into an array.
> - ArrayLength // A node that represents getting the length of an array.
> - Coalesce // A node that represents a null coalescing operation.
> - Conditional // A node that represents a conditional operation.
> - Constant // A node that represents an expression that has a
> constant value.
> - Convert // A node that represents a cast or conversion operation.
> - Divide // A node that represents arithmetic division.
> - Equal // A node that represents an equality comparison.
> - ExclusiveOr // A node that represents a bitwise XOR operation.
> - GreaterThan // A node that represents a "greater than" numeric
> comparison.
> - GreaterThanOrEqual // A node that represents a "greater than or
> equal" numeric comparison.
> - Invoke // A node that represents application of a list of
> argument expressions to an InvocableExpression.
> - IsNull // A node that represents a null test.
> - IsNonNull // A node that represents a non null test.
> - Lambda // A node that represents a lambda expression.
> - Delegate // A node that represents a lambda chain expression.
> - LeftShift // A node that represents a bitwise left-shift operation.
> - LessThan // A node that represents a "less than" numeric
> comparison.
> - LessThanOrEqual // A node that represents a "less than or equal"
> numeric comparison.
> - FieldAccess // A node that represents reading from a field.
> - MethodAccess // A node that represents a method call.
> - Modulo // A node that represents an arithmetic remainder
> operation.
> - Multiply // A node that represents arithmetic multiplication.
> - Negate // A node that represents an arithmetic negation
> operation.
> - New // A node that represents calling a constructor to
> create a new object.
> - NewArrayInit // An operation that creates a new one-dimensional
> array and initializes it from a list of elements.
> - BitwiseNot // A node that represents a bitwise complement operation.
> - LogicalNot // A node that represents a logical NOT operation.
> - NotEqual // A node that represents an inequality comparison.
> - BitwiseOr // A node that represents a bitwise OR operation.
> - LogicalOr // A node that represents a logical OR operation.
> - Parameter // A node that represents a parameter index defined in
> the context of the expression.
> - RightShift // A node that represents a bitwise right-shift
> operation.
> - Subtract // A node that represents arithmetic subtraction.
> - InstanceOf // A node that represents a type test.
> - Block // A node that contains a sequence of other nodes.
> ```
>
> ### Entry Point
>
> `LambdaExpression` will introduce a static method that returns a parsed
> Expression Tree:
>
> ```java
> public static <T> LambdaExpression<T> parse(T lambda) {...}
> ```
>
> ### Analysis and transformation
>
> There is a builtin support for a visitor pattern via `ExpressionVisitor`
> interface that defines `visit` methods for all the non-abstract node
> classes.
> In addition, there is an `abstract SimpleExpressionVisitor implements
> ExpressionVisitor`. By default, it performs a recursive traversal of
> expressions and their sub-expressions.
> If none is modified an original expression is returned; if any
> sub-expression is modified, then a new expression is created, recursively.
>
> With this a member expression transformer might look like this:
>
> ```java
> val transformer = new SimpleExpressionVisitor() {
> @Override
> public Expression visit(MemberExpression e) {
> ...
> return <transformed expression>
> }
> };
> ```
>
> Let's consider a hypothetical fluent builder that needs to extract
> property name:
> ```java
> Fluent<Customer> f = new Fluent<Customer>();
> f.property(Customer::getData)...;
> ```
>
> To extract the Member, the user will need to override a single `visit()`
> method. Like the following:
>
> ```java
> class MemberExtractor extends SimpleExpressionVisitor {
> private MemberExpression memberExpression;
>
> @Override
> public Expression visit(MemberExpression e) {
> memberExpression = e;
> return e;
> }
> }
> ```
>
> A more complete implementation for the Fluent class above can be found
> here
> <https://github.com/streamx-co/ExTree/blob/master/src/test/java/co/streamx/fluent/extree/Fluent.java>
> .
>
> ### Other Java ecosystem languages and "special" methods
>
> Some constructs do not have a direct translation into the Java byte code
> and are implemented using a runtime library method call. The runtime
> library developer might be interested in translating a method call
> expression into a different expression.
> Consider the following code:
> ```java
> var b1 = new BigDecimal(67891);
> var b2 = new BigDecimal(12346);
> if (b1.compareTo(b2) == 0) {
> ...
> ```
>
> There should be a possibility for a library developer to transform an
> expression calling to `compareTo()` method and then comparing to zero into
> a logical comparizon expression between the two decimals.
> Currently the mechanism to achieve this is not specified in this draft,
> but eventually must be introduced in this or a follow-up JEP.
>
> ### Practical usability / Existing art
>
> To be successful in achieving the JEP goals, there is a need for
> comprehensive POC projects.
>
> I have created 3 prototype projects:
> 1. ExTree <https://github.com/streamx-co/ExTree>. The project prototypes
> this JEP on top of ASM.
> 2. FluentJPA <https://github.com/streamx-co/FluentJPA>. The project uses
> Java to write SQL and integrates with JPA.
> 3. FluentMongo <https://github.com/streamx-co/FluentMongo>. The project
> uses Java to write Mongo queries i.e. Mongo BSON documents.
>
> These projects helped to shape this API.
>
> Testing
> ----
>
> TBD
>
> Alternatives
> -----
>
> An obvious alternative is to not integrate this into JRE and keep it as a
> separate project.
> Being considered low level and possibly "fragile" technology, it will be
> hardly accepted into the mainstream projects where it might serve the large
> Java community.
>
> Dependencies
> ----
> JEP 466
>
> --
> Regards,
> Konstantin Triger
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/babylon-dev/attachments/20240421/ab3df782/attachment-0001.htm>
More information about the babylon-dev
mailing list