JEP 222: Java Read-Eval-Print Loop (REPL)

[Archie Cobbs]

(resent from the correct email address)

Hi Brian,

Here are some notes from my experience.

A little background... this was needed for a Java-centric database project.
Like any other database, it required a command line interface (CLI) where
you can perform database queries, filter results, perform maintenance
tasks, etc. With this database all queries, predicates, etc. are specified
using normal Java, so for the CLI a Java expression parser was needed. So
the equivalent of

  $ SELECT name
    FROM Person
    WHERE name LIKE `Jon%' AND registered = 1

would be

  $ eval foreach(
       filter(
           all(Person),
           $x,
           $x.name.startsWith("Jon") && $x.registered),
       $y,
       $y.name)

Here eval is a command that evaluates an expression, and this is what the
parser was needed for. Note the language for "expression" contains but is
larger than Java expressions, e.g., global functions such as filter(),
foreach(), and all(), and variable expressions $x and $y. Also, the
parameters to the global functions are not always Java expressions, e.g.
all() takes a database type name while filter() and foreach() require a
variable expression (not evaluated) as the second parameter.

I had to write a custom (tail-recursive) parser; existing libraries and
ANTLR were not sufficiently flexible. Some custom requirements and issues
encountered were...

- Application-supplied functions that exist in the top-level namespace
(noted above)

- Custom parse "atoms" with arbitrary syntax. Used for variables like $x as
well as other things. For example, object ID's, which are 64-bit values,
can be expressed using an at-sign literal notation like @64a80cffdd9e7b7f.

- Tab completion. This is a bit tricky. Internally, on any parse error we
throw a ParseException which contains the index of the error and
(optionally) valid completions from that point. These are used by the CLI
main loop for tab completion. This makes is relatively easy to add
pervasive tab completion support incrementally over time. One subtlety here
is that completions are not necessarily limited to syntax. For example, if
you enter eval @64a80cff<TAB> it performs an actual query for objects in
the database whose ID has the prefix 64a80cff. This invalidates the whole
idea of parsing to get a valid AST first, then interpreting it semantically
in a separate step.

- What about generics and erasure? Will you emit unchecked warnings, or
simply disallow unchecked expressions? My parser punts: we don't allow
generic syntax, everything is raw types, and any subsequent
ClassCastExceptions are normal errors just like any other. On the other end
of the spectrum, could you support carrying type information along with all
values, eliminating erasure altogether and opening up possibilities like foo
instanceof Set<? extends Bar>?

- Additional operator semantics. E.g., the && operator applied to two Sets
computes the union, > and < can be applied to Comparables, etc.

- Other syntactic sugar: allowing p.name for p.getName(), p.name = "foo"
for p.setName("foo"); allowing the [] operator for List and Map lookup, etc.

-Archie

On Sat, Nov 1, 2014 at 11:14 AM, Brian Goetz <brian.goetz at oracle.com> wrote:

> What might be useful, though, is any *usability* experience that you had
> with people trying it out.  What did people complain about?  What
> non-obvious features did people seem to expect should be part of the
> feature set?
>
> (Note, though, that we're asking for *actual* experience of "I built X and
> people who used it complained of Y", not theorizing of the form "I could
> imagine I might want Y in some situation...".)
>
> On 11/1/2014 12:03 PM, Robert Field wrote:
>
>> Thanks Bruce.  I'm quite far along, so I doubt that would be useful.
>>
>> -Robert
>>
>> On 11/01/14 01:58, Bruce Chapman wrote:
>>
>>> I built (for some value of "built" near to the value of "hack") one of
>>> these back in 2009 for our local java user group [1] as a demo for the
>>> compiler API and an annotation processor test framework I built [2]
>>>
>>> If you are interested I can find and dust off the code and make sure
>>> it still all works (why wouldn't it).
>>>
>>> Bruce
>>>
>>>
>>>
>>> [1]  http://wellingtonjug.org.nz/talk200905.html
>>> [2] https://java.net/projects/hickory
>>>
>>>
>>> On 31/10/2014 10:56 a.m., mark.reinhold at oracle.com wrote:
>>>
>>>> New JEP Candidate: http://openjdk.java.net/jeps/222
>>>>
>>>> - Mark
>>>>
>>>>
>>>
>>> ---
>>> This email is free from viruses and malware because avast! Antivirus
>>> protection is active.
>>> http://www.avast.com
>>>
>>>
>>


-- 
Archie L. Cobbs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20141102/76457a33/attachment.html>