Initial and partial high-level REPL API review

[Paul Sandoz]

Hi,

I am not diving too much into the current JavaDoc at the moment. Looking more at higher level things and relationships between types. Expect some naive questions :-) In no particular order below...

Paul.


Concurrency
--
It's not clear to me what if any methods on JShell are thread safe, except for JShell.stop. I see certain methods on JShellImpl are marked synchronized, implying there is potentially some concurrent activity.

Can callback notifications occur on a different thread that registered the listener? same for unsubscribing a listener.


Naming
--
It's bike shed time! :-)

When i looked at the JavaDoc i could see many "Key" thingies. I had no idea what they could be at first glance, as it's such a generic name. When dug just a little deeper it became clear they are references to snippets of code. Why not call it CodeSnippet?

  CodeSnippet
  CodeSnippet.Declaration
  CodeSnippet.Erroneous
  CodeSnippet.Import
  etc.

  KeyStatusEvent -> CodeSnippetEvent

There seems no need to add CodeSnippet to the end of each nested sub-type, since one can use the root type as the namespace.

JShell.SubscriptionToken -> JShell.Subscription ?


Interfaces vs. Abstract classes
--

The current design is that one obtains a code snippet or subscription from the shell and then one goes back to the shell to operate on that snippet or subscription.  This means the developer might need to guard against an IllegalArgumentException under two conditions: 1) The snippet or subscription was created by something foreign; or 2) was created by another JShell instance.

For code snippets can their be a hierarchy of abstract classes, each with a package private constructor. AFAICT i think that would currently work. Would this box us in if new code snippets are added in the future?

Particular code snippets can have methods associated with the operations, for example:

  - All code snippets would have the source, status, diagnostics and subKind operations
  - The variable declaration code snippet would have the varValue operation

It would then be marginally easier to operate on snippets in bulk, rather than having to go back to corresponding JShell instance. For example, then one could do:

  JShell js = ..

  Stream< VariableDeclKey > variables = js.keys().stream()
      .filter(k -> k.status().tracksUpdates)
      .filter(k -> k.kind() == Key.Kind.VARIABLE)
      .map(k -> (VariableDeclKey)k);

with no need to refer or capture "js". The stream is entirely self-contained.

The same patterns can apply to a subscription. A subscription could be an abstract class and have the unsubscribe operation.


JShell construction
--

There is one creation method:

  public static JShell create(InputStream in, PrintStream out, PrintStream err)

But i dunno what it does with "in", "out, or "err". If i wanted to write my own jshell tool how would i use this to trigger reading from "in"?

I think this points to the general question of how can i write my own equivalent of jshell using this API?

I am guessing i would use the source code analysis functionality to determine if a code snippet string is complete and when so pass that string to JShell.eval. Correct? If so and if possible concisely it might be useful to have a simple and dumb example in the JavaDoc, which might also be a useful exercise (say at a more advanced hackathon).


JShell use-cases
--

I think the primary use-case so far for this API is to support jshell-like tools. Another good use-case is actual testing without having to open up the box.

There may be other use-cases, more specifically that don't require redefinition. For example, could we implement a Java scripting provider? It seems possible but i think the API is lacking a way to hook up runtime state to a JShell instance. The only way to hook up state is to load up a JShell via evaluations. So it's kind of a closed world mechanism, fine for the jshell tool, but perhaps for other use-cases not so if we deem them important.

It seems a viable use-case to me. For example, take the Jenkins continuous build application/system. It has the ability to to declare Groovy scripts and execute then server-side (i believe this is also the glue for the workflow plugin). Those groovy scripts require some context to the Jenkins instance. How could we hook up the JShell functionality to achieve a similar approach with Java?

Other use-cases might be IDEs. One obvious use-case is to evaluate in place in an editor window. Another use-case might be for evaluating expressions when debugging. I think these will require more input from the IDE vendors/developers.


Code snippets
--

Status events
-

What are the constraints for "previousStatus" and "status". I presume "previousStatus" can be of all Key.Status values, including NONEXISTENT, where as "status" can be anything but NONEXISTENT?

If "isSignatureChange" is true does that mean "key" is an instance of DeclarationKey?

If "isSignatureChange" is true is there a way to obtain the old signature?

Is there any meaningful relationship between the types of causeKey and key?


Kinds and SubKinds:
-

Is there actually a state machine for transitions of sub-kinds a key can undergo? I suspect there might be. Perhaps that is something that could be expressed as a table in the documentation? That might help get the bigger picture.


Evaluation
--

It took me a few iterations to realize the jshell command tool adds ';' where needed (e.g. variable declarations) where as the JShell.eval requires them. AFAICT there were no errors reported, just Status.REJECTED_FAILED.

Lesson learned: one should first analyze with SourceCodeAnalysis.analyzeCompletion to obtain Completeness.COMPLETE_WITH_SEMI.


Source code analysis
--

It's likely SourceCodeAnalysis.analyzeCompletion will need to be called in a loop e.g.:

String expressions = "int x = 1; int y = 2; System.out.printf(\"Output: %s %d%n\", x, y);";

SourceCodeAnalysis sca = js.sourceCodeAnalysis();
for (;;) {
    SourceCodeAnalysis.CompletionInfo ci = sca.analyzeCompletion(expressions);
    if (ci.completeness != SourceCodeAnalysis.Completeness.COMPLETE)
        break;

    String expression = ci.source;
    List<KeyStatusEvent> eval = js.eval(expression);
    eval.forEach(e -> {
       System.out.println(e.key);
        System.out.println(e.value);

    });

    expressions = ci.remaining;
}

I don't have any concrete suggestions at the moment but it feels (gut feeling) this experience could be improved. Need to play with it a little more.

Is there some non-string intermediate form produced from the source code analysis that could be passed to the evaluation for more efficient processing?