Initial and partial high-level REPL API review

[Paul Sandoz]

On Jul 4, 2015, at 7:47 AM, Robert Field <robert.field at oracle.com> wrote:
>> 
>> 
>> Naming
>> --
>> It's bike shed time! :-)
>> 
>> When i looked at the JavaDoc i could see many "Key" thingies. I had no idea what they could be at first glance, as it's such a generic name. When dug just a little deeper it became clear they are references to snippets of code. Why not call it CodeSnippet?
>> 
>> CodeSnippet
>> CodeSnippet.Declaration
>> CodeSnippet.Erroneous
>> CodeSnippet.Import
>> etc.
> 
> Brian has responded.  That is where I started.
> Here is one old doc version (for kicks): http://web-lands.com/kulla/doc/
> 

Ok, some related stuff below.


> 
>> 
>> 
>> Interfaces vs. Abstract classes
>> --
>> 
>> The current design is that one obtains a code snippet or subscription from the shell and then one goes back to the shell to operate on that snippet or subscription.  This means the developer might need to guard against an IllegalArgumentException under two conditions: 1) The snippet or subscription was created by something foreign; or 2) was created by another JShell instance.
> 
> These were interfaces, very recently changes to abstract classes at Brian’s suggestion to allow future additions.  But currently they might as well be interfaces.
> 
> We could/should disallow foreign implementations and thus (1) as you suggested by making it a package-private constructor and folding the implementation package into the API level.  Or going further, folding the implementations into then final classes.
> 
> Thoughts?

If moving to abstract classes does not bodge any future evolution it seems worthwhile using package-private constructors and thus not restricting for internal sub-classes.


> 
>> 
>> For code snippets can their be a hierarchy of abstract classes, each with a package private constructor. AFAICT i think that would currently work. Would this box us in if new code snippets are added in the future?
>> 
>> Particular code snippets can have methods associated with the operations, for example:
>> 
>> - All code snippets would have the source, status, diagnostics and subKind operations
>> - The variable declaration code snippet would have the varValue operation
>> 
>> It would then be marginally easier to operate on snippets in bulk, rather than having to go back to corresponding JShell instance. For example, then one could do:
>> 
>> JShell js = ..
>> 
>> Stream< VariableDeclKey > variables = js.keys().stream()
>>     .filter(k -> k.status().tracksUpdates)
>>     .filter(k -> k.kind() == Key.Kind.VARIABLE)
>>     .map(k -> (VariableDeclKey)k);
>> 
>> with no need to refer or capture "js". The stream is entirely self-contained.
> 
> As above.
> 
>> 
>> The same patterns can apply to a subscription. A subscription could be an abstract class and have the unsubscribe operation.
> 
> Currently JShell is the only thing with a life-span.  Key and subclasses have queries, but those are immutable values.

There are also those mildly annoying NPEs one has to deal with too.

It seems an artificial distinction to have to go back to the shell of where the code snippet was obtained. I took a look at the implementation; it may or may not confirm my hunch, since the JShell methods actual defer to the snippet :-) e.g.:

  @Override
  public String source(Key key) {
      return checkValidKey(key).source();
  }

I get a code snippet, and i want to operate on it. I think this somewhat connects with the naming, which is why i tend not to think of them as keys, they are just references to code snippets, where some state of which is constant and other state of which is not.


> 
> I see this could save one method on JShell, but at a cost of consistency.
> 
>> 
>> 
>> JShell construction
>> --
>> 
>> There is one creation method:
>> 
>> public static JShell create(InputStream in, PrintStream out, PrintStream err)
>> 
>> But i dunno what it does with "in", "out, or "err". If i wanted to write my own jshell tool how would i use this to trigger reading from "in”?
> 
> These are the in, out, and err of the executing code.
> 

How would i trigger a JShell instance to read from "in"? Or is that an implementation detail of the jshell tool leaking into the API? (i.e usage by jdk.internal.jshell.tool.JShellTool).


>> 
>> I think this points to the general question of how can i write my own equivalent of jshell using this API?
>> 
>> I am guessing i would use the source code analysis functionality to determine if a code snippet string is complete and when so pass that string to JShell.eval. Correct?
> 
> Yep.
> Or a simpler tool might even skip the first step.
> 
>> If so and if possible concisely it might be useful to have a simple and dumb example in the JavaDoc, which might also be a useful exercise (say at a more advanced hackathon).
> 
> Should be somewhere.
> 
>> 
>> 
>> JShell use-cases
>> --
>> 
>> I think the primary use-case so far for this API is to support jshell-like tools.
> 
> Yes, possibly with GUIs or integrated in IDEs.
> 
>> Another good use-case is actual testing without having to open up the box.
> 
> Indeed.
> 
>> 
>> There may be other use-cases, more specifically that don't require redefinition. For example, could we implement a Java scripting provider? It seems possible but i think the API is lacking a way to hook up runtime state to a JShell instance. The only way to hook up state is to load up a JShell via evaluations. So it's kind of a closed world mechanism, fine for the jshell tool, but perhaps for other use-cases not so if we deem them important.
> 
> Possible futures, I don’t want to open that can of worms now.

Ok. I anticipate there might be demand for this at some point.


> 
>> 
>> It seems a viable use-case to me. For example, take the Jenkins continuous build application/system. It has the ability to to declare Groovy scripts and execute then server-side (i believe this is also the glue for the workflow plugin). Those groovy scripts require some context to the Jenkins instance. How could we hook up the JShell functionality to achieve a similar approach with Java?
> 
> Yes, I like.
> 
>> 
>> Other use-cases might be IDEs. One obvious use-case is to evaluate in place in an editor window. Another use-case might be for evaluating expressions when debugging. I think these will require more input from the IDE vendors/developers.
>> 
>> 
>> Code snippets
>> --
>> 
>> Status events
>> -
>> 
>> What are the constraints for "previousStatus" and "status". I presume "previousStatus" can be of all Key.Status values, including NONEXISTENT, where as "status" can be anything but NONEXISTENT?
> 
> Yes
> 
>> 
>> If "isSignatureChange" is true does that mean "key" is an instance of DeclarationKey?
> 
> Yes
> 
>> 
>> If "isSignatureChange" is true is there a way to obtain the old signature?
> 
> Cache it
> 
>> 
>> Is there any meaningful relationship between the types of causeKey and key?
> 
> Nope
> 

Ok. Perhaps where appropriate the JavaDoc could be embellished with your answers?


>> 
>> 
>> Kinds and SubKinds:
>> -
>> 
>> Is there actually a state machine for transitions of sub-kinds a key can undergo? I suspect there might be. Perhaps that is something that could be expressed as a table in the documentation? That might help get the bigger picture.
> 
> Since the user can enter any input, the transition could be any to any (within the Kind)
> 

So basically for a kind of say TYPE_DECL the set of sub-kinds are {CLASS_SUBKIND, INTERFACE_SUBKIND, INTERFACE_SUBKIND} and such a snippet can transform from one to another one in that set.

I had to look at the code to get a better idea of the classifications. I found that easier than drilling down to the enum constant detail section of the JavaDoc. So perhaps some table would be useful at the top-level enum doc.


>> 
>> 
>> Evaluation
>> --
>> 
>> It took me a few iterations to realize the jshell command tool adds ';' where needed (e.g. variable declarations) where as the JShell.eval requires them. AFAICT there were no errors reported, just Status.REJECTED_FAILED.
>> 
>> Lesson learned: one should first analyze with SourceCodeAnalysis.analyzeCompletion to obtain Completeness.COMPLETE_WITH_SEMI.
>> 
>> 
>> Source code analysis
>> --
>> 
>> It's likely SourceCodeAnalysis.analyzeCompletion will need to be called in a loop e.g.:
> 
> Yes
> 
>> 
>> String expressions = "int x = 1; int y = 2; System.out.printf(\"Output: %s %d%n\", x, y);";
>> 
>> SourceCodeAnalysis sca = js.sourceCodeAnalysis();
>> for (;;) {
>>   SourceCodeAnalysis.CompletionInfo ci = sca.analyzeCompletion(expressions);
>>   if (ci.completeness != SourceCodeAnalysis.Completeness.COMPLETE)
>>       break;
>> 
>>   String expression = ci.source;
>>   List<KeyStatusEvent> eval = js.eval(expression);
>>   eval.forEach(e -> {
>>      System.out.println(e.key);
>>       System.out.println(e.value);
>> 
>>   });
>> 
>>   expressions = ci.remaining;
>> }
>> 
>> I don't have any concrete suggestions at the moment but it feels (gut feeling) this experience could be improved. Need to play with it a little more.
>> 
>> Is there some non-string intermediate form produced from the source code analysis that could be passed to the evaluation for more efficient processing?
> 
> Source code analysis is optional.  And Keep it Simple says String.
> 

Ok.

Paul.