JShell: source in langtools vs JDK?
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Thu Sep 17 09:43:06 UTC 2015
Here's a more precise description of what you'd run into if you would
like to run jshell on top of an existing JDK; assumption:
<JDK9_HOME> : points to a valid JDK 9 binary
<KULLA_JDK_BUILD> : points to a valid Kulla exploded build
#1 run jshell - just add jshell (and langtools) to bootstrap classes:
$ <JDK9_HOME>/bin/java
-Xbootclasspath/p:<KULLA_JDK_BUILD>/jdk/modules/*jdk.jshell*:<KULLA_JDK_BUILD>/jdk/modules/*jdk.compiler*:<KULLA_JDK_BUILD>/jdk/modules/*java.compiler*jdk.internal.jshell.tool.JShellTool
Exception in thread "main" java.lang.NoClassDefFoundError:
jdk/internal/jline/TerminalFactory$Flavor
at
jdk.internal.jshell.tool.ConsoleIOContext.<init>(ConsoleIOContext.java:70)
at jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:242)
at jdk.internal.jshell.tool.JShellTool.main(JShellTool.java:233)
#2 add jline
$ <JDK9_HOME>/bin/java
-Xbootclasspath/p:<KULLA_JDK_BUILD>/jdk/modules/jdk.jshell:<KULLA_JDK_BUILD>/jdk/modules/jdk.compiler:<KULLA_JDK_BUILD>/jdk/modules/java.compiler:<KULLA_JDK_BUILD>/jdk/modules/*jdk.internal.le*jdk.internal.jshell.tool.JShellTool
Exception in thread "main" java.lang.NoClassDefFoundError:
com/sun/jdi/Bootstrap
at jdk.jshell.JDIConnection.findConnector(JDIConnection.java:77)
at jdk.jshell.JDIConnection.<init>(JDIConnection.java:312)
at jdk.jshell.JDIEnv.init(JDIEnv.java:45)
at jdk.jshell.ExecutionControl.jdiGo(ExecutionControl.java:251)
at jdk.jshell.ExecutionControl.launch(ExecutionControl.java:66)
at jdk.jshell.JShell.executionControl(JShell.java:601)
at jdk.jshell.Eval.load(Eval.java:729)
at jdk.jshell.Eval.declare(Eval.java:591)
at jdk.jshell.Eval.declare(Eval.java:494)
at jdk.jshell.Eval.processMethod(Eval.java:393)
at jdk.jshell.Eval.eval(Eval.java:137)
at jdk.jshell.JShell.eval(JShell.java:349)
at
jdk.internal.jshell.tool.JShellTool.processCompleteSource(JShellTool.java:1360)
at
jdk.internal.jshell.tool.JShellTool.processSource(JShellTool.java:1349)
at
jdk.internal.jshell.tool.JShellTool.processSourceCatchingReset(JShellTool.java:479)
at jdk.internal.jshell.tool.JShellTool.run(JShellTool.java:465)
at jdk.internal.jshell.tool.JShellTool.resetState(JShellTool.java:399)
at jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:248)
at jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:243)
at jdk.internal.jshell.tool.JShellTool.main(JShellTool.java:233)
#3 add JDI
$ <JDK9_HOME>/bin/java
-Xbootclasspath/p:<KULLA_JDK_BUILD>/jdk/modules/jdk.jshell:<KULLA_JDK_BUILD>/jdk/modules/jdk.compiler:<KULLA_JDK_BUILD>/jdk/modules/java.compiler:<KULLA_JDK_BUILD>/jdk/modules/jdk.internal.le:<KULLA_JDK_BUILD>/jdk/modules/*jdk.jdi*jdk.internal.jshell.tool.JShellTool
Error: Could not find or load main class
jdk.internal.jshell.remote.RemoteAgent
| State engine terminated. See /history
#4 Give up :-)
My brief understanding of what goes wrong:
#1 fails because you need jline. But wait - jline is already in the JDK
I used to run jshell - so what? There seems to be an issue with jimage:
if some bootstrap mentions 'jdk.internal', all contents of
'jdk.internal' coming from the jimage are nuked out - in other words,
adding jshell to the bootclasspath prevents jline to be discovered.
Uuugh. Not sure if this will go away with new modularized options.
#2 fails because JDI is not part of rt.jar (or its new equivalent). So
that must be manually added. Ugh. Means we need a full build available
somewhere. Ok.
#3 fails because, when jshell is forking the agent VM, it propagates the
classpath options but not the bootclasspath options. This means that the
new agent vm won't see the jshell package. Again, not sure if this is
fixable or not.
Because of the combination of #1, #2 and #3 it is impossible to
(reliably) run jshell on top of an existing (and recent) JDK 9 binary.
Note that the failures above are made more explicit by the fact that
jshell is not in the binary snapshot used for launching it - if we were
using a jshell-enabled JDK the failure would be much more subtle - as
jshell would start, but the tool and the agent would be seeing two
different versions of the jshell module - with a lot of potential for
troubles.
Running tests is basically more of the same.
If we can fix #1, #2 and #3 (or show a path that leads us there), I
agree that the decision of where to put jshell is largely irrelevant.
But if any of these issues is there to stay, then I don't think having
jshell in langtools is really buying anything: jshell will end up being
a corner case of langtools that you need to build/run/test specially -
meaning most developers won't even bother with it.
Maurizio
On 17/09/15 09:17, Maurizio Cimadamore wrote:
>
>
> On 17/09/15 02:56, Jonathan Gibbons wrote:
>> I guess I don't understand why testing won't work as expected,
>> provided you have a sufficiently recent snapshot of JDK to use as a
>> baseline. In this case, the requirement is that the JDK baseline
>> has jline and whatever other recent API has been added to JDK.
> There are many factors that contribute to this - the main one being
> JShell too not passing bootstrapping setting on the remote agent VM,
> plus some weird bug I encountered where if I set bootclasspath that
> mentions some jdk.internal classes (but which doesn't have JLine -
> because it comes from langtools), _every_ class from that package will
> not be fetched from the jimage file - meaning that I'll be left w/o
> jline. In other words, bootstrapping doesn't work - and I have reasons
> to believe that bootstrapping with the new module options (when they
> will become available - i.e. moduleOverride) won't work too.
>
> Maurizio
>>
>> If kulla is routinely adding and modifying API in the jdk repo, then
>> I agree that would tend to suggest that the source bits should be in
>> the jdk repo as well. But I would have thought that the kulla
>> additions to the jdk repo will stabilize very quickly, and so after
>> one or two builds and promotions, if JShell ends up in langtools,
>> then it will be easy enough to run all the tests using a recent build
>> of JDK.
>>
>> I agree that we should not expect langtools developers to have to use
>> "make test" and I agree that we should not expect langtools
>> developers to exclude the JShell tests. That implies we should look
>> at and understand why the JShell tests might fail if they are
>> co-located in the langtools directory. If it is just because jline
>> is not yet standard in JDK builds, that's a temporary state of
>> affairs that we will get past soon enough. If there's something else
>> going on, I guess I'd like to better understand what that might be,
>> so that we can see if we can fix the issue.
>>
>> -- Jon
>>
>> On 09/16/2015 05:38 PM, Maurizio Cimadamore wrote:
>>> While I agree with many points you raise - I feel this discussion is
>>> very abstract; please try to run all langtools tests in the kulla
>>> langtools repo using your standard scripts - does everything still
>>> work for you? I found that I could not run tests using my usual
>>> scripts. The only way to run jshell tests reliably is through make.
>>> Now, if we were in a world where 100% langtools developers ran tests
>>> that way - I would agree with you; but the reality is somewhat
>>> different, and many developers are used to run langtools tests using
>>> some JDK N snapshot and bootstrapping the required classes. That
>>> technique WILL cease to work here. I don't think we can say to those
>>> developer - just don't run kulla tests and be happy (what if some of
>>> their changes accidentally broke something in kulla?). Are you
>>> saying that those developers should just stop doing what they are
>>> doing and start running langtools tests as part of a full JDK build
>>> and test cycle? That's a fine answer - I guess I'm just trying to
>>> guess where you are coming from.
>>>
>>> Maurizio
>>>
>>> On 17/09/15 01:04, Jonathan Gibbons wrote:
>>>> I didn't say I think JShell should be in langtools, and I thought I
>>>> was being careful to be very explicit about that. I was just trying
>>>> to make sure we were making the right choice for the right reasons.
>>>>
>>>> The fact that it depends on JDI and jline is IMO a red herring. If
>>>> they were both already in JDK 8 (i.e. the boot JDK) would we be
>>>> having this discussion?
>>>>
>>>> To me, the bottom line is that I don't think it matters where the
>>>> code lives, and as a result I personally don't care and have no
>>>> opinion where the code should live. I /do/ think that whereever we
>>>> choose to put it, we can make everything work well from a developer
>>>> standpoint, without inconveniencing any existing developer usage,
>>>> although I concede that we might choose not to integrate it with
>>>> the langtools make/build.xml infrastructure and/or the existing
>>>> NetBeans project. And from a somewhat selfish standpoint, I think
>>>> we have somewhat more latitude to set things up the way we like it
>>>> in the langtools repo than we would in the jdk repo.
>>>>
>>>> I agree that JShell is higher up in the layers of abstraction than
>>>> javac, but I also don't think that the langtools repo is defined to
>>>> be "core tools to be able to work with the JDK". The provenance of
>>>> the repo (as indicated by its name) is that it was code that was
>>>> managed by the old Sun Language Tools Group, from way back when,
>>>> meaning that it was more of a "team" repository than a repository
>>>> defined by a layer of abstraction.
>>>>
>>>> So, for me, I think this comes down to, "where does the Kulla dev
>>>> team think it would be most convenient for them to keep and work on
>>>> the code?". Once we can answer that, we can move on to, "what is
>>>> the most convenient way to satisfy all the requirements of all the
>>>> interested parties?"
>>>>
>>>> -- Jon
>>>>
>>>>
>>>> On 09/16/2015 04:02 PM, Maurizio Cimadamore wrote:
>>>>> Jon, let's reverse the question - why do you think it *should*
>>>>> live in langtools? Honestly it feels very different from all other
>>>>> tools that live there; there's dependencies on JDI, jline - so to
>>>>> me it feels closer to other tools (i.e. jconsole) which already
>>>>> are in the jdk repo. I guess my argument is - if langtools defines
>>>>> the core tools to be able to work with the JDK, jshell feels like
>>>>> one ore two layer of abstractions on top of that.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 16/09/15 23:31, Jonathan Gibbons wrote:
>>>>>>
>>>>>>
>>>>>> On 09/16/2015 03:06 PM, Robert Field wrote:
>>>>>>> As part of the JShell code review, Maurizio made the case that
>>>>>>> JShell should be in the JDK repo rather than langtools repo
>>>>>>> (where it is in the Kulla workspace). We would like additional
>>>>>>> feedback on this before proceeding --
>>>>>>>
>>>>>>>> On Sep 11, 2015, at 8:25 AM, Maurizio Cimadamore
>>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>>
>>>>>>>> One general high-level comment, which I also pointed out
>>>>>>>> elsewhere, is that I'm not sure jshell really belongs in
>>>>>>>> langtools; while it's semantically (obviously) related to
>>>>>>>> langtools - it is a rather different beasts w.r.t. all other
>>>>>>>> tools in langtools-land; the fact that it depends on the JDK
>>>>>>>> (for jline, and for JDI in general) makes it very hard to run
>>>>>>>> on top of a random JDK and then bootstrapping classes - which
>>>>>>>> is a technique widely used to be able to run langtools tools
>>>>>>>> w/o having to do a full build cycle. More specifically, talking
>>>>>>>> about IDE integration, I don't see how IntelliJ/Netbeans
>>>>>>>> langtools projects (and the langtools internal developer ant
>>>>>>>> build) could be updated to be able to run/debug jshell w/o a
>>>>>>>> full build.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> In the Big Picture view of the universe, folk would like to
>>>>>> restructure the repos in an OPenJDK forest, at which point the
>>>>>> distinction between the langtools and jdk repos will likely
>>>>>> disappear, or at least become very (very) blurred. While that
>>>>>> change is not imminent, IMO opinion, it significantly reduces the
>>>>>> impact of the reasons why JShell should not be in langtools.
>>>>>>
>>>>>> I also don't entirely agree with the IDE reasons either. If you
>>>>>> want to work on JShell in an IDE, you're going to have to do what
>>>>>> it takes to get rid of the red squiggly lines anyway, whatever
>>>>>> repo the code is in. To keep JShell out of Langtools because it
>>>>>> doesn't fit our current methodologies is a case of the tail
>>>>>> wagging the dog.
>>>>>>
>>>>>> Also, I note that JDK 9 now builds the product module by module.
>>>>>> In times past, we built the world repo by repo, and that would
>>>>>> indeed have been good reason to keep JShell out of langtools,
>>>>>> because of the dependencies of JShell on JDK 9 API, like jline.
>>>>>> But now, the build will build "interim javac", and can then build
>>>>>> JDK modules like java.base and whatever module jline ends up in,
>>>>>> and can then build the module containing JShell.
>>>>>>
>>>>>> As to how to organize IDE projects, in the worst case, we keep
>>>>>> the langtools IDE setup as it is now (no JShell) and have a
>>>>>> separate project for JShell itself. That is tantamount to what we
>>>>>> would do if we push JShell into the jdk repo.
>>>>>>
>>>>>> So, I'm saying all that while trying to be agnostic on which repo
>>>>>> it should live in. I don't really agree with the stated
>>>>>> reasons why it should not live in langtools, which means that we
>>>>>> can look for other reasons to choose one repo or another.
>>>>>>
>>>>>> -- Jon
>>>>>
>>>>
>>>
>>
>
More information about the kulla-dev
mailing list