JShell: source in langtools vs JDK?

Thu Sep 17 02:08:21 UTC 2015

Yes, I generally agree with your comments that testing is a good 
perspective from which to decide where to put the jshell bits, and I 
agree with your comments that because jshell is tightly integrated with 
javac, that would indicate that langtools is the preferable home.

But I also want to better understand Maurizio's comments as to whether 
and why he thinks there may be long term testing issues, so that we can 
work to address those issues.

I'm not so fussed about any short term inconvenience for a couple of 
builds after kulla is integrated into jdk9/dev.

-- Jon

On 09/16/2015 07:03 PM, Robert Field wrote:
> See the email I just sent speaking to some of this.
>
> As to JShell causing changes in the JDK repo, that would be rare. JDI 
> is existing, stable, and unchanged for JShell.  jline is a snapshot of 
> the external jline with a tiny tweak to integrate it cleanly. Only a 
> version update would likely change that.  In contrast, JShell has 
> required changes to javac code.
>
> -Robert
>
> On 09/16/15 18:56, Jonathan Gibbons wrote:
>> I guess I don't understand why testing won't work as expected, 
>> provided you have a sufficiently recent snapshot of JDK to use as a 
>> baseline.   In this case, the requirement is that the JDK baseline 
>> has jline and whatever other recent API has been added to JDK.
>>
>> If kulla is routinely adding and modifying API in the jdk repo, then 
>> I agree that would tend to suggest that the source bits should be in 
>> the jdk repo as well.   But I would have thought that the kulla 
>> additions to the jdk repo will stabilize very quickly, and so after 
>> one or two builds and promotions, if JShell ends up in langtools, 
>> then it will be easy enough to run all the tests using a recent build 
>> of JDK.
>>
>> I agree that we should not expect langtools developers to have to use 
>> "make test" and I agree that we should not expect langtools 
>> developers to exclude the JShell tests. That implies we should look 
>> at and understand why the JShell tests might fail if they are 
>> co-located in the langtools directory.  If it is just because jline 
>> is not yet standard in JDK builds, that's a temporary state of 
>> affairs that we will get past soon enough. If there's something else 
>> going on, I guess I'd like to better understand what that might be, 
>> so that we can see if we can fix the issue.
>>
>> -- Jon
>>
>> On 09/16/2015 05:38 PM, Maurizio Cimadamore wrote:
>>> While I agree with many points you raise - I feel this discussion is 
>>> very abstract; please try to run all langtools tests in the kulla 
>>> langtools repo using your standard scripts - does everything still 
>>> work for you? I found that I could not run tests using my usual 
>>> scripts. The only way to run jshell tests reliably is through make. 
>>> Now, if we were in a world where 100% langtools developers ran tests 
>>> that way - I would agree with you; but the reality is somewhat 
>>> different, and many developers are used to run langtools tests using 
>>> some JDK N snapshot and bootstrapping the required classes. That 
>>> technique WILL cease to work here. I don't think we can say to those 
>>> developer - just don't run kulla tests and be happy (what if some of 
>>> their changes accidentally broke something in kulla?). Are you 
>>> saying that those developers should just stop doing what they are 
>>> doing and start running langtools tests as part of a full JDK build 
>>> and test cycle? That's a fine answer - I guess I'm just trying to 
>>> guess where you are coming from.
>>>
>>> Maurizio
>>>
>>> On 17/09/15 01:04, Jonathan Gibbons wrote:
>>>> I didn't say I think JShell should be in langtools, and I thought I 
>>>> was being careful to be very explicit about that. I was just trying 
>>>> to make sure we were making the right choice for the right reasons.
>>>>
>>>> The fact that it depends on JDI and jline is IMO a red herring. If 
>>>> they were both already in JDK 8 (i.e. the boot JDK) would we be 
>>>> having this discussion?
>>>>
>>>> To me, the bottom line is that I don't think it matters where the 
>>>> code lives, and as a result I personally don't care and have no 
>>>> opinion where the code should live. I /do/ think that whereever we 
>>>> choose to put it, we can make everything work well from a developer 
>>>> standpoint, without inconveniencing any existing developer usage, 
>>>> although I concede that we might choose not to integrate it with 
>>>> the langtools make/build.xml infrastructure and/or the existing 
>>>> NetBeans project.  And from a somewhat selfish standpoint, I think 
>>>> we have somewhat more latitude to set things up the way we like it 
>>>> in the langtools repo than we would in the jdk repo.
>>>>
>>>> I agree that JShell is higher up in the layers of abstraction than 
>>>> javac, but I also don't think that the langtools repo is defined to 
>>>> be "core tools to be able to work with the JDK". The provenance of 
>>>> the repo (as indicated by its name) is that it was code that was 
>>>> managed by the old Sun Language Tools Group, from way back when, 
>>>> meaning that it was more of a "team" repository than a repository 
>>>> defined by a layer of abstraction.
>>>>
>>>> So, for me, I think this comes down to, "where does the Kulla dev 
>>>> team think it would be most convenient for them to keep and work on 
>>>> the code?". Once we can answer that, we can move on to, "what is 
>>>> the most convenient way to satisfy all the requirements of all the 
>>>> interested parties?"
>>>>
>>>> -- Jon
>>>>
>>>>
>>>> On 09/16/2015 04:02 PM, Maurizio Cimadamore wrote:
>>>>> Jon, let's reverse the question - why do you think it *should* 
>>>>> live in langtools? Honestly it feels very different from all other 
>>>>> tools that live there; there's dependencies on JDI, jline - so to 
>>>>> me it feels closer to other tools (i.e. jconsole) which already 
>>>>> are in the jdk repo. I guess my argument is - if langtools defines 
>>>>> the core tools to be able to work with the JDK, jshell feels like 
>>>>> one ore two layer of abstractions on top of that.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 16/09/15 23:31, Jonathan Gibbons wrote:
>>>>>>
>>>>>>
>>>>>> On 09/16/2015 03:06 PM, Robert Field wrote:
>>>>>>> As part of the JShell code review, Maurizio made the case that 
>>>>>>> JShell should be in the JDK repo rather than langtools repo 
>>>>>>> (where it is in the Kulla workspace).  We would like additional 
>>>>>>> feedback on this before proceeding --
>>>>>>>
>>>>>>>> On Sep 11, 2015, at 8:25 AM, Maurizio Cimadamore 
>>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>>
>>>>>>>> One general high-level comment, which I also pointed out 
>>>>>>>> elsewhere, is that I'm not sure jshell really belongs in 
>>>>>>>> langtools; while it's semantically (obviously) related to 
>>>>>>>> langtools - it is a rather different beasts w.r.t. all other 
>>>>>>>> tools in langtools-land; the fact that it depends on the JDK 
>>>>>>>> (for jline, and for JDI in general) makes it very hard to run 
>>>>>>>> on top of a random JDK and then bootstrapping classes - which 
>>>>>>>> is a technique widely used to be able to run langtools tools 
>>>>>>>> w/o having to do a full build cycle. More specifically, talking 
>>>>>>>> about IDE integration, I don't see how IntelliJ/Netbeans 
>>>>>>>> langtools projects (and the langtools internal developer ant 
>>>>>>>> build) could be updated to be able to run/debug jshell w/o a 
>>>>>>>> full build.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> In the Big Picture view of the universe, folk would like to 
>>>>>> restructure the repos in an OPenJDK forest, at which point the 
>>>>>> distinction between the langtools and jdk repos will likely 
>>>>>> disappear, or at least become very (very) blurred. While that 
>>>>>> change is not imminent, IMO opinion, it significantly reduces the 
>>>>>> impact of the reasons why JShell should not be in langtools.
>>>>>>
>>>>>> I also don't entirely agree with the IDE reasons either. If you 
>>>>>> want to work on JShell in an IDE, you're going to have to do what 
>>>>>> it takes to get rid of the red squiggly lines anyway, whatever 
>>>>>> repo the code is in. To keep JShell out of Langtools because it 
>>>>>> doesn't fit our current methodologies is a case of the tail 
>>>>>> wagging the dog.
>>>>>>
>>>>>> Also, I note that JDK 9 now builds the product module by module. 
>>>>>> In times past, we built the world repo by repo, and that would 
>>>>>> indeed have been good reason to keep JShell out of langtools, 
>>>>>> because of the dependencies of JShell on JDK 9 API, like jline. 
>>>>>> But now, the build will build "interim javac", and can then build 
>>>>>> JDK modules like java.base and whatever module jline ends up in, 
>>>>>> and can then build the module containing JShell.
>>>>>>
>>>>>> As to how to organize IDE projects, in the worst case, we keep 
>>>>>> the langtools IDE setup as it is now (no JShell) and have a 
>>>>>> separate project for JShell itself. That is tantamount to what we 
>>>>>> would do if we push JShell into the jdk repo.
>>>>>>
>>>>>> So, I'm saying all that while trying to be agnostic on which repo 
>>>>>> it should live in.    I don't really agree with the stated 
>>>>>> reasons why it should not live in langtools, which means that we 
>>>>>> can look for other reasons to choose one repo or another.
>>>>>>
>>>>>> -- Jon
>>>>>
>>>>
>>>
>>
>