JShell: source in langtools vs JDK?

[Maurizio Cimadamore]

On 17/09/15 02:56, Jonathan Gibbons wrote:
> I guess I don't understand why testing won't work as expected, 
> provided you have a sufficiently recent snapshot of JDK to use as a 
> baseline.   In this case, the requirement is that the JDK baseline has 
> jline and whatever other recent API has been added to JDK.
There are many factors that contribute to this - the main one being 
JShell too not passing bootstrapping setting on the remote agent VM, 
plus some weird bug I encountered where if I set bootclasspath that 
mentions some jdk.internal classes (but which doesn't have JLine - 
because it comes from langtools), _every_ class from that package will 
not be fetched from the jimage file - meaning that I'll be left w/o 
jline. In other words, bootstrapping doesn't work - and I have reasons 
to believe that bootstrapping with the new module options (when they 
will become available - i.e. moduleOverride) won't work too.

Maurizio
>
> If kulla is routinely adding and modifying API in the jdk repo, then I 
> agree that would tend to suggest that the source bits should be in the 
> jdk repo as well.   But I would have thought that the kulla additions 
> to the jdk repo will stabilize very quickly, and so after one or two 
> builds and promotions, if JShell ends up in langtools, then it will be 
> easy enough to run all the tests using a recent build of JDK.
>
> I agree that we should not expect langtools developers to have to use 
> "make test" and I agree that we should not expect langtools developers 
> to exclude the JShell tests. That implies we should look at and 
> understand why the JShell tests might fail if they are co-located in 
> the langtools directory.  If it is just because jline is not yet 
> standard in JDK builds, that's a temporary state of affairs that we 
> will get past soon enough.  If there's something else going on, I 
> guess I'd like to better understand what that might be, so that we can 
> see if we can fix the issue.
>
> -- Jon
>
> On 09/16/2015 05:38 PM, Maurizio Cimadamore wrote:
>> While I agree with many points you raise - I feel this discussion is 
>> very abstract; please try to run all langtools tests in the kulla 
>> langtools repo using your standard scripts - does everything still 
>> work for you? I found that I could not run tests using my usual 
>> scripts. The only way to run jshell tests reliably is through make. 
>> Now, if we were in a world where 100% langtools developers ran tests 
>> that way - I would agree with you; but the reality is somewhat 
>> different, and many developers are used to run langtools tests using 
>> some JDK N snapshot and bootstrapping the required classes. That 
>> technique WILL cease to work here. I don't think we can say to those 
>> developer - just don't run kulla tests and be happy (what if some of 
>> their changes accidentally broke something in kulla?). Are you saying 
>> that those developers should just stop doing what they are doing and 
>> start running langtools tests as part of a full JDK build and test 
>> cycle? That's a fine answer - I guess I'm just trying to guess where 
>> you are coming from.
>>
>> Maurizio
>>
>> On 17/09/15 01:04, Jonathan Gibbons wrote:
>>> I didn't say I think JShell should be in langtools, and I thought I 
>>> was being careful to be very explicit about that. I was just trying 
>>> to make sure we were making the right choice for the right reasons.
>>>
>>> The fact that it depends on JDI and jline is IMO a red herring. If 
>>> they were both already in JDK 8 (i.e. the boot JDK) would we be 
>>> having this discussion?
>>>
>>> To me, the bottom line is that I don't think it matters where the 
>>> code lives, and as a result I personally don't care and have no 
>>> opinion where the code should live. I /do/ think that whereever we 
>>> choose to put it, we can make everything work well from a developer 
>>> standpoint, without inconveniencing any existing developer usage, 
>>> although I concede that we might choose not to integrate it with the 
>>> langtools make/build.xml infrastructure and/or the existing NetBeans 
>>> project.  And from a somewhat selfish standpoint, I think we have 
>>> somewhat more latitude to set things up the way we like it in the 
>>> langtools repo than we would in the jdk repo.
>>>
>>> I agree that JShell is higher up in the layers of abstraction than 
>>> javac, but I also don't think that the langtools repo is defined to 
>>> be "core tools to be able to work with the JDK". The provenance of 
>>> the repo (as indicated by its name) is that it was code that was 
>>> managed by the old Sun Language Tools Group, from way back when, 
>>> meaning that it was more of a "team" repository than a repository 
>>> defined by a layer of abstraction.
>>>
>>> So, for me, I think this comes down to, "where does the Kulla dev 
>>> team think it would be most convenient for them to keep and work on 
>>> the code?". Once we can answer that, we can move on to, "what is the 
>>> most convenient way to satisfy all the requirements of all the 
>>> interested parties?"
>>>
>>> -- Jon
>>>
>>>
>>> On 09/16/2015 04:02 PM, Maurizio Cimadamore wrote:
>>>> Jon, let's reverse the question - why do you think it *should* live 
>>>> in langtools? Honestly it feels very different from all other tools 
>>>> that live there; there's dependencies on JDI, jline - so to me it 
>>>> feels closer to other tools (i.e. jconsole) which already are in 
>>>> the jdk repo. I guess my argument is - if langtools defines the 
>>>> core tools to be able to work with the JDK, jshell feels like one 
>>>> ore two layer of abstractions on top of that.
>>>>
>>>> Maurizio
>>>>
>>>> On 16/09/15 23:31, Jonathan Gibbons wrote:
>>>>>
>>>>>
>>>>> On 09/16/2015 03:06 PM, Robert Field wrote:
>>>>>> As part of the JShell code review, Maurizio made the case that 
>>>>>> JShell should be in the JDK repo rather than langtools repo 
>>>>>> (where it is in the Kulla workspace).  We would like additional 
>>>>>> feedback on this before proceeding --
>>>>>>
>>>>>>> On Sep 11, 2015, at 8:25 AM, Maurizio Cimadamore 
>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>
>>>>>>> One general high-level comment, which I also pointed out 
>>>>>>> elsewhere, is that I'm not sure jshell really belongs in 
>>>>>>> langtools; while it's semantically (obviously) related to 
>>>>>>> langtools - it is a rather different beasts w.r.t. all other 
>>>>>>> tools in langtools-land; the fact that it depends on the JDK 
>>>>>>> (for jline, and for JDI in general) makes it very hard to run on 
>>>>>>> top of a random JDK and then bootstrapping classes - which is a 
>>>>>>> technique widely used to be able to run langtools tools w/o 
>>>>>>> having to do a full build cycle. More specifically, talking 
>>>>>>> about IDE integration, I don't see how IntelliJ/Netbeans 
>>>>>>> langtools projects (and the langtools internal developer ant 
>>>>>>> build) could be updated to be able to run/debug jshell w/o a 
>>>>>>> full build.
>>>>>>
>>>>>>
>>>>>
>>>>> In the Big Picture view of the universe, folk would like to 
>>>>> restructure the repos in an OPenJDK forest, at which point the 
>>>>> distinction between the langtools and jdk repos will likely 
>>>>> disappear, or at least become very (very) blurred. While that 
>>>>> change is not imminent, IMO opinion, it significantly reduces the 
>>>>> impact of the reasons why JShell should not be in langtools.
>>>>>
>>>>> I also don't entirely agree with the IDE reasons either. If you 
>>>>> want to work on JShell in an IDE, you're going to have to do what 
>>>>> it takes to get rid of the red squiggly lines anyway, whatever 
>>>>> repo the code is in. To keep JShell out of Langtools because it 
>>>>> doesn't fit our current methodologies is a case of the tail 
>>>>> wagging the dog.
>>>>>
>>>>> Also, I note that JDK 9 now builds the product module by module. 
>>>>> In times past, we built the world repo by repo, and that would 
>>>>> indeed have been good reason to keep JShell out of langtools, 
>>>>> because of the dependencies of JShell on JDK 9 API, like jline. 
>>>>> But now, the build will build "interim javac", and can then build 
>>>>> JDK modules like java.base and whatever module jline ends up in, 
>>>>> and can then build the module containing JShell.
>>>>>
>>>>> As to how to organize IDE projects, in the worst case, we keep the 
>>>>> langtools IDE setup as it is now (no JShell) and have a separate 
>>>>> project for JShell itself. That is tantamount to what we would do 
>>>>> if we push JShell into the jdk repo.
>>>>>
>>>>> So, I'm saying all that while trying to be agnostic on which repo 
>>>>> it should live in.    I don't really agree with the stated reasons 
>>>>> why it should not live in langtools, which means that we can look 
>>>>> for other reasons to choose one repo or another.
>>>>>
>>>>> -- Jon
>>>>
>>>
>>
>