Looking ahead: proposed Hg forest consolidation for JDK 10

[David Lloyd]

One small additional consideration: a consolidated repository makes it considerably easier to maintain a git mirror (something I do today via some fairly horrific scripts) or even move to git or another SCM in the future.
--
- DML


> On Oct 13, 2016, at 01:53, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> 
> 
>> On 10/12/2016 09:25 AM, Andrew Hughes wrote:
>> 
>> ----- Original Message -----
>>> snip...
>>> 
>>>>>> I would appreciate if the runner could be included in the
>>>>>> root/test directory.
>>>>> I'm not quite sure what you are referring to by the jtreg runner.
>>>> I mean the code in http://hg.openjdk.java.net/code-tools/jtreg
>>>> 
>>> I think the problem with that is that the development of jtreg then becomes
>>> tied to OpenJDK. It might be ok for the development version of OpenJDK, but
>>> you then have to backport jtreg fixes into update releases, rather than just
>>> using the same jtreg from the separate repository.
>>> 
>>> We've experienced that bundling issue with IcedTea, and it's why we split
>>> out the plugin/javaws work (IcedTea-Web).
>>> 
>>>> As Andrew stated, some subdirectories are pretty stable. It
>>>> might completely make sense to merge these into one repository, but I'm
>>>> really concerned about jdk and hotspot.
>>>> 
>>>> In general, I think those people that are highly specialized on complex
>>>> subcomponents of the VM will suffer from this.  They often are fine
>>>> just working with hotspot / jdk etc..  In general, these people develop
>>>> new components in the latest branch.
>>>> Those people that have to maintain and test the VM really will profit
>>>> from the new setup.  They anyways always operate with the full
>>>> repo tree.
>>>> Having this said, I think it would make more sense to put the legacy code
>>>> base into merged repos, and not the development branch?
>>>> 
>>> I agree the biggest friction is going to be between HotSpot and the other
>>> repositories, much as it was for the build system changes which took an
>>> entire OpenJDK release to make it to HotSpot, because the development
>>> experience is quite different.
>>> 
>>> The HotSpot repository is still fairly slim and performs well. I get the
>>> impression that many HotSpot developers work on that repository in isolation,
>>> given their requirements to be able to build from within the repo rather than
>>> the top-level.
>>> 
>>> The JDK repository is already experiencing slowdown. It doesn't really work
>>> without the top-level repository and the additional components (CORBA, JAXP,
>>> JAXWS, Nashorn) are usually needed for a build, even if they are seldom
>>> altered.
>>> Given the size of the JDK repository already, adding in these components is
>>> not going to make a huge difference.
>>> 
>>> Build changes usually end up touching most of the repositories. Merges
>>> certainly
>>> do. So, yes, the benefits are greatest for those doing build and integration
>>> work.
>>> 
>>> I do wonder what percentage of the cross-repo commits touch HotSpot, and
>>> whether
>>> we could retain that as a separate repository if it's going to cause so much
>>> disruption for HS developers.
>>> 
>>> JDK builds can be done using an imported HotSpot and HotSpot developers can
>>> use an
>>> imported JDK, so I don't see a strong reason to join these together. We've
>>> had to
>>> swap out HotSpot with one that includes a newer port on several occasions,
>>> and
>>> having to duplicate all the JDK code in these cases would be a major pain.
>>> 
>>> HotSpot also operates a different commit process which could cause friction
>>> if
>>> the repos are merged.
>>> 
>> Further to that, for OpenJDK 8, the relative repo sizes look like
>> this (compressed):
>> 
>> -rw-r--r-- 1 andrew users 918K Aug  7 18:20 corba.tar.xz
>> -rw-r--r-- 1 andrew users 6.5M Aug  7 18:22 hotspot.tar.xz
>> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:21 jaxp.tar.xz
>> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:21 jaxws.tar.xz
>> -rw-r--r-- 1 andrew users  38M Aug  7 18:23 jdk.tar.xz
>> -rw-r--r-- 1 andrew users 2.0M Aug  7 18:21 langtools.tar.xz
>> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:25 nashorn.tar.xz
>> -rw-r--r-- 1 andrew users 327K Aug  7 18:20 openjdk.tar.xz
>> 
>> The JDK repository, even compressed, is over five times the size
>> of HotSpot. Adding the other repos into the JDK repository thus
>> wouldn't make that much of a difference to it, even if HotSpot is
>> included, whereas it will cause an order of magnitude increase compared
>> to the current side of the HotSpot repositories.
>> 
>> I think I'd thus prefer to see it cut down to two repositories. That
>> would give most of the benefits I described of getting rid of all
>> the superfluous repos, without bloating the requirements for HotSpot work.
> 
> 
> For my part, I always preferred a different grouping of repositories, along more
> functional lines, starting with
> 
> *    hotspot + java.base module + javac
> *    client/desktop/gui code
> *    XML code
> *    nashorn
> 
> But while I believe that would solve many/most of the problems for "coordinated" updates
> across hotspot/core-libs/javac, I have to accept it will not solve "the bisect problem",
> and so I have come to accept the desirability of a single repo.
> 
> That being said, I do see practical concerns here about working with a single repo, and
> want to make sure we address all those concerns with reasonable, demonstrable
> solutions.
> 
> -- Jon