Looking ahead: proposed Hg forest consolidation for JDK 10

[Jonathan Gibbons]

On 10/12/2016 09:25 AM, Andrew Hughes wrote:
>
> ----- Original Message -----
>> snip...
>>
>>>>> I would appreciate if the runner could be included in the
>>>>> root/test directory.
>>>> I'm not quite sure what you are referring to by the jtreg runner.
>>> I mean the code in http://hg.openjdk.java.net/code-tools/jtreg
>>>
>> I think the problem with that is that the development of jtreg then becomes
>> tied to OpenJDK. It might be ok for the development version of OpenJDK, but
>> you then have to backport jtreg fixes into update releases, rather than just
>> using the same jtreg from the separate repository.
>>
>> We've experienced that bundling issue with IcedTea, and it's why we split
>> out the plugin/javaws work (IcedTea-Web).
>>
>>> As Andrew stated, some subdirectories are pretty stable. It
>>> might completely make sense to merge these into one repository, but I'm
>>> really concerned about jdk and hotspot.
>>>
>>> In general, I think those people that are highly specialized on complex
>>> subcomponents of the VM will suffer from this.  They often are fine
>>> just working with hotspot / jdk etc..  In general, these people develop
>>> new components in the latest branch.
>>> Those people that have to maintain and test the VM really will profit
>>> from the new setup.  They anyways always operate with the full
>>> repo tree.
>>> Having this said, I think it would make more sense to put the legacy code
>>> base into merged repos, and not the development branch?
>>>
>> I agree the biggest friction is going to be between HotSpot and the other
>> repositories, much as it was for the build system changes which took an
>> entire OpenJDK release to make it to HotSpot, because the development
>> experience is quite different.
>>
>> The HotSpot repository is still fairly slim and performs well. I get the
>> impression that many HotSpot developers work on that repository in isolation,
>> given their requirements to be able to build from within the repo rather than
>> the top-level.
>>
>> The JDK repository is already experiencing slowdown. It doesn't really work
>> without the top-level repository and the additional components (CORBA, JAXP,
>> JAXWS, Nashorn) are usually needed for a build, even if they are seldom
>> altered.
>> Given the size of the JDK repository already, adding in these components is
>> not going to make a huge difference.
>>
>> Build changes usually end up touching most of the repositories. Merges
>> certainly
>> do. So, yes, the benefits are greatest for those doing build and integration
>> work.
>>
>> I do wonder what percentage of the cross-repo commits touch HotSpot, and
>> whether
>> we could retain that as a separate repository if it's going to cause so much
>> disruption for HS developers.
>>
>> JDK builds can be done using an imported HotSpot and HotSpot developers can
>> use an
>> imported JDK, so I don't see a strong reason to join these together. We've
>> had to
>> swap out HotSpot with one that includes a newer port on several occasions,
>> and
>> having to duplicate all the JDK code in these cases would be a major pain.
>>
>> HotSpot also operates a different commit process which could cause friction
>> if
>> the repos are merged.
>>
> Further to that, for OpenJDK 8, the relative repo sizes look like
> this (compressed):
>
> -rw-r--r-- 1 andrew users 918K Aug  7 18:20 corba.tar.xz
> -rw-r--r-- 1 andrew users 6.5M Aug  7 18:22 hotspot.tar.xz
> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:21 jaxp.tar.xz
> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:21 jaxws.tar.xz
> -rw-r--r-- 1 andrew users  38M Aug  7 18:23 jdk.tar.xz
> -rw-r--r-- 1 andrew users 2.0M Aug  7 18:21 langtools.tar.xz
> -rw-r--r-- 1 andrew users 2.2M Aug  7 18:25 nashorn.tar.xz
> -rw-r--r-- 1 andrew users 327K Aug  7 18:20 openjdk.tar.xz
>
> The JDK repository, even compressed, is over five times the size
> of HotSpot. Adding the other repos into the JDK repository thus
> wouldn't make that much of a difference to it, even if HotSpot is
> included, whereas it will cause an order of magnitude increase compared
> to the current side of the HotSpot repositories.
>
> I think I'd thus prefer to see it cut down to two repositories. That
> would give most of the benefits I described of getting rid of all
> the superfluous repos, without bloating the requirements for HotSpot work.


For my part, I always preferred a different grouping of repositories, 
along more
functional lines, starting with

*    hotspot + java.base module + javac
*    client/desktop/gui code
*    XML code
*    nashorn

But while I believe that would solve many/most of the problems for 
"coordinated" updates
across hotspot/core-libs/javac, I have to accept it will not solve "the 
bisect problem",
and so I have come to accept the desirability of a single repo.

That being said, I do see practical concerns here about working with a 
single repo, and
want to make sure we address all those concerns with reasonable, 
demonstrable
solutions.

-- Jon