Looking ahead: proposed Hg forest consolidation for JDK 10

[Joseph D. Darcy]

Hi Andrew,

On 10/11/2016 12:22 PM, Andrew Hughes wrote:
>
> ----- Original Message -----

[snip]

>>
>>
> As someone who regularly works with the whole set of repositories,
> and has experienced the pain that was the forest extension back in the
> day, I wholeheartedly welcome this change. For me, having to remember
> to check out or update all those separate trees is just additional pain.
> With the addition of yet another repo in OpenJDK 8 (nashorn), I thought
> things were going in the opposite direction, so this comes as a welcome
> surprise. While I can see the benefits of having HotSpot, langtools and
> jdk in separate repos - they are largely separate pieces with separate
> groups working on them - the reasoning to have JAXP, JAXWS and CORBA
> separate always seemed bizarre to me.
>
> I think this will also lower the headache for newcomers in getting hold
> of OpenJDK. Having to download all these separate repositories is unusual
> and confusing. Now if we only get rid of having both 7 and 7u, 8 and 8u, etc.
> as well...
>
> I'm curious as to how much the directory structure will change.

The JEP describes this in more detail; copying the description from there:

"In the prototype. the eight repositories have been combined into a 
single repository using an automated conversion script that preserves 
history on a per file level with the consolidated forest being 
synchronized at the tags used to mark JDK promotions. The changeset 
comments and creation date are also preserved. The prototype has another 
level of code reorganization. In the consolidated forests, code for Java 
modules is generally combined under a single top-level src directory. 
For example, today in the OpenJDK forest there are module-based 
directories like

$ROOT/jdk/src/java.base
...
$ROOT/langtools/src/java.compiler
...

In the consolidated OpenJDK forest, this code is instead organized as

$ROOT/src/java.base
$ROOT/src/java.compiler
...

An analogous but less aggressive reorganization is planned (but not yet 
fully implemented) for the test directories to go from

$ROOT/jdk/test/Foo.java
$ROOT/langtools/test/Bar.java

to

$ROOT/test/jdk/Foo.java
$ROOT/tests/langtools/Bar.java

Since the effort is currently a prototype, not all portions of it are 
entirely complete and the fit and finish can be improved in some areas. 
For example, there are still some files left in vestigial $ROOT/jdk, 
$ROOT/langtools, etc. directories, but these stray files should be 
relocated as part of follow-up work. The HotSpot C/C++ sources are moved 
to the shared src directory alongside the modularized Java code.

Supporting updates to the jtreg configuration files are in progress 
(JDK-8165187)."


>   The least
> disruptive would be to just have hotspot, jaxws, jaxp, jdk, langtools, corba
> and nashorn becomes regular subdirectories in the root repository, as this
> wouldn't require any build changes. However, the bug suggests that the tree
> structures will be merged. I hope this won't be as disruptive as the changes
> in 9, and will instead largely keep the same structure, but merged together.
> With 9, I find myself having to use find to locate where files have moved to,
> and I haven't really seen any gain from that change.
>
> I do think it's very important to retain as much history as possible. I appreciate
> that the changeset IDs will change - that happens in backporting too - but
> being able to trace individual changes to bug IDs is incredibly important.

Yes, all the history has been preserved, including the changeset 
comments with the bug ids.

>
> One concern I do share with Goetz is the speed of the resulting repository.
> The JDK repository is already pretty slow in operation.
>
> Is there an example converted repository available? I think this would answer
> a lot of questions.

We don't have one publicly available yet, but plan to do so. I'll make 
an announcement when it is available.

Thanks,

-Joe